Statistics - Just trying to make sense of something

What do you guys get as a reported wordcount in the “compiled” tab of the statistics panel ?

Or to be more precise, is the “compiled” statistics’ tab taking into account whether your documents inside the scope of the compile panel
image
are marked for inclusion or not.
Does it count everything, regardless of a document being excluded from compile ?

image

When
image
I get :
image

When
image
I get :
image

So, unlike specified in the manual, I do have to go back to the compile panel after compiling a single chapter…
(Unless I uncheck the “count current compile group only” option, but which option is the only way the “compiled” tab works - adding even more to my general confusion -, for me in some other projects.)

What exactly is the intended behavior here ?
And what use is the “compiled” tab of the statistics if it otherwise doesn’t reflect what will be compiled ?
(Shouldn’t it then be named something like “draft folder total content” instead, or simply discarded ?)

In all logic, what I’d expect it to do is to give me statistics for my draft folder, as will be compiled by the current compile format/compile settings, without having to leave my current binder selection – plain and simple. Thus allowing me to work on chapters, taking into account where I am at in the global scheme of my book’s statistics.

I am just trying to understand whether there is a bug here or not.
If there isn’t, then I have to say that this is very confusing, as I don’t see very much use to the “compiled” tab of the statistics… (Especially since we already get the “all included” result at the bottom of a draft folder scrivening.

image

What I did is made a collection of scenes in my WIP. Currently at 35000. As I write add to collection. To get the count use statistics ( everything marked to compile) comes up first. But if open collection you created and select all the files there And then click selected document tab get an accurate count of current count in novel

Yes, I understand, but having to select specific documents first, than using the “selected documents” tab, renders completely useless the “compiled” tab.
As I understand it by the manual, the idea is exactly to avoid the user having to change his selection first…
(In your case, having to go through making a collection just so that you get an accurate post-compile idea of your wordcount.)
Or for another user, having to select his draft folder first (turning it into a scrivening) and then to use the “selected documents” tab instead, where hovering the quick search menu in the toolbar already gives you a wordcount that takes inclusion/exclusion into account without any of the effort.

image

I marked a few documents to be excluded from compile, and :

image

So I am trying to figure out the purpose of that “Compiled” tab, and at the same time to figure out whether there is a bug here, affecting only a few people or what.

I noticed this feature when I changed my Compile Settings mid NaNoWriMo and got a Word Count of zero when updating it using Scrivener.

Scrivener will show the Compiled Word Count in the Compiled tab based on the setting in the Compile Overview window. You can Compile Draft, Folders, Subfolders and Collections other than Compiling a specific Binder Selection.

When you set the option to "Accurate(Slower), Scrivener will actually Compile before showing the resulting word count.

Of course, the Options Tab has settings that also have something to say about the results.

But in fact it ain’t. (At least not at my end.)
By all logic, if it was actually compiling, documents marked as excluded from compile wouldn’t be counted in.

What I mean is that although, yes, it is going through some processing first, whatever Scrivener is actually doing doesn’t go through the whole compile process – leaving out whether a document is to be compiled or not.

Have you checked the Options tab? Among others, it has settings for Counting Included or Not included documents.

When I create a Selection in the Binder and set the same Compile Group in the Compile Overview window, there’s actually a small difference between the two tabs that I don’t know where it’s coming from.

You mean this ? :

That only does what I showed at the beginning of my second post.
It makes the result be according to the compile’s scope.
image

Other than that, with that option unchecked, I just get the result for the whole draft (everything included).
(Note that I had it checked here because I was told to check it, and it seemingly fixed the “Compiled” tab for a smaller test project of mine. → Thus my growing conviction that there is actually a bug here.)
As for my real project, this option does nothing.

Else, the statistics selection options only affect the “Selected Documents” tab for me. Does nothing at all in the “Compiled” tab.

image

image
. . . . . . . . . . . . . . . . . .

image

image
. . . . . . . . . . . . . . . . . .

image

image
. . . . . . . . . . . . . . . . . .

image

image
. . . . . . . . . . . . . . . . . .

image

image
. . . . . . . . . . . . . . . . . .

image

image
. . . . . . . . . . . . . . . . . .

image

image

image

image

image

You’re right that the different options work for the different tabs. De headers group the options to Compiled and Selection Statistics respectively.
The “Count current compile group only” option obviously makes the settings in the Compile Overview window significant and count only would will be Compiled with these settings.

Yes, but to what avail as regard to the draft folder itself (or any other scope containing excluded from compile docs, for that matter), since it doesn’t actually simulate a compile ? We already have the info elsewhere ; at the bottom of a scrivening.

And here, from my smaller test project, where it somewhat seems to work properly :

image

image

image

image

image

(These numbers are actually accurate.)
So what to make of that ?

The confusion here is stemming from the compile settings. Going back to the first image of the compile window, to the right of the compile dropdown menu that has “Draft” selected is a funnel icon that is currently enabled (it’s blue in the screenshot), meaning that some of the filtering settings are on. If you click that, the top option there lets you choose what items are actually going to be included in the compiled document. The default setting is “Included Documents”, but you can choose instead to flip that and compile only the “Excluded Documents”, or—as it’s presumably set right now—“All” documents, i.e. compile them all regardless of their “include” checkbox setting.

I’m aware of a bug in the current 3.1.1 Windows Scrivener whereby this setting is not ultimately impacting the results of the compiled document; only the documents marked “Include in Compile” are being compiled. So it would be easy in this case to have changed the setting but forgotten about it since your compile results wouldn’t look any different. That bug will be fixed in the next release. Meanwhile, though, the setting is still (correctly) impacting the compiled word and character counts in the project Statistics, so that’s where the number is coming from.

3 Likes

That was it.

I seriously have no recollection of ever changing this filter setting (was indeed set to “all”), and as you said, never had an issue with compiling since…

That solves it.
Thank you.

But… P.S. So, in the end, we actually do need to have the “count current compile group only” option checked for the “compile” tab to take inclusions/exclusions into account.

image

image

. . . . . . . . . . . .

image

image

Meaning that the manual is wrong when stating that after compiling a single chapter, a user wouldn’t have to go back to the compile panel to have statistics about his/her whole project…

image

Perhaps it would be a better idea to instead have “count all” , “count only included documents” , “count only excluded documents” as checkbox options directly on the “Compile” tab … ? (Or in a section of the options tab – or to make them, as they are already there, not exclusive to the “selected documents” stats.)
( → Among other things, that would certainly be less confusing…)
And this way a user wouldn’t have to check “count current compile group only” from the option should he wish to have stats that take inclusions/exclusions into account for his draft folder, without having to go back to the compile settings, should he have the compile dialog set to a section of the project rather than the draft, just like says the manual it should be.
That would free the “Count current compile group only” option, allowing then a user to quickly compare the stats of the section he’s working on versus the whole draft (by having that other section set as the compile scope in the compile dialog, and checking unchecking that specific option in the statistics), without having to change the selection of the document(s) he is currently working on.

One would then easily get a perspective of his current document vs the current section vs the whole draft.

First a disclaimer: I speak here of ideals and intentions in the design, there may still be bugs here and there with the scope of what I’m saying. For instance:

But… P.S. So, in the end, we actually do need to have the “count current compile group only” option checked for the “compile” tab to take inclusions/exclusions into account.

Well, yes and no. That is a bug, likely related to the already mentioned bug, so in the end end, it should have no bearing. The correct behaviour is as described in the documentation for this checkbox. It only impacts the group selector at the top, not filter settings, not include/exclude, nor front/back matter, etc. We would word this setting differently were that the intention.

I haven’t gone through all of the text here, but to interject on the question of what the purpose is for the Compiled tab vs other counting mechanisms: simply put there are things that happen during compile that cannot be factored for in real-time counters, such as the editor footer bar with Scrivenings mode or the overall project goal bar. Some examples:

  • A Replacement might turn one word into three.
  • Section Layouts might insert headings or other material from metadata fields or Document Notes that add to the word count.
  • Likewise a Layout might only print a heading and exclude the main text, in the use case for folders by default, where text typed into a folder’s content area is treated as a form of inline “chapter notes” or what have you.
  • Styles may delete text that is in the editor (same goes for inline annotations).
  • Style prefix and suffix values will add characters or words.
  • There are many more, the point is suitably made.

The Selected Documents tab operates much more like the footer bar output in Scrivenings mode, though with additional options for excluding offline notes, annotations and such in the Options tab—mostly stuff that would cause significant typing lag if it were to be constantly running these filters. It also gives one a way to gather stats on stuff that will never compile, such as a large repository of world-building notes in the Research folder.

So overall the difference between Statistics and the real-time stats boils down to that point: real-time stuff must be super fast, since it recalculates with every single keypress. Statistics, being a panel you pull up, can be slower (much slower in large projects), but it will usually be more accurate.

That said, the Draft tab will not always be subjectively accurate, as compile settings can have a massive impact on things. Consider for example applying the “Enumerated Outline” compile format, which prints only headings and maybe synopses for each item. A 100k manuscript will drop down to a small fraction of that, so long as that format remains applied. Not ideal—but at the same time not really something the software can discern between what is intentional or not. We leave it up to you to decide when that is no longer relevant and switch Formats to something less radical. This gives you control you may need. You may want a word count on your outline output. You may want to create a special format that manipulates the word count a certain way.

To dive more deeply into accuracy, it will only account for source output, not subsequent or final output which results from post-processing. In the vast majority of cases the two will be one and the same. But there may be small variations with some formats, like ebooks, where Scrivener cannot and is not counting what is in the actual .epub file, but what goes into making it. Also consider with plain-text, Markdown and such workflows where custom post-compile automation is a possibility, vast changes may be incurred by tools that operate on Scrivener’s source output. One can essentially program the compiler, at a certain level of usage. The final output that is saved to the disk by the compiler is not what Scrivener uses for stats, in other words, nor could it be as it may well have no clue how to parse the final output usefully (like a .tex file).

And this way a user wouldn’t have to check “count current compile group only” from the option should he wish to have stats that take inclusions/exclusions into account for his draft folder, without having to go back to the compile settings, should he have the compile dialog set to a section of the project rather than the draft, just like says the manual it should be.

The manual states things in terms of the default behaviour, here. If you enable the option that only uses the compile group, then like many things in the manual that state default behaviour, the literal reading will be “incorrect”. An improvement I can see to this paragraph you’ve snipped is adding the words “by default” in there, which I’ve done.

As to further thoughts on whether we should start porting compile settings that impact word count into the Statistics options area, I don’t see the objection to having those factors driven by the compiler. It is not difficult to change a setting in the compiler, and as you can see from the list above, there are a lot of potential factors that can change the statistics. The main reason we have the compile group setting is given in the manual: in most cases one firing off a proof of chapter 18 won’t want their statistics impacted by that choice.

How could we justify stopping with just include/exclude checkbox, as you suggest? The governing logic here is that if something should not be counted as part of the compiled draft output, then you probably very much want that to be set in what actually produces the output, as a formal compile setting, rather than having a set of overrides that “fakes” their omission, leading one to confusion and surprise when they get a totally different result in the end.

For most everyone, what you get when you compile is what you want to have counted. Sure there might be edge cases, but one can end up with a nightmare of an interface if you cater to every edge case specifically. So to that end, we cater to them by proxy: need a weird workflow where you want Document Notes in the final output but don’t want them counted in statistics? You can do that: just have two compile Formats set up that are otherwise identical, and save the one that includes Notes for when you actually compile.

1 Like

I agree.

But if one can’t get a wordcount that takes into account inclusions/exclusions from compile for his draft without having that option (“count current compile group only”) checked and having his draft as being the compile group, that makes the manual wrong.

So, just to make things clear : are you saying that the “compiled” tab should normally display a wordcount of the draft → that takes inclusions/exclusions into account ← even when the “count current compile group only” is unchecked ?
(In other words : by default)

If yes : that makes sense (and just needs to be fixed).
If not, then the manual says it wrong, as one inevitably has to go back to the compile dialog and change the compile group to “draft” there.

I just can’t see why a user would 1) want to use the stats to get a wordcount that has nothing to do with the content he actually plans to compile,
and 2) consciously do so in the stats, since that very same (close to useless) info is already available much faster elsewhere.

It isn’t meant to work that way. If you put some notes into the Draft and mark the item as excluded, it should never be counted, unless you go into the Filter popup and tell the compiler to output everything (or only excluded for that matter, if you just want to dump out the notes)—but then we’re back into compile settings being the primary driver for compiled draft stats, as should be.

Now it is perfectly clear.
Thanks, AmberV.