Sequential numbering between projects

I have a document that compiles to 1000 pages.

I use <$n> to make sequential numbering of each small part. There are thousands of parts.

Scrivener freezes on compile, likely due to the size of the output document.

If I break my project up into smaller projects, how can I have all the parts keep sequential numbering from project to project?

Any help appreciated.

Firstly, I would very much consider testing your theory before taking a step as drastic as that. Compile freezes are almost always a result of bad data somewhere in the project—typically a corrupted JPG somewhere, or an invisible character that the processor doesn’t like.

Best way to figure it out is to divide and conquer. The most efficient way to do this:

  1. Run a project search for * (single asterisk) with the Draft folder constraint, and otherwise default settings.
  2. Using the scroll bar as a guide, select from the first result to around halfway down the list, with Shift-click. Create a new Collection from that selection.
  3. In Compile, set the compile group dropdown to use this collection as its compile group.
  4. Compile with your standard settings.

If that works, then empty the collection and put the other half of the search results into it.

The idea then is to keep halving by whichever half causes the freeze. Thanks to how dividing by two works, you can narrow down even the most massive binders down to a few culprits very quickly. If you get multiple halves that freeze the compiler though, you’ll probably need to hunt down more than one problem.

Of course is both halves compile without freezing—then it might be worth considering a splitting up the compile job, but as you can probably see by now, having basically done that with collections—the notion of using different projects for such a task is entirely unnecessary. That doesn’t solve your numbering problem, but depending on what your output file type is, that may be better solved outside of Scrivener anyway.

For example if I were compiling to a word processing file I would want my numbering to come from the style I apply to my headings, rather than a number that is “manually typed in” (sure, Scrivener is automatically doing the typing, but a static number is still the result, and makes for a very rigid and difficult to edit file). That would solve your problem as .docx A and .docx B, once pasted together, would be numbered correctly by the stylesheet.

Thanks for this. Using it, I have found no errors. The freeze continues.

I have issues with my hands, so the least amount of effort is the best. If I can reduce the number of operations I make, and the time it takes, the longer I can write. Using the placeholder is 26000 operations and the least time.

I’m open to other options, however.

Thanks again for your efforts.

I’m not clear on how to proceed with this information, sorry. If the instructions were followed entirely, then my interpretation of what you are saying is that any single document freezes the compiler? Because by following them entirely you would have at some point run out off things to halve: ended up with one item left.

If that’s the case, then the problem probably has nothing to do with content, and we can look at it from that angle. I probably should have said that if every half, as they get smaller and smaller, freezes then you can probably stop trying to look for problems in the content. That’s indicative of a much broader problem. I suppose it is possible in theory that thousands of items in the binder have bad characters, but that’s less likely.

When you say you found no errors, what does that mean? What were you looking for, and how, since we didn’t really go into any of that, not knowing for sure if that was even the source of the problem.

I’m open to other options, however.

I did not go into any detail on the matter since we don’t even know if it is necessary, but I assure you, I would not suggest numbering 26,000 headings by hand! :sob: I’m talking about something that would involve one simple act of importing a compiled file into a template that is formatted with numbering.

l&l note

I’m in these circumstances now:

  1. Quit all other software and turn off wifi

  2. Use three styles to simplify the document, but more are required

  3. Use this: P<$n> to number items, where P is an indicator of a version of an item

  4. Apply styles and placeholders by hand as the items to number are within huge files as well as many small ones. Styles apply inconsistently when applied from drop-down after text being selected with cmd-a or Select Similar formatting or Select All Style from styles Panel to apply style en masse. Using layouts in compile may work, but when I try, it freezes before I can find out.

  5. Merge all files in a project into one, as many creates a freeze, or compile the largest part of the project only, or the whole thing, or just the second largest part, which all fail, but small groups work fine, but the numbering resets.

  6. Compile to rtf or pdf

  7. Watch computer to prevent sleep. Sleep creates freezes.

  8. Five minutes in, freeze. Leave it overnight, freeze, whether or not I stop sleep.

If I break this project up into much smaller projects, the numbering resets between them, but I do get usable output. Is there a way to increment P<$n>? The whole point of this is to have consecutive numbering.

I’ve been at this for years. No kidding.

Example of a doc’s formatting:

P<$n> Title style

text style 1

text style 2

Font is Baskerville 5pt

Before diving into complex maneuvers (splitting project etc, when you shouldn’t have to), did you try by any chance to simply create a new blank project and import the one that’s problematic in it (using File / Import / Scrivener project...) ?

Usually freezes at compile means a faulty document, you could look for the one using the 50% method. (I’ll explain below.)
But if your problem is something else, importing it in a blank project might overwrite it.

Searching for a faulty document using the 50% method :

(Set the compiler to “current selection”, make sure the filter is off, after which simply select what to compile in the binder.)

Compile the first half of your project. (Freeze ? Issue in this half.)
Compile the second half. (No freeze ? Good, you know for sure the issue is in the first half. Freeze ? Might have two or more issues, or something else completely.)

Compile the first half of the first half.
(Freeze ? Issue in this half of X. No freeze ? Issue in the other half of X.)

And so on and so on. 50% of 50% of 50% until you’ve got the issue (the document causing it) cornered.

(In case it is not clear the way I explained it, you always split in two halves whatever segment caused the freeze ; the other being most likely fine. – Unless you are unlucky enough to end up with an issue in both. – You could take notes so that you don’t have to redo the whole process, should this happen, or take the time to take nothing for granted, and test both halves, just to confirm as you go.)

I have created new projects and moved my data into them many, many times. The halving method shows no errors. I think my post shows I have tried a variety of ways to get things to work.

What seems left is size. I think my project is too large. There’s no pdfs or other file types, just the rtfs Scrivener uses.

That returns me to: Is it possible to increment the placeholders so I can reduce project sizes? That way I can have items 1-1000 in one project and 1001-2000 in the next, etc. I want to avoid this, and remain open to other choices.

Ok, but manually, or via import ?

How big is your project, more or less ? 6tb ? :stuck_out_tongue:

How many snapshots per documents, on average ? This is size you could potentially bring down. If you have many snapshots, you could do a backup, name it YourProjectSNAPSHOTS, then clear the snapshots from the project.
Or do a save as, renaming to YourProjectSNAPSHOTS, backup, close this and keep it in case you need to consult snapshots (removing all labels in the binder is a good way not to confuse the two, if you use labels’ color in the binder). Then clear the snapshots from the real project.

The advantage of the second option (with first a save as) is that you have the project ready to load (no need to unzip) and with a different name than the real project. (You still want to secure a zipped backup of that version with all snapshots.)

. . . . . .
If you have a lot of research, you can move that to a separate project if it is not part of the final compile. Then link to it in your mother project.
. . . . .

You could turn on line numbering for the time required and create a document that contains nothing but a thousand <$n>…
Have it as the topmost document of your project/compile.

Insert it once in your second project.
Twice in your third project. Etc etc

And ditch it out of the resulting compile, later.

There is no way that I know of to set a starting value for a numbering placeholder. (That’d be convenient. Wishlist material.)

. . . . . . . .
Perhaps you have sectors damaged on your HD (if not SSD) and unluckily, your project(s) are overlapping those ?
Or a defective RAM bracket ?
A badly fragmented drive ? (If not SSD.)
Your antivirus messing things up ? (You could try whitelisting Scrivener, if not already. – Although it’d be surprising that this would be a problem only with big projects rather than always.)

To reiterate sage advice from others, it’s unlikely your project is “too large”. My experience with “too large” projects only relates only to the time it takes to make a backup zip file on huge projects, which I’ve instructed Scrivener to do automatically on open and close. Yes, if that backup takes “too long” I break out stuff.

Scrivener can surely handle the size of project you report. Something going on inside one or more of the documents, or your documents are “too large”–Scrivener designed for reasonable size documents inside projects.

Curious… how big is your project on disk? Through windows explorer or Finder on Mac.

Could you be running out of disk space during the compile and that’s why it’s freezing?

Do you have access to another computer that you can try to compile the project on?

1 Like

Does Scrivener do any logging to Windows or Mac event logging or system logs? Maybe there’s an error message that could point you in the right direction.

I’ve started over. I split my files by halving them and only the parts compile. The only change was file size. Still, they don’t compile in any combination together and the placeholders now fail, where before they did not.

This project is a boat full of holes. I fix something and something else breaks.

I appreciate the help.

I’m moving on from Scrivener after years of use. It’s too much effort and time.

1 Like

On the Mac, Applications/Utilitites/Console.app is where most programs will be producing any debugging, warning or error information. For something like this though, it would most likely be more a matter of using Activity Monitor to watch for RAM and CPU usage to see if some kind of ceiling is getting hit. Plus that program can “sample” any running process to query what it is currently up to at the code level, producing a report similar to a crash log. While often not terribly helpful, it can sometimes give a broad idea of something being wrong, like the program being stuck in an infinite loop.

In this case it sounds more like a scale problem though. I haven’t seen issues like that since 64-bit came along, but who knows if the RTF file is hundreds of megabytes in size, and then it’s probably going to be a struggle to work with in any program you try to open it with, too.

2 Likes

I’ve been using Scrivener forever. I won’t give it up. I regret my tantrum.

In the end, reducing document size solved the issue.

There are now issues with styles getting applied, which, no doubt in my mind, will be solved the same way. Smaller seems to work better. Too many is also bad, so there is a limit on too small somewhere.

Exports to Word and Pages work fine. LibreOffice has style issues.

Thanks again.

2 Likes