Maximum size?

How large a project can Scrivener support? I’m working on a project that will eventually have 300,000 words’ worth of notes and a manuscript of about 100,000 words. So far I have only 80,000 words’ worth of notes and the program is briefly becoming nonresponsive. Thanks.

There’s no real limit. I once took the King James Bible and loaded it into the draft folder, splitting by book, and then by chapter; once the words were split up into “book” sized chunks, Scrivener was perfectly responsive unless I tried to load the whole thing into a “Scrivenings” session.

The unresponsiveness may have to do with how many words are being loaded into the editor at once. How do you have your text broken down? What is likely the largest (in number of words) individual file in your binder? Do you combine texts in scrivenings sessions a lot? What is the rough total of words in such a session?

I have a large and complex data tree as well, and will occasionally freeze up. Scrivener would benefit from a progress meter on Search All, which I surmise sometimes requires immediate reindexing. Also from some configurable protection against an overly heavy Scrivenings load request. But these require clicks that a user can learn to avoid.

There’s a needless lockup that can come from passing the cursor too slowly over Move To. This causes Scrivener to build a series of cascading menus to set up the destination. It’s a brief lock rather than a freeze, but too often inadvertent. Better if this function awaited a click, or were available in a nested menu, or were otherwise reengineered so that it would not tie up the program on a top-level hover.

As someone who works in long form (75,000 to 100,000 words), short form (2,000 to 10,000) and screenplay, I also wonder just how large a project can be. From what I have read, there appears to be no technical limit. But, before I go and divide my projects into three groups (i.e., three .scriv folders), I’d be interested in learning about any possible drawbacks.

For what it’s worth, my project is 230,000 words and Scrivener is having no problems with it. A few versions ago, they improved the speed, so even doing a backup only takes about 5 seconds, full compile in 10. Global searches are 1-2 seconds. I used to have about 50k worth of notes in the same project, though I’ve since moved those to a different program. It felt the same, speed-wise.

My project is divided into about 120 parts, so Scrivener doesn’t need to display more than a few thousand words at any given time.

For reference, I run it on a 4-year old laptop with a dual core pentium @ 2 ghz with 3gb ram. When writing, I use a power profile with 50% cpu (to keep the fan noise down).

MANY thanks for the response!

The indexing issue is a known bug, and yes that can cause things to seize up in a larger project right now. When it is functioning normally, the index is written to the disk and kept up to date in memory while you are working on the project. The bug is causing the software to periodically lose track of the index in memory, and rebuild the entire thing from scratch, rather than incrementally as you work, and for a large project that can take a few moments.

The idea is to fix that bug at the source, rather than build UI like progress bars around its presence. When it is functioning as it should, searching should always be lightning fast no matter how big the project is, and the disk should always have the most up to date version of the index when the software is loaded. Fully rebuilding the index should only be a troubleshooting step that is rarely done.

As rscullison points out, we have done a lot of optimisation in the past few versions, so if you haven’t updated recently you may wish to check your version number. Older versions of the software did become cripplingly slow with large projects on older equipment. These days that really shouldn’t be a problem, save for this issue described with the search index, and huge Scrivenings sessions, as Robert mentioned.

On the latter point, we do plan to implement seamless cancelling of Scrivenings view while it is building. Basically that means you’ll be able to just click elsewhere and move on, if you accidentally click on a folder that contains many tens of thousands of words. The only reason this doesn’t already exist is that “forking”, as this is technically referred to, can be tricky business to program for. It’s easy to make a huge mess when dealing with forks (data going off into nowhere and crashing the software, etc.), so it needs to be approached carefully.

That’s one I hope you’ll reconsider. I expect an index to get out of synch now and again because of abrupt termination, overly rapid copying and pasting, garbage collection, the overall pace of change in a project. Scrivener should mitigate the effects of an out-of-synch index, whatever the reason for it. I’m glad the team is working on the bug, but, to the user, the freeze is the bug.

You’ve made the great design choice of avoiding a single point of failure database in favor of folder-based documents, most of which are closed at any given moment. With its document tree as a table of contents, Scrivener works splendidly with a damaged search index. That is, until we search for something. Then we’re frozen out for a spell, just as if our project resided in a damaged data table.

It would be friendlier to repair the index in a background process so that the interactive program could remain responsive. And to think of it as a selling point that Scrivener can be used productively when the project index is unavailable.

All Best — Jerome

Sorry, I didn’t mean to insinuate that we have no interest in progress bars. There will be progress bars added to everything that may take more than a split second to accomplish. I was only saying here that the goal is to fix the bug with the search index first. That is more important than progress bars at this point in time.