As book grows, Scrivener slows

Hi,

I’ve finished a 100k word manuscript on Scrivener and I’m thankful the book wasn’t any longer because the Scrivener UI has been slowing down continuously to the point of really disturbing the flow of work. I have a 2 year old (Intel Duo Core) Macbook with 4 GB of memory running OS X 10.5.8.

I’ve read the suggestion that chapters should be written in separate projects, but this would kill the utility of Scrivener 50% IMO. Dragging around folders and files between chapters and being able to search the entire book in one place is essential.

The slow down mostly happens with cut/past and moving between files. Sometimes it takes 3-4 seconds when switching pages.

Just a note to encourage some optimization! Scrivener is awesome and I’d like to use it for future books.

Hi,

I spent more than a month recently optimising text editing for 2.0, but I’m not sure if what you are describing is related to that. Are you saying it slows down during typing (a known issue that I have been working on), or something else? I’m surprised you’re seeing these issues on Leopard, too, because most of the slowdown issues with typing are more apparent on Snow Leopard (though the underlying problem exists on Leopard too).

If you could give more details on the problem and how I can reproduce it, I’ll gladly look into it, as I do want to make Scrivener as fast as possible. Certainly you shouldn’t need to cut the project up into other projects, and I have never suggested that. I think you maybe thinking of a separate thread where someone was experiencing slow saving because they had imported 1,600 PDF files, which is excessive as Scrivener isn’t optimised as a database. It should, however, stay quick for projects with a couple of hundred research files and any amount of text files, so as much as you can tell me about your project, the sorts of documents it has in it, how many documents it has in it, and where the slowdown is happening (during saves, moving document moves or whatever) would be much appreciated.

A 100,000 word project should definitely not cause slowdowns!

Thanks and all the best,
Keith

Keith, I agree that 1600 pdf files is excessive, but how difficult would it be to build in a link to a separate folder with ‘open file in creator’. That way anybody working outside the normal guidelines would be able to transfer excess files to a separate folder. It could be provided by a menu item, 'insert/create link to…"

I appreciate this rather distorts the purpose of Scrivener, but it might prove useful for some people if it was done on the basis of it’s there if you need it, but you’re not forced to.

I’m partially thinking of myself as I use a lot of pictures of original documents stored as jpg files. I can degrade them to minimise the space they take up, but there are limits to the extent you can do this.

I don’t think that would make any difference. You can already drop the PDF files into the references if you want (although 16,000 would be difficult to trawl through in a flat list - mind you, it would also be difficult to trawl through so many in any structure), which does as you suggest.

The problem is that if you bring in 16,000 PDF files, presumably you are going to want to search within them. For that, Scrivener needs to build a search index. And the search index gets updated and re-saved every time you make changes to any document (as the search indexes have to be stored in one file). In the case of the user with all the PDF files, it was just that with so much text in the project, saving became slow because of the search indexes file being so big. I’m working on a couple of solutions to this for 2.0. One is to try to save those indexes in a separate thread. Another, which came to me as I started to write this reply, may be to save the search indexes for research files that cannot be edited in a separate file. But that becomes problematic because the user may edit these files in a different program…

All the best,
Keith

This may not be relevant since my research folder is fairly modest, but I’m sitting here with a script that is in excess of 100,000 words, divided into 37 chapters, in one project, on a Macbook Pro 2x2.3/3 GB RAM (max for this model), 10.4.11 – and I don’t experience any slowdown whatsoever.

Switching between chapters and between cork boards, etc. is almost instantaneous, less than 1/2 second. But this is on Tiger, which is still by far my preferred OS. :wink:

FWIW,

Joachim

Hi Keith,

Yes, I did see that post you mentioned and misinterpreted the advice.

I’ll give you an example of a slowdown. When I switch views from one file to another, it takes about 3 seconds and I see a CPU spike of ~60-70% under the Scrivener process during that time.

Another example is when I hit the “help pull-down”. It will take ~5 seconds to see the pull-down appear with a similar spike in process time.

Also, I have version 1.51. I haven’t made the last two upgrades because I hadn’t seen any optimization changes in the release notes and changing versions at this late stage of the book (when everything else is working fine) doesn’t sound like the best idea. Are there any changes that would impact this problem in 1.52 or 1.53?

Thanks,
Clint

Hi Clint,

No, there have been no optimisations between 1.51 and the later versions that would affect this (although I still recommend updating - it won’t hurt your files).

The CPU spike when you hit the Help menu is “normal”, or at least unavoidable - an Apple problem, unfortunately. Several of Scrivener’s menus are built dynamically - the Go To, Move To, Scrivener Link etc menus get built only when you click on them, because they are based on the current state of the binder. The Help menu in Leopard and Snow Leopard unfortunately builds every single menu before opening so that it can point you to the correct menu item, so it will have to cycle through the binder four or five times before it opens, which can take a while on large projects. I filed an enhancement request with Apple some time ago asking them to provide some way for developers to exclude certain menus from this process, but it didn’t make it into Snow Leopard, even though they replied that they would put it on the list. I have made some optimisations for 2.0 so that these menus should be built much faster, at least, though.

The Help menu spike should have nothing to do with the other stuff, though - the view changing. For that, could you please try the following?

  1. Find a situation in which you know that you can create a spike and slowdown.
  2. Open up Terminal and type in “Sample Scrivener 5 -file scrivenerslowdownsample.txt”
  3. Hit return and immediately switch to Scrivener and do something to cause a slowdown.

The above will sample what Scrivener is doing for five seconds and then write the file “scrivenerslowdownsample.txt” to your home folder. If you do that and then send me the file (support AT literatureandlatte DOT com), or post it here, then I can take a look to see what Scrivener is actually doing during that time. It would help if you did it two or three times, actually (change the file name slightly each time of course) so that I have more than one sample to go by.

I have spent a lot of time on optimisation for 2.0, so that it is faster in menu building, saving and typing, but if there’s anything more I can do then I’d like to do it, obviously.

Thanks and all the best,
Keith

Hi Keith,

Done and the file is sent to that address.

Thanks!
Clint