Do I need more computer power for Scrivener?

I’m using Scrivener 3.2.2 on a 2015 MacBook Pro 15” 2.8 GHz Intel Core i7 with 16 GB of RAM running Mojave 10.14, and I’m starting to get worried about performance aspects.

Currently I’m at roughly 100,000 words spread out over 19 chapters. When I select all the chapters to get them to display in one continuous flow, it takes Scrivener 30 seconds to paginate through the 470-odd pages. I can do searches while the counter is running up, but I wonder how compromised my system is if pagination takes a half minute to conclude. I’m used to see computer latency slow down pagination of Word documents that run 300 to 500 pages on my day job, and I’m a bit concerned about this entire scenario, especially since my target word count for this novel is 500,000 words. Not to mention that I often open this Scrivener file on a few different iPads.

Do I need to buy a faster MacBook Pro? Or start deleting pictures from this Scrivener file to shrink down the size of the file? (Currently it’s at 220 MB.)

do you really need to select all of the chapters at once?

So long as your computer and Scrivener operate acceptably once the Scrivenings session has loaded up all of the documents, there’s really nothing to worry about. It just has to open up and load every single file in your book, and then stitch together that many instances of the editor. I doubt it’s even using that much RAM (unless you have a lot of pictures embedded in your text documents).

You can use Activity Monitor to see how much Ram Scrivener has allocated to it, but if you have a hard drive with moving parts (i.e. not a solid state drive), then the slow-down may also be due to the speed of your drive; replacing it with an SSD would probably help (and will make your whole OS experience snappier).

Unlike with a traditional word processor which loads everything to RAM when you open the document, Scrivener does this on demand, so it may interrupt your flow a bit, but there’s no danger in that slowness unless your computer is starved for RAM. Everything will be snappier if you select fewer documents for Scrivenings sessions, or select a non-Scrivenings view mode when selecting the draft folder and use “Navigate->Editor->Lock Group View Mode”. That way, whenever you click on the Draft, it will display a non-Scrivenings view by default, even if you have been using that mode for other selections of documents and document containers.

Yes. How else would I search the entire book for a specific term and be able to “next” through all of them to get a count?

Perhaps there’s another way to do this, but I don’t know of one.

Very measured and detailed response. Thanks!

Ok. Thanks for clarifying that your purpose of selecting/displaying chapters into one continuous flow is to search, count, and then “next” through all. While this is alluded to in your top post, there were so many other things were thrown in I needed to ask you specifically “why” in order to help. I’m assuming, and thus might be incorrect, that what you call “one continuous flow” is what Scrivener calls “Scrivenings View”.

Try This: Before opening up everything into “one continuous flow”, click on the “Search” icon in the Toolbar or Menu → Find → Search in Project and search for that specific term of interest. In the Binder (if not shown turn on with Menu → View Binder) you should see “Search Results” a simple list of each chapter having that term, along with the count (that you seek) shown at the bottom of the Search Results Window. You can pick in the Binder’s Search Results list each chapter title to simply cycle through each “next” by mouse clicking the chapter title, or (my preference) select the first chapter title with a mouse click and then for subsequent “next” views press the down arrow key.

I don’t know the internals of the Scrivener architecture, but my hunch is that this way of doing the search you need (find, count, and cycle through “next”) might be more performant as it may avoid the performance hit you report using (assumption by me) using Scrivenings View of entire manuscript. I don’t have a large enough manuscript to test this. On my 3-year old iMac, this method seems very quick on a 300-pager I have available to me. Section II.11 (page 252) of the Scrivener User Manual covers search in more detail.

Is it better?

2 Likes

Yes, the method RMS describes should be faster. Scrivener builds a separate index document for exactly this sort of task. It searches the index, rather than having to load all the files individually, and then presents a collection showing only the relevant documents.

Once you’ve identified the relevant documents, you don’t need to count manually, though. Select the entire collection, then go to Project → Statistics, and choose the Selected Documents tab. Then open the Word Frequency list, and click the Word column to sort alphabetically (click again for reverse alphabetical order).

Yes, that is correct.

Wow, that is very cool. Nice. A list of all the chapters where the term appears. The only drawback to this approach is that I don’t see an easy way of click-searching through the whole mess. The advantage of running a search query in a Scrivenings view is that I can click-Find one-by-one through the whole thing. I can’t figure out how to do that (or if it’s even an option) in the Project Search view you’ve described. The “Next” and “Previous” buttons in the UI jump forward and backward chapters rather than instances of the search string. I looked over the Manual entry on this but couldn’t seem to find that functionality.

Well one thing to consider is that if you’ve narrowed down how much material you are working with to only the material that matters for what you’re looking for, then building a Scrivenings session out of the result list may no longer be as much of a burden on the system.

It’s worth a try anyway: at the top left of the search result header bar, click the hook arrow button to load the search results into the main editor, where you can work with them using any view mode, including Scrivenings.

As for the Next/Previous Document buttons you’ve found, yes they navigate between items rather than searching around inside of them (that would be really annoying if you just wanted to get to the next item quickly). But they can certainly be used in conjunction with Find’s own Next button. Once you get to the end of the current section, hit the Next Document shortcut and then continue.

So that’s another approach, if Scrivenings is still too much with the search results.

Due to a variety of computer needs, I may be purchasing an M1 MacBook Air, so the whole problem may soon be moot. There’s a rumor that the entire MacBook line is going to shift to a screen-lid build that has a notch (ack!) so I might pull the trigger and buy one of these machines over the weekend.

I’m worrying about this possibility since they removed the super convenient flip out coffee cup holder years ago. :scream:

2 Likes

Ha! And the new MacBooks run so cool that you can’t make waffles on the lid anymore. So sad.

2 Likes

I’m updating this topic, now that it’s about 2 years later. The MacBook Air M1 was fine for a while, but not anymore. My book has turned into a series, and I’m keeping everything in one file. So selecting a book (160,000 words) to search through it is a problem now. The indexing required takes 2 to 3 minutes to complete—and I must wait. For those who say “but it’s indexing in the background, so you can go ahead and start your search,” no, you can’t. The pointer goes crazy, flashing like a machine gun. You have to wait, and it’s annoying. Word, though it has a similar (and worse) flashing pointer software bug, will at least allow you to search a 900–1600 page document without delay or untoward UX issues.

Are you using the project search, or building a big Scrivenings session and searching through that? Yes, building a 160,000 word Scrivenings session might take some time.

To clarify, you must be referring to something other than reindexing the project search index, the one that comes up after syncing with iOS for example, right? Because that one can’t be put in the background last I checked (I’m not sure how useful that would be since modifying content during a full scan of said content sounds like a sure-fire way to have to start all over again once it’s done).

Otherwise, I think Katherine probably has the right answer, you’re not talking about indexing but building a Scrivenings session out of +160k words? That’s way outside of Scrivener’s efficiency zone. While we won’t stop you from doing it, there are intentionally built alternate pathways toward what you are trying to do. I don’t think I’ve ever deliberately loaded my entire book in a single session, and can think of no really good reason to do that.

At the very least, one could go chapter by chapter, which is hardly an imposition if your using Find Next over and over. Just click on the next thing in the binder to move on.

But myself, how I look for text throughout the whole work, is to start with Project Search and then load project search as Scrivenings and Find Next through that. This way at least I’m only loading the text that matters (items that contains the text I’m looking for rather than the 90% that don’t even qualify). And if even that takes too long and isn’t terribly useful, then I just combine F3 + Shift+Alt+DownArrow to jump to the next document, or on the Mac, ⌘G and Shift+Cmd+DownArrow.

P.S. Comparing Word with anything Scrivener is doing isn’t terribly relevant. The resources required to scan a single open file that is partially or entirely loaded in RAM is quite different from the resources required to load several thousand files at once. Try doing that in Word, and using its “Find Across All Open Documents” feature (if it has one) and tell me how it goes. I bet your computer crashes or seizes up before you even get 30 minutes into loading all of those files though. :wink:

It’s just one of those cases that highlights the differences in where you put your energy with optimisation, though. Scrivener puts its energy and code into handling huge quantities of files, not mammoth whole-novel-length files. Word necessarily puts its optimisation, energy and code into handling very long files (somewhat, you’ll find mixed reviews on how well it actually does that, in my experience). And it’s knowing things like that, that can help us use a more optimal approach in the software—like working with smaller chunks at a time in Scrivener instead of overloading the editor with 90% chaff for a search result.


Edit: I thought of one other thing. You mentioned that the laptop used to work better, and that’s maybe something to look into as well. SSDs get slower the more data they hold, particularly as they start to get full. If it’s pretty jammed up with stuff at this point, you could consider off-loading things you rarely need immediate access to, to an external drive. That might help.

SSDs do also get slower as they age, which can’t be avoided or fixed like in the old days with reformatting or defragmenting. It has to do with how the internal chemistry of it naturally degrades, and the hardware has to compensate by routing around broken storage cells. Modern SSDs are a lot better at this than they used to be, but it’s still an unavoidable problem, and one that unfortunately (as far as Macs are concerned) cannot be really be fixed. I wouldn’t suspect you’d be in the phase where that’s a major factor though. It can take years for such to become noticeable, though I suppose how heavily you use it can accelerate things.

1 Like

How heavily you use it, and how much free space there is. A good data storage algorithm will spread the load over the physical disk, rather than pounding on a particular group of bits, but obviously that’s harder to do if space is tight.