I have been investigating a recurring Scrivener for macOS crash during Search Index reconstruction on a large project. The problem first appeared in 2021 and remains reproducible with Scrivener 3.5.2.
I have sent Literature & Latte a detailed technical report referring to my original support case #188968.
For users facing the same problem, this workaround repeatedly restored access to my project.
Before making any change inside the .scriv package, create a complete backup copy of the project and quit Scrivener.
Open the .scriv package and go to Files.
Remove user.lock.
Keep the existing search.indexes.
Reopen the project.
Scrivener then opens normally and uses the existing Search Index.
This indicates that the existing index remains usable while the failure occurs during its reconstruction.
I will update this thread after receiving Literature & Latte’s response.
This indicates that the existing index remains usable while the failure occurs during its reconstruction.
While this workaround may allow the project to open in this particular case, you should know that if the search index is never allowed to rebuild, it will grow increasingly out of date with the content of the project. This will not only impact searching, but icon states, tooltips, and even content areas like the corkboard. Basically we use that as a high-speed proxy for the actual data which would require opening and reading hundreds if not thousands of files otherwise. So that isn’t a good long-term solution. Finding the source of the problem will be your best bet.
The first thing to suspect, if search index rebuilding is crashing—and especially when couple with periodic crashing while using the software—is some form of damaged media in the text of the project. This is most often caused by an image placed into a text item that can’t be read properly, but more rarely some bizarre formatting glitch or unexpected control character can cause it, too. It could also be a PDF file, as the software will attempt to index any text content it can find in those, too.
When search indexing occurs, it prints the name of each item being processed. While this can go by quite quickly, there is usually a point when it crashes where you can see the last thing it was trying to index, and that is most likely going to be the host binder item for the problem.
If it is crashing at the same exact spot each time, then I’d start with that. First make a back up, as everything following will necessarily be a bit destructive.
With the project opened, close the split editor if applicable, and then lock the editor in place (Navigate ▸ Editor ▸ Lock in Place). This will allow you to select items in the binder without the software trying to load it from the disk, and potentially crashing.
Use Project Search in title searching mode to locate the item by name, that you identified from the search index rebuild crash.
Right-click, and select “Copy Document Link”, and then paste that into a plain-text editor (TextEdit is fine). You’ll get something that looks like this (except with Mac-style paths):
Back in Scrivener, use the File ▸ Show Project in Finder menu command, and then close the project.
In Finder, right-click on the project in the window the previous command opened, and “Show Package Contents”.
Navigate into the Files/Data subfolder, and then look for the folder that matches the random letters and numbers following the “id=” in that link.
Drag this folder in its entirety out of the project to some location, as a backup.
Reload the project, which should reload to search results with this item still selected, and the editor still locked.
Trash the item and then empty the project trash. This will ensure any snapshots (if it is text), are also removed from the project.
At this point, I would then attempt search index rebuilding, by holding down the Option key on the File menu, and selecting the “Save and Rebuild Search Indexes” command.
Keep a close watch on that, if it crashes again you’ll be repeating all of the above, but on the next item that halts it.
Once the project has reopened successfully, create a full Scrivener backup. Then use the project restored from that backup as the working project, and keep the recovered original untouched.
In my case, everything then appears to function normally in day-to-day use.
What remains unclear is what Scrivener is actually doing with the Search Index at that point. Is the restored project still using exactly the preserved search.indexes, does Scrivener modify or refresh it in some way during backup/restore or subsequent use, or does something else happen internally?
That is one point I would be interested in clarifying.
Basically the search index is loaded at project launch, and then it is from that point on operated on in memory. Whenever you change anything that might impact searching, such as adding a label, modifying a title or writing text, it will be updated in memory. When you close a project or run a manual save, the file is updated on the disk with these modifications. This is why, in a crash, the software will attempt to rebuild it because it knows it cannot be up to date at that point. The last disk copy will be from when the project was opened, typically, and not contain edits from the last session that crashed.
That is how it can get out of date, since it can only assume that it is up to date when loading the project, if it is already out of date it will work with out of date information throughout the session.
It’s not too surprising that you aren’t experience crashes all of the time, if it is indeed content related. You would have to visit the specific items that are causing problems. Compiling might crash, but it might not if the bad data is in a place that isn’t ordinarily compiled (document notes, synopsis, metadata). Crashing during search index rebuild is more common, because then it is trying to load everything containing text that you can search for, which is almost everything.
I sent Mac Support the full technical report on 23 August, referring to the follow-up to ticket #188968 from 2021.
It documents the controlled tests and the configuration-dependent behaviour in detail.
Please help ensure that Support reviews it and that it reaches the developer responsible for macOS Search Index reconstruction.
Yes, I have reviewed that, but I think it is important to stress that we cannot do anything about a content-related crash (which this almost certainly is), without said content also crashing on our systems in a debugging environment. There is no way to describe to us how to reproduce it without including the content that crashes in the description.
This is why I suggested the troubleshooting workflow above, to locate, remove and isolate the content causing the crash. We could take a look at that content and see if it something we can fix, but often in these cases it is impossible to fix.
Look at it this way, if you put a PDF into your project that has corrupted formatting instructions in it, and it causes Apple’s PDF viewing framework to crash, taking Scrivener down with it, then the only people that could potentially fix that are Apple—but even then it often isn’t something that can be fixed. Crashes are software’s way of saying: whoa, I’m getting really bad data here and need to stop before I mess up anything. It’s a fail-safe, not the problem in and of itself.
If the media is corrupted, that is the real problem, not software aborting its attempt to use it in the safest way possible.