Compile is S L O W

Hi everyone,

Lately it’s taking hours to Compile a 20,000 word docx - is this normal? I’m guessing not!
How do I fix this please!

For strictly the word count, that wouldn’t be normal. Are there any other things in the documents other than words? A lot of complex tables, large graphics, that sort of thing?

I would try compiling with RTF and see if that works faster. When you use the .docx choice it creates the RTF and then passes it along to a conversion engine that is rather slow itself. If you need it in .docx you can always save it that way from your word processor.

Thanks for replying, but RTF doesn’t work either.
In fact, a short time after the compile process starts, the “Compile” button actually becomes clickable again, and it’s possible to start the process again without it ever finishing in the first place!

Yes, I have a lot of images in there, most linked to files in folders…

If the Compile button becomes clickable again, that means that an exception (error) has been thrown in the background - a bug causing Compile to stop and not resume normally, in other words. Could you please try Compile again, and this time, as it compiles, take a look on the console (/Applications/Utilities/Console.app) to see what errors Scrivener outputs there. Also, if you could zip up the project and send it to support, that would be great, so that we can see the error for ourselves and locate the bug.

Thanks and all the best,
Keith

Hi Keith,

I’ve attached the Console output and will email the file and send to support.

Thanks

Darren
Scrivener error output from console.txt (7.53 KB)

Hi Darren,

The log shows that your system ran out of memory during the Compile. My guess is that you have some large images in the text. Scrivener’s .docx Compile has to run through a Java component, which can run out of memory on very large files.

One solution would be to compile to RTF and then open that in Word. Another would be to optimise the images in Scrivener. I’ll look out for your project in the support queue, though (I checked, and it doesn’t seem to be there yet).

All the best,
Keith

I’ve sent the file through. Check this Console output - pretty scary!

7/09/2015 11:32:04.000 am kernel[0]: process Scrivener[487] thread 7346 caught burning CPU! It used more than 50% CPU (Actual recent usage: 71%) over 180 seconds. thread lifetime cpu usage 90.081408 seconds, (87.558698 user, 2.522710 system) ledger info: balance: 90005861757 credit: 90007610807 debit: 1749050 limit: 90000000000 (50%) period: 180000000000 time since last refill (ns): 126702224384

Hi Keith,

My email server is down, hence the reply here.

The Images linked to file total only 62 MB

Darren

Just to jump in here, have you tried renaming your image folders so that the compiler cannot find them, and compile with no graphics? If it still hangs up, then we can at least eliminate images as the potential culprit. It could be a null character or some other gremlin in the text that is causing the hang.

Hi,

I tried your suggestion, renaming the image folder as to hide the linked images and Scrivener will compile a docx. However, re-linking the images sometimes results in a successful compile to docx, sometimes not.

I’ve attached some recent console output text files. I spotted the following line upon booting Scrivener and wonder if it is the culprit?

11/09/2015 11:38:33.896 am Scrivener[7556]: NSSoftLinking - The ShareKit framework’s library couldn’t be loaded from /System/Library/PrivateFrameworks/ShareKit.framework/Versions/A/ShareKit.

also this line:

11/09/2015 11:39:34.202 am Scrivener[7556]: GetDYLDEntryPointWithImage(/System/Library/PrivateFrameworks/CacheDelete.framework/CacheDelete,CacheDeleteCopyPurgeableSpaceWithInfo) failed.

This getting to be a major hindrance to my PhD work, I cannot send documents to my supervisor for editing. I think that Scrivener should be able to handle this print quality images, especially as I have 24GB RAM installed on a really fast machine.
Console Output - successful DocX compile with no images.txt (1.41 KB)

Argh - I could really use some help with this issue - really getting in my way now.

Darren

Sorry to bump but I really need some help with this…still waiting for a response from LL

Darren

I don’t actually recommend using print resolution graphics in Scrivener, especially in large quantities. The user manual itself has a caution about it—programs that can handle a full payload of print-ready graphics typically use techniques that are a good deal more sophisticated than what we have at our disposal to work with images in text.

It may be that in the end you’ll need to compile in chunks, stitching the files together in another program after they are all compiled, if you can manage to do so, or rely entirely upon thumbnail sized images until you get to the final polishing phase in whatever software you need to use for that.

In the meanwhile that latter option is what I would try to see if it alleviates the current issue enough to where you can output the work in progress reliably. It sounds like no images worked fine, so I bet small images will work fine too. Just duplicate the image folder and then in the one that Scrivener uses, downsize the images to the smallest scale you can work with for referencing and proofing. If you have Graphic Converter or Photoshop that should only take a few minutes. It should also improve the performance of the software while working in this project all around.

Hi,

This issue of very slow compilation is still an issue for me, even when there are very few images in my document.
one thing I notice when compiling is the Java app launches, hangs around for quite a while, then disappears. Sometimes I can make it disappear if I place the mouse cursor over it.

As I am heading towards submitting my thesis, this is getting really burdensome. To reiterate:

Images are linked to file.
Mac OS 10.10.5
24 GB 1600 MHz DDR3
Late 2103 iMac

Maybe too late for you if you don’t have time to change format (but may help someone else with a PhD length project in Scrivener), but I had a very large 30,000 word project with around 50 300dpi images taking > 200MB, but because I used multimarkdown, compile always remained super quick. In MMD you link to your images in plain text, so it is just way more efficient. You also get professional level figure captions (to make table of figures effortless), proper block quotes, proper headings (to make TOC effortless), proper equations etc. etc.

If you only have a few images then why not leave them out for now? After you compile you can open the file in Word and insert the images then.

Otherwise I can only reiterate the points of advice that have been given over the course of this thread. Not all of these would be necessary, usually only one suffices, do what works best for you:

  • Don’t use the Java converter, just use RTF. (May not work for you as you mentioned before, but in general there is little reason to first make an RTF file and then convert it with Java, Word is a better converter in every way, the Java converter is there for people who use Pages or simple mobile apps that can’t work with RTF.)
  • Use smaller images that are sized to fit the page rather than shrunk in Scrivener.
  • Compile in chunks instead of the whole thesis at once.
  • Since your images are linked, duplicate the image folder and then batch process the live images Scrivener uses to thumbnail scale. You can replace them later with full-res in Word as a final step before submission.

As to quantity of images: it doesn’t matter as much as overall weight of the images. I can break the compiler with one single image… if it’s the size of a roadside billboard. I could also break the compiler with one very small image using an arcane format that crashes Scrivener. The amount of RAM you have also doesn’t matter much since the limitation is in the toolkit which cannot use all of the RAM you have anyway.

Links: they only make things efficient for the project and while you are working. When you compile every image must be imported, processed and fully embedded into the RTF file during compile. RTF itself does not support linking to images.

Exception: the sort of linking nontroppo mentions, where the file format is text and thus the link remains text throughout the entire compile process. In systems like that, the file is not stored inside the text file, and thus the overall memory load is distributed across dozens of files rather than one huge file.

Dear Forum admins,

I could really use some help with this. As I stated in my last post, even compiling a document with just a couple of images is really slow.

Darren

I am not clear on what you mean by that. Are you saying none of the above worked? Because some of the tips in there involve removing images entirely and deferring the integration process until post-compile, and surely that would speed up compile.

Hi Amber,

I just saw your Dec 10 post. Will try it out.
The Java converter I refer to is the one that Scrivener invokes when compiling a Word Document.

Thanks

Darren