Scrivener to InDesign

Scrivener
1

I use Scrivener for my PhD thesis together with Sente as reference manager and InDesign to produce the final output.

The basic problem that kept me from moving to Scrivener since it first came out was the lack of proper paragraph styles conditioned by Apple’s RTF text engine. Of course, there are Word and Mellel with their find/replace functions that can be used to turn formatted text into paragraph styles. But neither of them is an option for me: Word because it drives me crazy within minutes, Mellel because of other things (no South Asian text, a collaboration nightmare, not good for archiving). In addition, neither Mellel nor Word offer the layout and typography features I need and neither produces tagged, linked, proper PDF.

Therefore, I was looking for a solution to migrate a Scrivener produced RTF to a structured InDesign document, complete with footnotes, paragraph and character styles. To this end, I wrote a simple AppleScript that post-processes an imported RTF in InDesign CS4 (see attachment). Admittedly, this is a tailor-made workflow that will not suit everybody. But I thought I’d share it here for future reference in case others have similar requirements.

My text formatting needs for academic use are quite simple. I need italics, a hierarchy of maximum three titles, a paragraph and a block quote style, footnotes, integration with a reference manager, and cross-references (for the latter I have not yet found a solution).
The conventions I defined to turn formatted rich text into paragraph and character styles in InDesign are the following (Scrivener/RTF to the left, InDesign to the right)
font size 12pt > paragraph style “Paragraph”
font size 11pt > paragraph style “Blockquote”
font size 18pt > paragraph style “Heading 1”
font size 16pt > paragraph style “Heading 2”
font size 14pt > paragraph style “Heading 3”
italics > character style “Italics”
Scrivener footnotes > paragraph style “Footnote”

This is the workflow:

  • Export as RTF from Scrivener without overriding text formatting. Either work with Scrivener created folder/text-container titles and set them up to match the criteria above, or simply include the titles in the documents themselves ( I chose the latter option and use Scrivener in scriptwriting mode to quickly apply formatting to titles and entire paragraphs)
  • Run it through Sente (or any reference manager that scans RTF) to create the references and bibliography.
  • Create a new InDesign Document and define the paragraph and character styles as listed above.
  • Import the RTF
  • Run the AppleScript (copy it to the Scripts folder if you wish in order to make it easily available in InDesign)

Tested with a 100’000 words document. Works seamlessly in my setup; not tested anywhere else, though.
RTF_PostProcessing.scpt.zip (10.8 KB)

2

Thanks very much for this xaxa. I have been wondering for a long time how to get my teaching materials from Scrivener into InDesign. I produced a lot of stuff a couple of years BS —“Before Scrivener” :slight_smile: — and now want to do a lot of revision and make quite substantial additions. I’d rather do the heavy lifting in Scrivener and the refinement in ID, but how to automate the style processing as much as possible was something that was exercising my mind — I’m thinking immediately of what might amount to 200+ pages.

So I’ll explore your Applescript. The only thing is that currently I only have CS1 — never having had the money to upgrade at Adobe prices — and though I discovered recently that it would be possible to get a teacher’s licence here for CS5, and was going to visit Adobe’s reseller in Guangzhou, I baulked at it when I discovered that they would charge RMB 4,500, the equivalent of USD 650, when the US price is only $299, and even in the UK, unusually, it’s only approximately the same as the US plus VAT. Result, for the moment, I’m stuck with CS1, but I hope your script will still work.
:smiley:
Mark, a.k.a. Mr X

Edited for word substitution :confused:

3

One trick I use to bring formatting from Scrivener (or any text application) into InDesign is to code the three most common text formats this way:

«Text» For italic.

««Text»» For bold.

«««Text»»» For Bold Italic.

You can do that in any Mac text editor using the Opt-\ and Shift-Opt\ keys. You can even do it on an iPhone/touch using the extended characters shown by holding your finger on the " mark on the numbers keyboard.

Then I write a GREP search and replace (CS2 and above) to find each of the above combinations and apply the appropriate InDesign text style. When I need to convert a text document to add text styles inside InDesign, I run the above searches in reverse order. «««Text»»» catches all the bold italic text and removes the marks, ««Text»» finds the bold, and then «Text» finds the italic.

As a start, here’s the code I use for italic:

In “Find what” I put:

(\«)([^»]*)(\»)

And in “Change to” I put

$2
Along with a Change Format that specifies my italic text style.

The coding $2 grabs the actual text for the replace while leaving out the « » marks. GREP, a tool from Unix, is much more powerful than standard S&R. I just wish Apple would add GREP S&R to their standard text tools, so it could be included in Scrivener.

Notice that this also is a clever way to do something InDesign doesn’t otherwise permit, placing formatting like italics and bold inside an index. Code your index entries as above and, after an index has been generated, run the necessary GREP search and replace on the index alone. It’s a great way to include italicized book titles in an index.

I’ve toyed with coming up with a similar way to code paragraph styles into basic text using something like to tag a paragraph as a quotation.

–Michael W. Perry, Untangling Tolkien

4

That would be awfully nice if Apple added regular expressions to their standard dialogue, but I highly doubt they ever will, it is just too arcane for most, and they have been trending toward non-specific fuzzy searching for some time now.

The only thing I would amend with your pattern is to not use parenthesis around the surrounding pattern. Parenthesis are most expensive, and since you aren’t use them for either grouping or data collection, you could without them entirely.

\«([^»])\»

ought to suffice, and of course then you would use $1 to collect the data in the replace field. The only thing this pattern may not catch is multi-line italics. That depends on how the engine is configured in InDesign. Most terminate the search pattern at \n, unless you tell it to search through multiple lines. So if the italic field is several paragraphs in length, this pattern may not catch it.

5

Mmm … very interesting. One of the things I like about Nisus Writer Pro is that “Powerfind Pro” uses GREP. I’m not a wizard at it, but I still prefer it to regular search and replace, and if one needs to do something really complex, Martin, Kino and others are there on the forum and will help.

I had a look at xaxa’s script … of course the first thing I will have to change to try and get it to work is CS4 to CS1; then, from InklingBooks’ comment, I take it that CS1 doesn’t have GREP. I don’t remember ever using it in InDesign, as much of what I’ve written, I wrote directly in ID.

Oh well, I’ll just have to work out how to get a teacher’s licence for CS5 without having to pay a premium of more than 100% … or else investigate switching to Quark Express 8, except that they don’t seem to have any resellers anywhere nearby, and their Hong Kong people only mention QE7 on their website, not 8 … and I wouldn’t be surprised if they only carried a Windows version.

Mark

6

I’d also like a cheap license to Adobe products. Searching on the Internet I found this website:
– Removed due to security risk –
I haven’t tried it, but if it’s real, it will surely be an excellent and very cheap way to buy a license. Maybe someone else in the forum has used this kind of websites before and can tell us if it works.

7

You’ll want to be careful with sites like that. OEM software is not meant to be sold individually like that—part of why it is so cheap is because a hardware company purchased thousands of them with a special licence that prohibits its sale except when bundled with that hardware. Just buying an OEM disk all by itself, or as a download (that’s a big red flag), is often times pretty “grey market”—and can sometimes just be downright black market: somebody is selling bootlegs and calling them OEM. It can be legitimate, and I don’t know whether this site is one of those. Just be careful. Often “OEM” is a bit of a code word for buying pirated software. For Mac software, just think about how many times you’ve actually seen an OEM version of a program sold with a new Mac. Apple hasn’t bundled third-party software in many years, and certainly not from Adobe. Mac OS X Photoshop CS4 OEM doesn’t make much sense, chances are it’s a cracked version with the activation stripped out and replaced with a dummy activation to make it seem legitimate.

In nearly all cases, the way to get cheap software is to get alternatives. If you can’t buy InDesign at street price, then look for programs that do something like InDesign for $60. There are a lot of dedicated hard-working developers out there making great applications like these. In some cases you can even find good open source alternatives, too.

Mark’s comment on getting academic status is probably the next best way, but not possible for most people. Plus, academic versions also often come with strict restrictions on how the software can be used. Most (from major companies) cannot be used commercially at all.

8

If you use a Windows computer:

DO NOT enter the norsemaille site. Kaspersky Internet Security claims that the site tries to infect Windows computers with a Trojan horse.

Franz

9

Well, and that should pretty much tell anyone with any kind of computer all they need to know: that they do not want to go to the website.

10

Thanks for the report Franz. I’ve blanked out the link in the original post.

11

One less-expensive way to get Adobe’s pricey products is to buy an older version that’s recent enough to allow an upgrade. Just make sure you are buying (or can get transferred) the license. Adobe does allow a license to be transferred, although I’d check with Adobe to make sure your transfer fits their policy.

You can find Adobe’s requirements here:

adobe.com/products/indesign/upgrade/

And here’s what they say:

YOU HAVE YOU WANT YOU PAY
Adobe InDesign CS4, CS3, or CS2 Adobe InDesign CS5 US$199
Adobe PageMaker® 7.0 or 6.x Adobe InDesign CS5 US$199

Note especially that Adobe is still converting PageMaker licenses for 7.0 (2001) and 6.0 (over a decade ago) to the current version of InDesign for $199. Since Pagemaker is discontinued and was never ported to OS X, you should be able to get a copy very cheaply. If you get it for $25, you’ve got the latest InDesign for $224, about one-third the list price.

Just be aware that upgrading this way often means a call to a special Adobe rep who’ll talk you through a secret procedure to upgrade without an old CD/DVD. Very cloak and dagger stuff.

As an alternative, look for a (rare) good deal on the just-supplanted version, InDesign CS4. It’s the one loaded with long document/book goodies.

CS5, to my great grief, is mostly about Flash. And oddly, it can’t do the one thing that I might find useful, create a Flash promo for a book with various pans and zooms on multiple pages. For that, it’s easier to export pages to JPEG images and pan/zoom inside iPhoto. That’s why I’m taking a pass on CS5.

–Michael W. Perry, Seattle

12

Thanks for making this. I’m also doing a dissertation and would rather use InDesign than Word or LaTex. Any advice as to how I might get this to work with CS3?

Thanks for the help.

13

Apologies for this very vague answer. I do not have a copy of CS3 around and cannot check. But as far as I remember, InDesign CS3 featured a different AppleScript syntax for find/replace. The rest should more or less be working (but no guarantees…).

14

PS - a technical detail.
Just had a look at the script again. If you intend to use it, have a look at lines 12 to 14. The version I posted includes a workaround for an InDesign bug that occurs with certain OpenType font families such as my beloved Arno Pro (“italics” somehow becomes “caption” during export). For simpler font families uncomment line 12 and comment out line 14 instead.

15

so,

how is your workflow for getting images that are placed on scrivener-pages into indesign? am i missing anything here, or am i really supposed to re-add all the images to indesign after i exported my compiled draft from scrivener without any images? that would really cost me like 3 days -_-

thanks!

16

I don’t see a solution for your problem, I am afraid. The RTF format uses embedded images with no links to the originals. Only compressed JPG and PNG are supported; no TIFF, PSD, or any other format suitable for print. In addition, all metadata is lost; thus, it is not possible to get any kind of link/path/uuid related to images out of Scrivener. Above that, InDesign does not support images in its RTF import, as far as I remember.

I used a moderate amount if images in my dissertation (around 80). In my case, I decided early on that the text and the images (with their captions) should be two separate but intertwined narratives, at times confirming each other, at times producing a certain amount of friction. For the text I used Scrivener; for the work on the other narrative told in images/captions I worked with Aperture and InDesign (using these sripts: http://www.macosxautomation.com/applescript/aperture/indesign/index.html)

17

That’s how it looked to me as well, from when I was testing it out. Using InDesign CS5, the imported RTF was actually rather basic. Adobe doesn’t seem to have paid a lot of attention to creating a good importer for it. Scrivener will actually fully support a linked image format with 2.0, but unfortunately the only way of doing that is with RTFD, which InDesign doesn’t fathom.

Xaxa’s advice about image purpose is also sound. There is usually a pretty big difference between the images you drag into Scrivener and images that are ready to print. High resolution images are going to bloat the project size quite a bit, and their format will end up being RGB instead of CMYK due to supported formats—that’s not the type of conversion you want to leave up to the computer to handle (unless colour shifting doesn’t bother you). If you are working in B&W, that’s another matter.

Then there is the issue of layout. Scrivener, not being a layout engine, is just going to put your images into the text stream wherever they are called for. So even assuming you could get them into InDesign easily, they would all probably still require a little tweak here and there to either get the text flowing around them properly or whatever else you wish to do. The InDesign (or comparable desktop publishing) workflow, more than any other most likely, has the harshest line between Scrivener and final production. You can, with word processor auto-format features, macros, and other tools, create a pretty efficient Compile and Submit workflow using word processors, but with InDesign—just given the heavy graphical nature of the beast, once you do your final compile and start spending hours and days in Indesign—that’s where your work will now be and the Scrivener project can be archived, all text edits done in InDesign from that point forward, as you’ll be needing to worry about page colour, hyphenation optimisation, and other typesetting tasks.

I’m not sure what capacity InDesign has for this, but you might be able to do something with placeholders in Scrivener. If you “label” your images with their filename in brackets after each image, perhaps InDesign can be scripted to look for these references and link them to a directory of ready-to-print graphics. I’m doing something similar to that right now. I have a project in Scrivener that needs to use two different sets of graphics, one in full colour and another in B&W. I’ve set things up so my project uses “placeholders”, which get swapped out with the appropriate high-res images depending on output purpose.

You might also want to check out LightRoom, which might have better integration with InDesign, if you don’t already have a heavy-duty image organiser. I’ve used both, and they are (while very different) roughly equal in ease of use and learning curve.

18

Hm, I spoke too soon. Acutally, InDesign CS4 does have an option in its RTF import dialog called “Import Inline Graphics” (you need to tick the checkbox “show import options” in the “Place”-dialog to get to it). I tried it out with Scrivener RTFs (both with JPG and PNG settings) but no images were imported. However, if you open the same Scrivener exported RTF in Word and save it again as RTF (or as .doc) under a different name, Indesign does indeed recognise the image and imports it!
This means that if you can live with the compressed JPG/PNG that Scrivener creates and if you have a copy of Word to pipe the RTFs through, there is a possible workflow with Scrivener and InDesign including inline images/graphics.

Unfortunately, neither the name of the image nor any other metadata is kept, as far as I see. So, for me its only half the rent as I still do not see a way to replace the images with high-quality Tiffs or JPGs automatically via a script. But maybe there is a solution I just do not see yet. I would love to find one. Any suggestions welcome.

InDesign extracts the images from the RTF (or .doc) file and puts them in a rather weird location: /Applications/Adobe InDesign CS4/Scripts/XHTML For Digital Editions/. Nevertheless, it seems to treat them as “embedded” in some way and the “package”-function fails to copy them all into a dedicated folder. But these things might be mendable.

xaxa

19

All good to know, though it sounds like the mechanism it uses to extract them is one step away from horrific! I must have missed that checkbox (or they moved it some place less obvious in CS5 maybe) when I was trying to do that.

And PNG isn’t a terrible format for print, so long as you don’t need CMYK, which it can’t address. While it is a compressed format, the type of compression it uses is lossless (like zipping files are), which is why PNG files are generally way bigger than JPEG. Whether or not the printhouse will accept them though is another matter. Definitely ask the people who will be printing it, before investing a lot of time.

20

I did some more testing and found that some sparse exif-data is indeed preserved throughout the Scrivener-Word-Indesign pipeline, including a timestamp called “Create Date” or “Date/Time Original”. This could be used to match the embedded image with its original again (within the limitations that a timestamp in the form of 2010-08-22 21:39:25 is not necessarily a unique identifier; think bursts of images from a fast DSLR). No time to pursue this any further at the moment, but chances are high that a full-fleged, automatic Scrivener-Indesign workflow is indeed possible.

It this works out, it would make Scrivener an even more compelling choice for a variety of scenarios that require the flexibility and features of a professional desktop publishing solution. I would start using it for drafting brochures, funding applications, storyboards, etc. - documents where text/graphic and image cannot be separated.

“Unembed Link” works beautifully and copies the extracted images to a folder of choice. So this is a non-issue.

xaxa