PDF Bookmarks from Scrivener - help please?

I’m trying to end up with a PDF file from a Scrivener project where the PDF bookmarks correspond to document titles. After some trial and error I have succeeded in getting a TOC with hyperlinks through to the PDF (via an RTF export to Open Office, and then PDF export from there). The Open Office document has bookmarks evident in its navigator panel, successfully exported from Scrivener. However these bookmarks just don’t seem to export to the PDF - even though all the appropriate “export bookmarks” settings are set. I’ve tried setting heading styles in the Scrivener Compile Formatting panel, but that doesn’t seem to remain defined as a heading style after exporting as RTF. If that worked then it would be fine because exporting to PDF happily creates bookmarks based on heading styles.
Looking at the Scriverner User Manual, it is a PDF file with lovely nested bookmarks in it as well as internal links and a hyperlinked TOC. I have far too large a project to be doing this manually using Acrobat. Presuming it was written in Scrivener, can someone please explain how to do this? Any helpful suggestions would be greatly appreciated.

I have figured out a workable solution if anyone is interested.

I am. I’ve been considering various possibilities for doing just this, including MMD->LaTeX, which I think is how the Scrivener manual is done. I’d be interested to hear how you have managed it.

That is correct, the Scrivener manual is utilising MMD/LaTeX’s innate ability to automatically label each section that is emitted as a title, and then cross-reference to those sections in the text. The pdflatex utility takes that file and outputs a PDF with a proper ToC that matches the printed ToC depth, and hyperlinks all cross-references, automatically. It’s a bit awkward because technically speaking, the user manual is 100% made in Scrivener, as it should be, but it utilises a more advanced end of Scrivener that most are not familiar with, and so leads to queries about how such-and-such is done.

So that is all a substantially different way of working and requires a little investment of time and learning to break in to; more so if you’ve never used LaTeX before.

Thanks for the clarification. I don’t use MMD in Scriv, not because I’m worried about the difficulty of setting it up (I’ve used LaTeX a bit), but because of the difficulty of coming up with an MMD workflow that can easily produce both rtf/doc/docx and pdf output as required (essential for my purposes as an academic).

However I’d like pdf output to contain proper links and a pdf table of contents and my current workflow (Scriv -> Mellel -> pdf or Scriv -> Mellel-> rtf or Scriv -> Word) can’t do that (Mellel does only some of the links, and not the pdf ToC). Hence my interest in mikejcsmith’s ‘workable solution’.

I was going to say, by the way, that it is now possible to get pdfs with ToC and links out of Nisus Writer Pro, with the recent release of 2.0. I’m trying that out at the moment as a replacement for both Mellel and Word in my workflow – reluctantly, because I really like Mellel.

For those who are interested in this discussion, it may be worth knowing that all of the workflows which go via a word processor (Mellel, Nisus or Word) to rtf/pdf are clunkier than the MMD->LaTeX workflow to pdf because the output has to be marked up with styles, cross-references and so on in the word-processor, and this cannot be fully automated.

Quite right, that has been its Achilles’ Heel for a long time, however that is changing. MMD3 supports .fodt export natively, and it does a pretty good job of it too! You get real stylesheets and all the good stuff, sidestepping that whole can of worms. Your compiled documents will be agile after you’ve exported them from Scrivener. FODT files are not widely supported by word processors, but the free LibreOffice (an OpenOffice.org fork that has broadened support for things like this) can open them and then save them to RTF or DOC/X as needed. It’s worth giving it a shot and seeing if it will do what you need. Main limitation I know of right now is that it doesn’t support enumerated lists—they’ll end up as bullets.

This is true, if I fixed up my tables in the .scriv project, I could get the user manual PDF out of Scrivener in seconds. I still have a little manual labour I need to do though, so in reality that works out to more like 15 or 20 minutes—a number which is shrinking as I have time to make the automation better. That said, stylesheets are of varying importance to individuals. Some just don’t need stylesheets so they can skip all of that—and if you don’t need them, using a word processor is significantly easier than LaTeX, unless perhaps you work in the sciences. But if you do need a proper ToC that is based on document structure—yup, it’s a lot of work, and probably advances the point at which people have to stick in a word processor moving onward, a bit higher; pity it’s a tough nut to crack. You can do cross-references in Scrivener, as you note, and any word processor that can deal in RTF bookmarks will recognise them. Getting that to PDF from the word processor is another matter.

Good point. I meant something like: (of workflows whose output needs to be marked up) MMD/LaTeX is less clunky. Of course if the desired output is a traditional manuscript with no more formatting than bold, italics and a bit of paragraph indenting then the rtf workflow works perfectly. I’ve done it myself: thank god for publishers who want to do all their own heading styles.

Do you mean the feature whereby compile will export bookmarks for each heading? I’ve never used that actually – I’ll investigate. But what I’m more interested in is getting cross-references to auto-number items to come out as rtf (and then pdf) links. That’s not currently possible is it?

About x-references, I think he refers to stuff like Table <$n:t:label>, and then “See Table <$n:t:label>”. There are other types as well. It works very well for me. I think they’re called “placeholder tags”.

Right. I mean cross-references to placeholder tags.

Indeed! Writers shouldn’t have to be type design experts.

Ah, yes, in that case what I was referring to is sectional cross-referencing only. The ToC thing I referred to merely demonstrates one implementation for a coupling of features which can make printed cross-references look nice. One is to Opt-drop the section title from the Binder that you wish to link to into the text area, so that you have a Scrivener Link by name. The compiler can be set to insert any prefix necessary to keep it synchronised with the printed section name. The other is to link up a <$p> tag to the same document, which uses the aforementioned RTF bookmarks to dynamically print out the page number.

For item linking like figures and tables, the named counter syntax that chantun points out is best, so long as your references are backward looking only. Since a named counter is has no ref vs anchor distinction, whichever one occurs first will increment the instance—thus if the cross-reference appears before the actual figure in the text, it will increment the counter, not the figure. Only work-around for that is creating a junk document at the top of the compile that lists all of the figures/table named tokens in the correct order that they appear in the book. This way the numbers are all incremented and set before any instances in the document appear. Then you just need to wipe out one row of numbers every time you compile—or if you can’t, set their text colour to white and font size to 1px.

Since the counters just turn into ordinary text, you’ll have no problem with them in PDF. Hyperlinks are another matter, I’m curious about the OP’s solution, too. I know web hyperlinks are no problem with Scrivener’s PDF, but internal linking is something I’ve never found a way to get working. I have a feeling they forgot to turn on notifications though. :slight_smile:

The publishers who do want marked up styles in rtf/doc don’t expect you to design the book (usually). What they want is the semantics of the text explicitly indicated: what is a level 1 heading, what is body text, what is a blockquote etc. They often try to get this by demanding that authors use their Word template, and use the styles in it. The last part is beyond most academics.

I know. This is a pretty big problem… (I’m not complaining, by the way. I know Scrivener was originally designed for fiction manuscripts. It’s amazing how in recent versions it seems to be the best place to draft just about any kind of writing.)

Thanks for this. I will try it.

I think he may have discovered a truly marvellous method for exporting links, which this forum is too narrow to contain.

Margin width has been the bane of civilisation for far too long.

Hi all and thanks for the comments and replies to my question. I get that I need to learn about MMD/LaTeX to get this really under control. I’m pushing deadlines for a draft right now, so in the meantime here’s my workable solution.

I have recorded a screencast here to show the details. A brief text description is further below.
http://www.workplaceenglish.com.au/file.php/1/public/scrivenerheadingsdemo.mp4/index.html
Direct download the video file here:
http://www.workplaceenglish.com.au/file.php/1/public/scrivenerheadingsdemo.mp4/media/scrivenerheadingsdemo.mp4.mp4

Essentially I want to output to PDF from Open Office using heading styles to generate bookmarks in the PDF document. To do this, the process is:

  1. In Scrivener’s Compile settings define unique font sizes for document headings based on their level in the Draft - top level will become heading 1 so we define it as 19pt, second level as 18pt, third as 17pt and so on. These must be unique font sizes, not used for anything other than these level headings, because we are going to search and replace on them later.
  2. Create a Table of Contents in Scrivener using the Copy as TOC method and copying/pasting the draft outline into an appropriate place in the draft. Create any other internal links you need using the <$p> tag. These tags are what ends up as the Table of Contents (not bookmarks) in the PDF file.
  3. Compile to RTF and open in Open Office Writer.
  4. Search and replace on the font sizes and redefine those lines as heading styles. I used a crude macro to achieve this.
  5. Export to PDF from Open Office. The heading styles become the bookmarks in the PDF, correctly sorted and in the orginal heirarchy.

The thing to understand is that exporting RTF bookmarks from Scrivener into an RTF file then opened by Open Office Writer only partially works. But even if this did work well, these bookmarks don’t transfer to PDF upon exporting out of Open Office. (I tried all the settings - to no avail.)

If anyone would like to describe a better way, I’d really like to know because within two months I’ll need it. Perhaps someone can point to or create a screencast on LaTeX. I did some reasearch here:
http://theoval.cmp.uea.ac.uk/~nlct/latex/pdfdoc/pdfdoc/pdfdoc.html
and here:
http://www.math.rug.nl/~trentelman/jacob/pdflatex/pdflatex.html
but I’m just a bit lost on all that right now.

In particular I’d like to be able to preserve links that render as text rather than a page number (as produced by the <$p> tag). My solution doesn’t do this (so far). What the method does achieve though is it can process my very large project of 1500 pages. It did so while still maintaining the relatively small file size. Using MS Word in place of Open Office Writer blew the file size out to 15 times, even staying with the RTF format. With this rough-and-ready method I can output my PDF in around 15 minutes of effort, which is fine by me.

Thanks again and regards,
Mike Smith

The adventure continues…

I’ve now installed LaTeX, and run my demo file through it. It produces a beautiful PDF but there’s obviously a learning curve to fine-tuning the typesetting, and at present I have no bookmarks. I’m not Mac-guru enough to understand adding the pdflatex utility (as yet) to get the bookmarks happening.

AmberV, you mentioned pdflatex. Can you describe how this works? Do I really need to work from the command line? Perhaps there are some useful tutorials (for beginners like me).

Cheers,
Mike

A little more progress…

I can see the depth and power of LaTeX and have found the pdflatex output in the Macros menu of TeXShop. I now have bookmarks, but it is two levels short of what I need. A little digging in the LaTeX document indicates the following are the limits of the table of contents generator.

\section{…}
\subsection{…}
\subsubsection{…}
\paragraph{…}
\subparagraph{…}

I would need subsubsubsubsections - levels 1 to 5. Does anyone have any suggestions here?

I’ll persist in reading up more on LaTeX, but at this stage I’m back to my “workable” method to get the 5 levels of bookmarks.

That aside, I would say the output from LaTeX is quite amazing but, as others have said, involves a learning investment effort.

Cheers,
Mike Smith

One of the preferred ways to handle .tex files on the Mac is to use a free text editor called TeXShop. It comes installed with many modern LaTeX distributions, and so you might already have it installed in your Applications folder, perhaps under a folder called “TeX”, but if not it is easy to find and install via MacUpdate or some other repository. TeXShop makes it so you can load a .tex file, press Cmd-T to typeset it (a second time to resolve all cross-references correctly), and a few seconds later you’ll see a preview window pop up. When this happens, you’ll already have a .pdf file in the same location as the .tex file.

Are you using an article style? By default the section layout has part, chapter, section, subsection, and subsubsection. There is your five levels, if you have parts. If you don’t have parts then getting five levels of sections will be a little more involved. You could investigate changing the part style to look and act different.

The two codes you need to change numbering and ToC depth are:

\settocdepth{subsubsection} \setsecnumdepth{subsubsection}

This will need to pasted in anywhere near the top of the file, before the [b]\tableofcontents[/b] code. They will cause headings down to sub-sub-section to be numbered in the book, and cause the ToC to list them.

This by the way is for the default book Memoir class that gets used by MMD, so unless you’ve changed that, it should work okay. Otherwise there are more standard macros to use:

\setcounter{secnumdepth}{3} \setcounter{tocdepth}{3}

Note these will not work in the Memoir class (you can tell what class you are using by the [b]\documentclass[/b] flag at the very top), I’m just putting them here in case you wish to use another class. Most of the MMD classes are derivatives of Memoir, however.

Document classes are a bit like stylesheets in LaTeX, except they cover more ground—it might be better to say they are like Word template files in that they can set up styles, define macros, and handle the overall structure and layout of the file, including the definition of heavy-duty book features like glossaries and appendices to name a few. Switching classes can often be done on the fly, but some classes are more involved than others and might require a little tweaking to the source .tex file to work. Memoir is one such class. It is enormously powerful, the class itself has a nearly 600 page user guide. But if you intend to tweak the layout, that’s the place to go for answers as Memoir has a lot of heavy macro usage on top of standard LaTeX, and not all of the tips you read on the Web will be relevant to it; though certainly do try them first.

Macros in LaTeX are slightly different than you might expect from the name. They define typesetting macros. When you use [b]\chapter[/b], you’re actually calling a macro that does all of the detailed typesetting work to make it look like a chapter. Thus, changing the chapter style (and Memoir comes packed with a bunch of heading styles; the Scrivener user manual uses a derivative of the ‘thatcher’ style) is usually just a matter of supplying a new macro definition in the top part of the document. This can either be a setting switch, if you intend to use one of the pre-built header styles (I do recommend that, starting out), or a complete redefinition where the whole macro is rewritten for new typesetting. Try adding a meta-data key in the Scrivener compiler called ‘chapterstyle’, and type in ‘veelo’ into the data area, recompile, typeset, and check the difference in chapter headings. The linked Memoir user manual above has a list and demonstration of all the available presets. There are presets like this for many aspects of the book’s layout—so using a little mix and match and experimentation might yield a result you are happy with, without having to mess with any low-level typesetting.

To the top of your .tex file and rendering that out to see what I mean.

Definitely, though in my experience once you do pick up a few fundamentals, it isn’t too difficult to do most things. I have to look up documentation here and there, but once you do get things looking the way you want them, you can just save all of that work and use it going forward. The last time I did a big round of typesetting adjustment was for the Scrivener user manual, and I’ve hardly touched it since then, but have made countless copies, so once you do get it working the way you want, it’s quite easy to use for production work. It’s probably equivalent in work to someone very adept at Word, and capable of programming macros to convert a plain RTF into a styled document. A lot of up-front work to get the macro right, but once you have it you can carry on at full speed. Both are probably easier in the long run than manually fixing styles every single time.

Another alternative entirely is to use LyX, which is a bit like a word processor that just so happens to use LaTeX as its save code. You can import the .tex files you create in Scrivener, and then use palettes and otherwise somewhat normal GUI interactions to mess with the file. A lot can be done in the document settings palette. Main drawback is that the Memoir class and MMD do some non-standard stuff and it will show up as “code” in LyX, little red boxes everywhere. This could probably be optimised so that MMD produced a bog standard LaTeX file that LyX likes and takes in cleanly, hmm. Might look into that some time.

Thanks AmberV for that comprehensive reply,

I have now installed a full LaTeX distribution that includes TexShop and was playing with that over the weekend. I can see some of the things that you’ve mentioned such as document class and that part, chapter, sections code. It led me to read through Tobias Oetiker’s “Not so Short Introduction to LaTeX2e” and then I re-read the entire Scrivener User manual. I also found some useful references in the MMD forum here that you and others wrote some years ago (referring to Lyx). So now I’m getting deeply into the TeX code itself and figuring out what Scrivener is outputting.

Let me work through all that and I’ll get back to you, hopefully with some happy progress. I have decided that it’s definitely worth the investment, but I’m not sure if I can quite get there before this month’s draft deadline. Perhaps with your guidance I can. But, there are always more deadlines and I have more courses to write… and perhaps a book about my journey in writing them…

Thanks again,
Mike

Sure thing, and one thing I would add at this point in history is to download MMD3, install that, and then download the MMD3 Mac support installer as well, which will be necessary to fool Scrivener into using the new version of MMD. This is the direction things are heading, and in the new direction it is considerably easier to adjust the “containment” TeX code—that is all of the stuff that goes around your material, which is where you declare things like ToC depth and numberings and all of that. It’s a little more difficult to customise MMD2 in this regard. The trade-off is that with MMD2 you could more seamlessly adjust the actual codes that get output, as well. This is a definitely a step even more advanced than everything else so far, I wouldn’t worry about the distinction just yet, but if you do need to adjust the low-level output (like changing how a figure is inserted), you can still do so with MMD3—you aren’t giving up that power—it’s just that the default way of producing .tex documents has been streamlined into an easier to use package.

So I’d focus my research on learning the new MMD3 system (there are a few substantial differences, and it would be a pain to have learned them only to realise they are already legacy methods), which is Scrivener compatible (and will become more so in time) with the optional support installation. The main hitch right now is meta-data order. For now I recommend using Scrivener’s ability to use a document named “Meta-Data” at the top of the compile list to supply MMD meta-data, instead of using the built-in compile interface. This is just a temporary work-around, and it will be easy to revert back to using the interface in the future. You’ll be able to copy and paste text meta-data blocks into the Scrivener compile table and it will convert them all to its system.

The roadmap for MMD3 and Scrivener: next update, 2.1, will allow shuffling of meta-data fields, will respect the order you shuffle them to, and will let you turn off the legacy “Format:complete” line that currently gets added automatically (MMD2 needs it; MMD3 doesn’t use it).

After that, we’ll maybe be working on better MMD3 feature integration; such as adding support for making the new .tex include files much easier to work with (you could select from a number of presets, or paste in your own preamble and footer into the compiler). The details aren’t hammered out yet, so I don’t want to promise too much on that score.

Hint: to get your meta-data in text form, just do a quick straight MMD compile out of Scrivener, and copy and paste the top portion of the file with "Title: " and all that into a new document called “Meta-Data” in the Draft outline.

Hi AmberV,

Oh, OK I see what you mean. Will do. Several of my lines of of research are converging on MMD3. It’s good to see that active development is happening in the document production workflow from Scrivener. Yes I think I will need to modify tex output fairly substantially because the Part, Chapter, Section, … structure doesn’t suit me at all. I just need Document, Section, Sub-section, Sub-sub-section, Sub-sub-section, and perhaps a couple more levels below this to be the basis of the PDF Bookmarks. (I know it sounds odd from a document layout point-of-view, but I’m producing an online course with heirarchical depth and using Scrivener to manage its rapid development and long-term off-web master reference version.)

When playing with TexShop/pdflatex, one thing I discovered quite quickly is that I have to move away from my habit of defining paragraph spacing. When going to TeX notation, a blank line is needed to define a new paragraph. I’d stopped doing that a long, long time ago when I moved from XTree Gold as my thesis editor (yes a looooong time ago) to MS Word and started using paragraph styles. It (the double carriage return) is a fundamental principle of document layout notation in TeX it seems, so I will need to reformat my entire project and library of source texts to suit that workflow. Thank goodness for Text Wrangler!

I’ll need to be careful how that maps to the

tag in HTML output so I don’t get double spacing. Perhaps the extra CR needs to be a step in TeX production rather than its removal being a part of HTML production.

Regards,
Mike

Hmm, actually I think I can envision a solution that doesn’t require modifying the output, and just relies upon adjusting the preamble boilerplate. Getting rid of parts and chapters is easy enough to do by using an article documentclass (or memoir in article mode, which is what MMD does), but there isn’t a “Document” sub-division by default anywhere, which means the chapter heading will need to be redefined in the preamble—that can be done without any lower level fixing. Everything so far is just adjusting the preamble, which MMD3 doesn’t touch. The nice thing about Memoir’s article emulation is that it does have the \chapter command available, something most article classes for LaTeX do not—so it would be there to tweak.

Point of research: \renewcommand

That’s one way to override LaTeX vanilla to do your own thing. Easiest is to get a code sample of roughly what you are going for, and tweak from there. Do check the Memoir docs first, though, as many typesetting level adjustments can be made using its built-in macros, or by renewing only parts of the full chapter environment. The Memoir documentation displays the various chapterstyles available, and produces code for them, so it is pretty easy to work backward from that and create your own chapterstyles (renaming them entirely to “Document” if you wish).

The trickier part will be getting the 5th and 6th (and beyond) levels of depth. MMD itself does not address below the 6th level of hierarchy, a document that goes that deep is fairly unusual. Most LaTeX document classes don’t really go beyond a 4th level, either, though you could use the \paragraph and \subparagraph levels, if you don’t mind in-line headers at that depth. Numbering can be applied down to that level, too, by adjusting the code I provided to state ‘subparagraph’.

Still, at this level, we are operating at the preamble level, which is good—that’s how you ideally want to work in LaTeX. Keep the material agile by removing all dependencies to application and then bend the application itself in the preamble by defining and adjusting macros and commands. Adjusting the look and feel of a document by its containment, not by adjusting the material itself.

For getting 5th, 6th, and greater levels, I would try this approach. Set up the compiler’s Replacement field with something like:

Replace: \five
With:

Then turn off auto-titles in the Formatting pane below level 4.

Replacements work like search and replace, only they run while compiling and don’t change the source. So you just type in ‘\five’ at the top of the fifth level document, and Scrivener injects the LaTeX code for this, using the document’s Binder title, in an HTML comment, which is how you pass through raw LaTeX so that MMD doesn’t escape all of the special characters.

Then set up another for level six (and beyond if needed, perhaps using your own LaTeX commands for deeper levels). This way you don’t need to mess with XSLT, and can stick with the faster binary method in MMD3 (and it’s way faster! The user manual takes about a minute and a half to go from Scrivener to .tex in MMD2, and literally a fraction of a second with MMD3).

The other alternative is XSLT, but that is another programming language to learn, and if you’ve never dabbled in functional, as opposed to procedural or object oriented programming—it’s a bit weird.

This is true in MMD as well. Both are plain-text based, and there is no good other solution for the problem of distinguishing paragraphs without an empty line—at least not one that reads well. Single-spaced paragraphs just look like word vomit without some kind of rich text system to visibly push paragraphs apart or indent them.

I’m totally used it at this point, so don’t even think about double-tapping return whenever I’m done with a paragraph—but Scrivener does have a tool in the compiler that may help you transition. In the Text Options check for an option to convert rich text spacing to plain-text spacing. I recommend setting it to paragraph spacing only. So long as your styles in Scrivener supply at least 50% of the line height in padding space, it will insert a blank line for you. This can cause problems in some places however. Tables, for instance, and verse, so I don’t recommend it in general unless you already have thousands of words written with single-spaced paragraphs. It will be a life saver in that case—so will TextWrangler.

I think you are okay on that score. Well, for one if you use the MMD->HTML compile format instead of the stock HTML generator (which you should, because it produces a more semantically driven document than the Cocoa routine), but even using Scrivener’s Cocoa generator, it won’t double up and create empty paragraph sets. It does seem to either do a Tidy style post-process, or just never produce blanks in the first place.