How to compile a Scrivener document to Latex?

[AmberV]

I have never worked with LaTex, and assumed it would not be a good idea to plan to send the press a LaTex document

I’d say it depends on whether they have a boilerplate all set up the way they prefer, and then you would only need to provide the content, which wouldn’t be too complicated. Most of the complications and learning curve in LaTeX are designing the look and behaviour of document itself.

It is something I would consider testing a bit, before committing yourself to a major transition. Put in a chapter or two, see what it takes to get it working properly (you’ll probably have to adopt styles if you aren’t using them already), and then give them a sample, to see if that will work with their system.

One advantage of this approach though is that it was designed to make it possible to use Scrivener more like a word processor to generate .tex files—a bit more like LyX, if you’re familiar. There are some notable exceptions (like the aforementioned lists and tables), but it shouldn’t be as difficult as converting the workflow to Markdown (though on that point, do test the Convert rich text to MultiMarkdown compile setting, in the General Options tab of the main overview screen. That may get you a long way toward what you need—particularly if you create or modify some compile styles that are marked as “Raw markup” in their settings—that will make it possible to paste in LaTeX equations without them being converted to verbatim characters by the RTF → MMD conversion engine.

But perhaps I should just get comfortable with the fact that since my book manuscript has very few fancy elements beyond equations, I should just use your Gen Nonfiction (LaTex) template, insert LaTex equation code where needed, and compile to PDF for my viewing pleasure, with the ultimate aim of supplying the press with the final LaTex file.

To clarify one point: you would be compiling to .tex and then using the LaTeX engine to create a PDF file, not Scrivener. If you compile to PDF from Scrivener then your equations will be raw text. It is possible to automate that last step, rather than opening the .tex file in a dedicated LaTeX or coding editor and typesetting from there. Look in the Processing compile format pane, and enable post-processing. I’ve set this template up to perform a simple straight to PDF workflow.

[jpkell]

@AmberV

One advantage of this approach though is that it was designed to make it possible to use Scrivener more like a word processor to generate .tex files—a bit more like LyX, if you’re familiar. There are some notable exceptions (like the aforementioned lists and tables), but it shouldn’t be as difficult as converting the workflow to Markdown (though on that point, do test the Convert rich text to MultiMarkdown compile setting, in the General Options tab of the main overview screen.

Thanks for your patience with me. I really appreciate the sage advice. I have been playing with this all weekend and this morning and I feel confident this is my best way forward. One problem though: I cannot for the life of me get the equations to show up numbered. This is why I desperately wanted to see your Scrivener project for the Sample PDF, because I could see how it’s done there and why my equations (using your Equation binder item) get centered and printed perfectly, but never get auto-numbered. I very much need numbering to look like “(1.1)”, and I can’t even get them to show up numbered in the “(1)” format.

[AmberV]

Are you using LaTeX to typeset the compiled .tex file? If not then no, you won’t see any numbers. LaTeX handles all of those mundane details for you, all you do is supply the semantic request for numbering, and by extension how those numbers will be used in cross-references, also automatic.

[jpkell]

@AmberV

Are you using LaTeX to typeset the compiled .tex file? If not then no, you won’t see any numbers. LaTeX handles all of those mundane details for you, all you do is supply the semantic request for numbering, and by extension how those numbers will be used in cross-references, also automatic.

I’m sorry, I’m not sure I understand what you are asking when you ask “Are you using LaTeX to typeset the compiled .tex file?” Compile is preset for “Compile for: Plain Text (.txt),” and I’m also leaving the preset format "LaTex (Memoir Book). Once compiled, the resulting document is a .tex file. It is that file that is not producing numbered equations when it is converted to PDF (with, e.g.,) Pandoc or TexWorks).

[jpkell]

@AmberV

Are you using LaTeX to typeset the compiled .tex file? If not then no, you won’t see any numbers. LaTeX handles all of those mundane details for you, all you do is supply the semantic request for numbering, and by extension how those numbers will be used in cross-references, also automatic.

P.S. I’m sorry for being so dense on this. I got myself into this jam because I wrote an entire book with Scrivener (which I love!!), and now I have a manuscript with MathType equations that need to be code and not pictures. I feel like using Scrivener as a rich text editor for LaTex is the right approach from here, but the way your template is handling equations is still a mystery for me. Unless we can figure out why I’m having such a hard time with the template’s preset rules for equations, I would love to be able to write with rich text and blockquotes using your template’s styles, and then insert an equation with standard LaTex syntax when needed, e.g.

\begin{equation} V = \int_0^\infty {{N_t}u({c_t})\,{e^{ - \delta t}}dt} \label{eq:SWF} \end{equation}

But even that is not producing a LaTex file on compile that yields numbered equations once outputted as a pdf with Pandoc.

[AmberV]

I’m not familiar with using Pandoc to typeset like that. TeXWorks would be a better place to take the .tex file that Scrivener compiles, I think. I haven’t ever used it myself, but it looks like the kind of tool that would help simplify the typesetting process for you. I’m more familiar with TeXShop, where you can load the .tex file and hit ⌘T (usually two or three times if there are cross-references to resolve) to typeset it.

This is a bit of an “is it plugged in?” question, but do you have LaTeX installed on your Mac? I recommend MacTeX if not. You could perhaps get away with the slim distribution (I haven’t tested it with this template) rather than the huge +3gb download. If you can afford the latter, I’d recommend it purely because figuring out how to augment the slim install with the packages you need would be a bit of an awful way to introduce yourself to LaTeX. Might as well just have the big huge distribution so you will very likely never run into missing package errors.

If you do have LaTeX installed, then try going into the compile settings for this “LaTeX (Memoir Book)” format, and in the Processing pane, enable Post-process on command-line. The rest is all set up to go, so long as you have a typical distribution installed, that should create a PDF when you compile, instead of having to take it into other programs.

Here is what I get if I copy and paste the equation you provided above, into an “Equation” type binder item, with a couple of lorem ipsum text sections around it. I then enabled the “Equation Packages” module in the front matter folder:

[size=80][/size]

P.S. I noticed in your equation snippet you’re including your own \label. You’ll be working at odds against Scrivener’s attempt to label these automatically, that way. In fact it might even throw an error, which might impact numbering.

[jpkell]

@AmberV

If you do have LaTeX installed, then try going into the compile settings for this “LaTeX (Memoir Book)” format, and in the Processing pane, enable Post-process on command-line. The rest is all set up to go, so long as you have a typical distribution installed, that should create a PDF when you compile, instead of having to take it into other programs.

Wow, OK, so my issue was Pandoc. Converting the .tex to .pdf with Pandoc was producing no numbers, but when I try with TexWorks I get exactly what you got. Huzzah! Unfortunately, however, my native post-processing isn’t working. This is the error I get:

Thoughts?

Finally, with regard to the reference workaround offered here: [url]Raw LaTeX, Scrivener 3 and compile process - #28 by AmberV]. How can I learn the “scrivauto” id for a specific equation so I can use your referencing trick to block out the name of the equation?

And THANK YOU both for creating this template and for your assistance over the last few days. I really appreciate it!

[AmberV]

You’re welcome, it’s my pleasure!

TeX log files are quite long, and generally what broke the process will be located close to the bottom, so I can’t really tell you what is going on from this screenshot.

I suspect though, as noted in my post-script, if you haven’t deleted your own custom \label syntax from the equation, it’s aborting on account of that. That’s how I noticed the problem myself, as I got an error the first time, noted it was complaining about multiple labels, removed \label{eq:SWF} from your equation, and then it compiled and converted to PDF cleanly.

Now what might have happened, to make a guess, is that Pandoc actually found this problem and stripped both labels out. An equation without a label won’t have a number.

How can I learn the “scrivauto” id for a specific equation so I can use your referencing trick to block out the name of the equation?

Have you tried the workaround yet? If it isn’t working as expected, let me know what your desired input (in the editor) and output (in the .tex file) should be. This workaround adjusts how links are created in the output, it requires no modification in the source text—and thus as implemented, it does not require you to know the autoscriv number. It still generates that for you, only it does so on the next line, leaving Scrivener’s title output commented alone.

But to answer your question, there is a way of getting that number and it is same placeholder code I use in compile settings, <$linkID>. So one could manually type in \autoref{scrivauto:<$linkID>} into the editor, select the placeholder part of it, and then internally link it to the intended target (drag and drop from the binder onto the selection in the editor to create a link).

But again, the workaround does not require you to do that. You’d only need to do things manually like that if you wish to use different types of links on a per-link basis.

And also, I just noticed that at the moment there is a small bug in Scrivener with how linked placeholders work in conjunction with the Markup compile pane. Links like this, that are really only links to help enable the placeholder to work properly, shouldn’t be exported as links, but they are. So right now if you try to make your own <$linkID> link in the editor you’ll get a broken output with multiple nesting autoref calls. So don’t actually try to make your own custom links just yet. It’s a useful trick to have in mind though. We’re automating with the compile settings, but all of this can be typed in by hand as well, which can be good to know as automation can be sometimes overly generalised.

[jpkell]

@AmberV

I suspect though, as noted in my post-script, if you haven’t deleted your own custom \label syntax from the equation, it’s aborting on account of that.

That wasn’t it, as I created a new project and used your method from start to finish – no custom \label syntax at all. But yeah, sorry for thinking I gave you the entire error message. Didn’t realize it was much longer than that snippet.

Have you tried the workaround yet? If it isn’t working as expected, let me know what your desired input (in the editor) and output (in the .tex file) should be. This workaround adjusts how links are created in the output, it requires no modification in the source text—and thus as implemented, it does not require you to know the autoscriv number. It still generates that for you, only it does so on the next line, leaving Scrivener’s title output commented alone.

But to answer your question, there is a way of getting that number and it is same placeholder code I use in compile settings, <$linkID>. So one could manually type in \autoref{scrivauto:<$linkID>} into the editor, select the placeholder part of it, and then internally link it to the intended target (drag and drop from the binder onto the selection in the editor to create a link).

The workaround works perfectly if I already know the Equation binder item’s scrivauto id (e.g. by digging into the compiled .tex file:

(The workaround is implemented for the last sentence only.) But if I use your recommended method, i.e. linking “$linkID” to the Equation binder item, I get a reference to the section instead of to the Equation:

Is this the error you mention at the end of your last post? If so, is the only route to hiding the name of the Equation binder item the route where one already knows the scrivauto id number by digging into the .tex file? I hope not!

[AmberV]

(The workaround is implemented for the last sentence only.) But if I use your recommended method, i.e. linking “$linkID” to the Equation binder item, I get a reference to the section instead of to the Equation:

The workaround adjusts how links are created by the compile process itself, not how you type them into the editor. This wouldn’t be something that would be “on” for one link in the paragraph and “off” for another, it would broadly impact how all cross-references work throughout the project. Make sure you are implementing the workaround as suggested, to the Markup pane.

Again, no modification to your workflow is necessary in the editor. You just drop your equation document into the text where you want the cross-ref.

I would strongly recommend not trying to manually do this yourself in the editor, and instead implement the workaround as suggested in the compiler. You are using serially generated numbers which are calculated based on the entire binder structure. If you so much as add one small equation to chapter one, every single hand-coded autoref to scrivauto:482 (or whatever) would have to be researched all over again from the .tex file. It is scrivauto for that reason.

But yes otherwise, as noted the specific method of linking safely without having to know the number, using the linkID placeholder, is not working. If you do need hand-coded links, I’d recommend placing them into inline annotations for the time being, to avoid confusion and potential syntax errors. You can enable them later by toggling annotation formatting off. That’s another type of busy work to fix later on, but it will be a whole lot more useful and a lot less time consuming than fixing out of sync numbering.

[jpkell]

Make sure you are implementing the workaround as suggested, to the Markup pane.

Aha! I didn’t realize that was the suggested way to implement the workaround. I just did as you commanded, and it worked beautifully! After two weeks of despair and struggle to figure out how to proceed, I think this is all I need to get back to writing. Thank you!

Last question (for now, I think): Using the workaround via Markup, the name of the equation is suppressed, and instead I get “Equation 1.1”. Is there a way to have the “E” in “Equation” be lowercase, instead of upper?

[AmberV]

Ah, and now you begin your descent into the wonderful and mad world of LaTeX customisation.

The first thing you need to learn is how to figure out when it is Scrivener and when it isn’t. The best way to do that is to open the .tex file, scroll to the spot, and see what you have there. Does the word “Equation” prefix any numerals in the file? Nope… so that must be coming during the typesetting process—much like the numbers beside the equation exist nowhere in the .tex file.

It is producing that label automatically, and “Equation” is its default way of referring to equations. That is flexible, if you want it to refer to them as bananas, you could do so. Copy and paste the following line into a convenient place, like the Equation Packages module in front matter:

\renewcommand{\equationautorefname}{equation} 

The quick summary of what this does:

• \renewcommand is how we change macros that already have been defined.
• In this case the \equationautorefname macro is very simple, it prints the word “Equation” wherever you use it.
• So the first part of the renewcommand stipulates which command to renew, the second part in brackets stipulates the value. If you’ve ever done programming this is a bit like variable=value. Where it gets trickier is when the right hand side is comprised of more macros. Fortunately in this case it’s very straightforward.
• Thus this line says we want the system to print “equation” wherever this macro is called.

You can make it “eq.” if you want, as well.

To come back to something I said earlier, this might be the kind of thing your publisher is all set up to do with a house style. The nice thing about LaTeX is this in fact: in your text you have \autoref{someequation}, and that’s it. How that prints is up to the “preamble”, which is the configuration for the document. What that means is that if you hand your publisher a file with the autoref stuff in place, and they have a comprehensive configuration for how equation cross-refs should be printed, you needn’t worry about the particulars yourself.

Just squint around the defaults in the meanwhile—but if they really bother you, you’ll usually have to do a little digging. For that I recommend the TeX StackExchange site. Every newbie to extremely advanced level question has been asked there, I’d bet.

Learning how to find the answer is as much a learning curve as anything else about this system. I also suggest the WikiBook, which will help you find the right words to communicate with search engines, as well as provide you with some basics of the system.

LaTeX is pretty simple on some levels, especially at the level a writer is meant to use it at. Once you get into the design or even minor customisation though, expect the learning curve to very promptly and abruptly go vertical. It’s a very old system that has had many decades and many thousands of users contributing to it over that span of time. There are layers upon layers of macros doing the stuff we take for granted, like \autoref.

My advice, stick to using the defaults unless you really, really need to change something and your publisher says its definitely up to you to do so (like maybe they don’t use hyperref, which includes the autoref command, and then you would need to adjust the Markup pane to use a different macro).

Another piece of advice: go slow. Import one chapter, convert it, clean it, test it both separately and then integrated with the whole thus far. Then move on to the next chapter.

[jpkell]

@AmberV

Ah, and now you begin your descent into the wonderful and mad world of LaTeX customisation. :lol

The first thing you need to learn is how to figure out when it is Scrivener and when it isn’t.

Got it. Thanks, the explanation that followed this helps.

To come back to something I said earlier, this might be the kind of thing your publisher is all set up to do with a house style.

I’m actually sure you’re absolutely right. I will, as you say, squint in the meantime.

Another piece of advice: go slow. Import one chapter, convert it, clean it, test it both separately and then integrated with the whole thus far. Then move on to the next chapter.

Will do! Thanks again!!!

[AmberV]

Two pieces of news for you:

1. The bug with linking placeholders has been fixed and will be available in the next minor update (which will be soon). Consequently you should be safe in using that approach—though as noted it shouldn’t typically be necessary unless you want to create a cross-reference in a style that doesn’t match how the Markup pane is set to work.
2. The next update will also include an improved version of the LaTeX project template. I’ve added documentation for equations to the help file, and have also added an “Equation” style, which can be used to create equations in the main text rather than having them in separate binder items. On the one hand it should make sections with a lot of display equations easier to work with (not broken up so much in the binder), but on the other hand the automation with cross-references and label creation depend upon section layouts. So you’ll have to supply your own label to them in the manner documented, and refer to them with typed in \autoref{…} calls (though I suppose you could make a style to handle the syntax for you, if you wanted to).

I’ve attached the template for now (the updated template is now available in the official version of Scrivener). To “upgrade” your project (I would recommend File ▸ Back Up ▸ Back Up To… first, just in case something goes wrong!):

1. First import the template with the Options button along the bottom of the New Project window. You’ll get two entries, you can check which is which by right-clicking on their tiles and seeing which one allows you to rename/delete.
2. Create a throw-away project with the new template and open your WIP.
3. From the WIP, use the Format ▸ Style ▸ Import Styles… menu command, and target the throw-away project. Replace any style collisions when asked.
4. Next you’ll need to update the compile format. Open File ▸ Compile… in both projects.
5. In your WIP, either rename or delete the “LaTeX (Memoir Book)” Format. I’d delete, but if you’ve already done some customisation to it, you might want to set it aside and merge those changes into the updated format.
6. From the throw-away project, drag and drop its copy of “LaTeX (Memoir Book)” into the Format sidebar of your WIP to import it.

That should be it! There are a few other very minor adjustments I made (like fixing that natbib typo), but nothing worth bothering over.

I considered adding some sample material, but I’ll have to save that for a future iteration.

[jpkell]

@AmberV

Two pieces of news for you

Wow, this is great. Thanks!

The bug with linking placeholders has been fixed and will be available in the next minor update (which will be soon). Consequently you should be safe in using that approach—though as noted it shouldn’t typically be necessary unless you want to create a cross-reference in a style that doesn’t match how the Markup pane is set to work.

I’m not sure I ever followed what the bug is…

The next update will also include an improved version of the LaTeX project template. I’ve…also added an “Equation” style, which can be used to create equations in the main text rather than having them in separate binder items. On the one hand it should make sections with a lot of display equations easier to work with (not broken up so much in the binder), but on the other hand the automation with cross-references and label creation depend upon section layouts. So you’ll have to supply your own label to them in the manner documented, and refer to them with typed in \autoref{…} calls (though I suppose you could make a style to handle the syntax for you, if you wanted to).[/list]

This is great. I think I personally will stick to having my non-inline equations being separate binder items, as the automated cross-referencing via “Link to Document” is really handy and quick (especially now that I can suppress the equation name–via the changes in the Markup pane–to yield just “Equation (1.1)”.

One other small item that I’m not sure can be addressed in an ideal way for me: When I copy an equation in MathType, the copied LaTex code is flanked by “[” on the left and “]” on the right, a la:

\[2 + 2 = 5\]

But your Equation binder template wants only “2+2=5”. Is there any way to allow me to just paste in \[2 + 2 = 5\] and have the flanking slashes and brackets stripped away in compile? I just ask because it would be great to not have to remove the slashes and brackets each time I copy an equation from from MathType and paste into your Equation binder item. That said, it’s also not the end of the world.

[AmberV]

Is there no setting in MathType to remove that? It seems rather presumptuous to me that assume that is the type of environment you want whenever you copy an equation out of it. But what do I know, I’ve never written anything that requires extensive formulae.

Well I’d get them all pasted in incorrectly and then fix them all at once. One nice thing about keeping your equations in separate binder items is that you can run a project search for them. Search for “Equation” by Section Type, click the hook arrow button in the search results header bar, and switch to Scrivenings mode. Now you can use simple search and replace to nuke them all at once (probably make a backup before doing that, just in case!)

[jpkell]

@AmberV

One nice thing about keeping your equations in separate binder items is that you can run a project search for them. Search for “Equation” by Section Type, click the hook arrow button in the search results header bar, and switch to Scrivenings mode. Now you can use simple search and replace to nuke them all at once (probably make a backup before doing that, just in case!)

Amazing, I am continually amazed by how flexible Scrivener is.

[jpkell]

@AmberV

One nice thing about keeping your equations in separate binder items is…

Hi AmberV, I’m baaaack Today question: Is there a way to tweak the LaTex template so that references to Equation binder items use “\eqref” instead of “\autoref”? Or would a change to implement that also change all other references (e.g. refs to chapters, etc.)?
Thanks!

[AmberV]

Given autoref’s goal of providing adaptive cross-reference styles depending upon the type of element you are referring to, my first question would be: what qualities of eqref are you looking for, that you wish autoref did? Chances are you can make autoref work the way you want for equations, without changing how it works for other things.

Here’s something to get you started: Combining \autoref and \eqref.

To answer your literal question: you can see the full range of options in the Markup compile format pane. The two types of links it distinguishes between are external and internal links. It would need a much more complicated UI and implementation to change the prefix/suffix based on the Section Type of the thing the link points to.

There may be a solution that involves the Document Title Links compile format pane being enabled, so as to get prefix/suffix link adornment, then a whole wad of regular expression based replacements to basically undo the effects of that setting for most of those (so you don’t get “\chapter{Something} (\autoref{scrivauto:38})”, and more replacements to convert how some links like the above are cleaned up to eqref—but I think we’d really be pressing the definition of utility and clean output design by going down that path.

Again, I think this problem can (and should) be delegated to LaTeX, and to the hyperref package specifically.

[jpkell]

@AmberV

…my first question would be: what qualities of eqref are you looking for, that you wish autoref did?

Thanks very much. I wanted eqref because it adds the parentheses around the equation name. The trick given in the link you provided didn’t work for me, although I suspect I just implemented it poorly (due to my poor understanding of what it does).

I found a solution that works fine for me: After compiling to LaTex from Scrivener, I open the .tex document with a lightweight Tex editor named Geany. That has the ability to find/replace strings with a carriage return, so that I can find all instances of

%Equation \n\\autoref

and replace them with

%Equation \n\\eqref . (That’s the search string because I have suppressed “Equation” by using your internal link prefix trick in Compile > Markup.)

This search/replace trick works only if all my equations are titled Equation (i.e. the stock Template name) in the sidebar. Some people might find that limiting/annoying, but I’m living with it just fine. With this workflow, I am able to refer to any equation using its “parenthesized” name, adding an word before that name that I want. And that gives me all the flexibility I want.