Scrivener/MMD/Latex

I’ve been using Scrivener for a few months now. I’m just beginning to learn about TeX/LaTeX/BibDesk/LyX – really just starting. I assumed that the way to go would be to import RTF formatted Scrivener documents into LyX. That seems not to be so straightforward or simple as I assumed.

It occurred to me that Scrivener might be able to export or compile to LaTeX. A couple of searches made me aware of MMD and the possibility of compiling MMD-coded documents to LaTeX. I’d just like to confirm that that is the case.

There are some things regarding this possibility that I am confused about: I gather MMD is already present somewhere/somehow in Scrivener, yet I also came across an exhange which seemed to indicate that, at least for some purposes, it may be necessary to have another separate installation of MMD. Second, from what I’m able to discern from my very brief and recent exploration – within the last hour – MMD coding seems to be simple and intuitive. Does it handle footnotes [or endnotes] and bibliography?

I could probably get the answers to these questions by continuing to search and read in the forums. I hope you won’t take offense at my asking again questions that have probably already been answered numerous times over in the forums.

Thanks,

Eric,

Nice to see you’re still getting on well with Scrivener!

Correct. The main problem in going between RTF and LaTeX is that they are both quite different, philosophically speaking. This is even more true for a system based on Cocoa RTF, which lacks internal stylesheets and simply encodes everything as font and paragraph modifications, whereas LaTeX is (ideally) purely semantic. You specify that a line is a “chapter”, not that it is 24pt bold, but that is all the RTF specifies. Since there is no standard for what a chapter title should be styled like, that makes writing a converter difficult.

MMD is the easiest way to get a LaTeX file out of Scrivener. There are some who also like to just compose raw LaTeX in Scrivener and then compile as a plain-text document, but that requires a good familiarity with the LaTeX syntax, and the tolerance to stare at all of those codes while you write.

Both of those statements are correct. It depends on how far you want to take customisations. If you have no intention of learning Perl or XSLT, then you’ll be just fine using Scrivener’s built-in version. Those looking to install their own custom copy are looking for a way to hack the MMD system itself to produce custom results. That’s the more advanced end of the spectrum. For simple usage, just follow the (Multi)Markdown documentation for marking up files, and compile out of Scrivener using the MMD->LaTeX format, using LyX or some other tool to polish it off. Not too different from a word processor workflow, really.

One other reason to upgrade, which may or may not interest you, is that MMD is now up to version 3, officially. Version 3 is a whole new engine under the hood—one that is significantly faster than version 2. So you might be interested in that. The speed difference really is amazing. The Scrivener documentation currently takes about 90 seconds to go from a raw MMD file (pre-compiled from Scrivener) to a .tex file. With MMD3, it takes about 0.5 seconds.

The only real catch is that Scrivener is not yet fully MMD3 aware, so a little rewiring might be necessary. I have not yet myself tried to hook up Scrivener and MMD3. I have too many legacy documents right now which are using such a heavily modified MMD core that I want to wait until I have some breathing room, in case the upgrade doesn’t go smoothly.

I really should do that in a test account though, so I can answer this question better in the future.

On that I would agree. I think MMD gets a reputation for being more complicated than it really is—in part because it attracts the tinkerer crowd (of which I am guilty of being a part of), and so a lot of discussion on this board revolves around a level of customisation that the average person really doesn’t need to mess with. Scrivener+MMD->LaTeX->LyX is by itself hardly more complicated than using an ordinary word processor. It’s just a little more unfamiliar.

I myself have never used the bibliographic features of MMD, but do check out the documentation for it, and this forum, as I know I’ve seen BibDesk and MMD in discussions around here.

For footnotes though: just use Scrivener’s footnote feature. As with section titling, it will handle the conversion to MMD’s footnote syntax for you. It will also do that for embedded figures. What it won’t handle are bullet lists (you’ll need to type those in using MMD’s basic method), tables, and of course any other RTF based formatting.

If I’m not mistaken, the compile to MMD output from Scrivener also converts bold and italics to MMD syntax, in addition to transforming Scrivener footnotes.

Not in the compiler, you might be thinking of the Format/Convert/Bold and Italics to MultiMarkdown Syntax. It’s a one-shot static process, mainly because the conversion from RTF bold and italic is often not very “clean”, producing odd asterisk placement that won’t work.

Ah, thanks. Good to know for when I (eventually) start publishing blog entries using MMD.

Actually, I haven’t been spending much time with it lately, Amber. But that’s only because I haven’t been writing. Pull of other responsibilities. I’ve taken on several new pieces of software over the last several months. Scrivener is the one I know for certain I’ll be sticking with. It is so supportive of writing, i.e., the actual composing and editing, so powerful, and so intuitive. And the forums are always helpful!

Thanks for the explanation. Makes sense now. At this point my reason for upgrading MMD would just be to take advantage of the improvements in 3.0. However, since my needs and usage are for now at least probably going to be limited – I’ll start working with simple documents and work my way up from there – I think I’ll take a pass on upgrading for now. Perhaps by the time I’m ready to move up Scrivener will have incorporated 3.0.

Starting fresh, that would not be a problem for me. However, if I were go ahead and start working with the version that’s in Scrivener now, might I be creating a “legacy” problem for myself, or is that only a consequence of the “heavily modified core” you’ve been working with?

Glad to hear that. My first experiences with word processors were Wordstar, Perfect Writer, and in the years before I finally gave in to Windows, Zywrite. All involved putting code in the text, though Zywrite had a way to hide it so it didn’t clutter up documents so badly. From what I’ve read about MMD so far, it sounds not only similar, but simpler and more natural – less distracting from the writing process.

I’ve read much of the documentation. Seems pretty clear. And sounds like Scrivener will do most of the coding I’ll need. [My documents always have as little formatting as I can possibly get away with, with footnotes, citation, and bibliography being the most complicated parts. Suspect when I get into integrating BibDesk, with which I have no experience, and using it to do citations and bibliography, I may need a little hand-holding .

One thing I don’t see in the manual is any description of the syntax itself, or any reference to descriptions of it. It must be out there somewhere.

Thanks again for the helpful response.

You’re fine. The basic document syntax is the same, so a file composed now will work fine in the future, too. The main difference at that level is in the Meta-Data block (which you set up in Compile), but even then, unless you are doing something custom there aren’t any changes to that segment which will impact you.

My problem is that I’ve heavily modified the scripts that handle the .md files. So to upgrade I have to go through and figure out where to insert all of my custom stuff again.

For most stuff, there isn’t even any syntax at all. Maybe a block quote here and an italic phrase there, and in my opinion it’s easier to type in "> " at the beginning of a line than using a pull-down menu to select a “Block Quote” preset—for italics it’s no more bother than the usual way. Shift-8 instead of Cmd-I on each end of the italic phrase. So yeah, when it comes to actually typing in this stuff, the biggest difference is hitting enter twice at the end of every paragraph.

Correct, it seemed a bit wasteful to re-print the Markdown and MultiMarkdown guides. You do have a good point in that these guides are not clearly stated in the manual near the top. Here are the links:

  • Markdown: The basics. This is the core that MultiMarkdown builds from, and contains all of the basic formatting you’ll need to know.
  • MultiMarkdown (PDF): Full MMD documentation, for the extra syntax it provides, Ch. 4 is what you want.

Thanks for the clarifications on the other points, Amber. Seems like MMD has almost been default all along without my knowing it.

The stuff about “syntax” at the Markdown site is what I was looking for. Unfortunately the site doesn’t make it available in a format that can be printed off-site – or bought as a printed document/manual/book. I did find a compact two-page cheatsheet that wasn’t easy to get a copy of. [This site scribd.com/doc/28031525/Mark … heat-Sheet requires you to give them access to you facebook profile data in order to get a copy :open_mouth: ] I was eventually able to get a copy, though. It would be helpful if the MultiMarkDown site provided something like this.

That it can be pretty much distilled down to a two-page cheat sheet indicate the simplicity, even transparency, of the “syntax” – as you say it’s not really very syntactical. My sense is it won’t be long before the cheatsheet is dispensable.

Thanks again,

A version of the Markdown cheat sheet which is not not locked up by the abomination called scribd can be found here:

hw.libsyn.com/p/8/3/3/8339a864bb … _Sheet.pdf

Thanks for the link. And a big amen to “the abomination called scribd.” I couldn’t believe it when they wouldn’t let me print or download the cheatsheet, which I gather they had no role in creating. Utterly amazing.

A note on the second page of that PDF, though labelled “PHP Markdown Extra”, all of these with a few exceptions are applicable to MultiMarkdown as well. Here are the differences:

Header ID: This is applicable, however the method in MultiMarkdown uses square brackets instead of curly brackets. Here is an example of how you would use this to cross-reference work within the same document:

See also: [Chapter][chap-example].

## Some Chapter Name [chap-example]

The only time you need to use override IDs are to shorten cross-references in the case of long titles, or if there is a title conflict. For example in the Scrivener manual there are cases where sub-sections have the same name, like “Format Bar”. In order to cross-reference to both cases individually, one of them needs to have a custom ID. Without using an override, you can cross-reference by just typing in the name of the section. In the above example, without the override, it would be [Some Chapter Name][]. You can omit the second part of the cross-reference if the visible text is the same as the section name.

The advantage of using the real section name in Scrivener is that you can easily get automatic Scrivener Links if you turn on the [[Scrivener Link]] detection in the Auto-Correction preference pane. Just type in your cross-references like this: [[[Some Chapter Name]]][]. As you type that in Scrivener will find the “Some Chapter Name” binder item, link it, and remove the superfluous brackets. Now you have a link both in your working draft and a cross-reference in the final product (which by default will look like they do in the Scrivener manual, with a parenthetical mention of the section, it’s section number, and a PDF link to that section, making it useful for both digital and paper use).

The other difference is that you shouldn’t supply a hash in front. It will do that for you.

Footnotes: The example shows the use of a numeral, this should be avoided in MMD as the footnote reference will be used as the HTML ID, and HTML IDs cannot start with a numeral. This isn’t a huge thing to worry about unless you type in your own by hand. If you use Scrivener’s footnote feature, it will assemble all of the footnotes for you, automatically assign IDs, and generate the appropriate referencing. Scrivener uses [^fn#] for inline notes, and [^en#] for linked notes. This makes no difference to the parsing engine, but makes it possible to hook into the system and use separate methods for each type—essentially give you two footnote streams. The difference can be totally ignored for most usages.

Abbreviations: Not supported in MMD.

I’ll respond to your last response later, Amber. I was just checking in to report that I compiled my first MMD to Latex export, loaded it in TeXShop, and typeset it. There were a couple errors – files that didn’t load – but the typeset went through.

The result was kind of a mess, but that’s because I didn’t do anything to set the document up for the compile. I gave no thought to what formatting, appropriate or inappropriate, it contained. I imagine it would have looked a little better had I have given some thought to how comments and footnotes should be handled. But for knowing nothing, giving no thought to what I was doing, just mechanically, stupidly, going through the compiling steps as little as I understood them, it came out pretty well. I recognized the result.

I can see I’m going to have to learn some LaTeX. There were things about the formatting that I didn’t like – e.g., paragraphs were distinguished by indents, while I prefer no indents and a blank line – but the LaTeX generated by the compile was totally Greek to me. Maybe I’ll load it in LyX and see what it looks like there.

I will check back in later on your comments on the cheat sheet.

Thanks again,

I did it. It wasn’t so “Greeky.” Maybe with LyX’s help I can learn what I need to learn. Maybe I’ll be able to do this. Goodbye wordprocessors! :smiley:

I’ve been a bit late in getting back to this, Amber, but thanks for the clarifications. Three comments:

[1] I don’t understand the comments on cross references, but then I don’t believe I would ever have reason to use them. I doubt that I will be producing any book-length manuscripts. The most complicated document is likely to be a long article with notes, citations, and bibliography.

[2] Regarding footnotes, I am unclear about the distinction between “inline notes” and “linked notes.” There will be occasions in which I will be using footnotes and occasions on which I’ll be using endnotes. By chance is this the same distinction? If not, while I’ve only dipped my toe in LaTeX, I haven’t encountered anything about how endnotes are handled. I assume an APA style template would have a way.

[3] It is wonderful that Scrivener handles so much of the MMD coding. Kinda gets in the way of learning MMD, but solves a lot of problems, too, especially for an MMD/LaTeX/LyX/TeX, etc. novice like me.

Thanks again,

Cross-references are merely when you want to have something, like in the user manual, where you can refer a reader to another section of the document in a way that is meaningful and useful. When printing, these are output as the actual section number, so you can look up that location and flip to it, in PDF they are also hyperlinked automatically so you can just click on them. So yeah, if you don’t need to refer to other sections, or the document is not long enough to necessitate it, you can ignore all of that.

With footnotes, it doesn’t matter which you use in Scrivener. They will both be handled effectively identically in the end product, so you can leave that up to taste, and even use both styles all mixed up together. In this way they work the same as they would with a word processor. The footnote anchor is inserted in place of the entire inline note in the editor, or if it is a linked note—the reference is added at the end of the linked range. In both cases, the content of the note is used to create the actual footnote itself. There is a subtle distinction between the two, which is why I threw in the word “effectively” above. Scrivener uses a different naming convention for the internal MMD that is used to link an anchor reference to the note content, so you could if you needed to, use linked footnotes for one stream of notes, and inline for another stream. By default though, this distinction is ignored and you’ll in fact never even see it unless you examine the raw MMD file that Scrivener generates.

I am new to Scrivener, but I have been using LaTeX for more than two decades. I have been reading here and there in this forum about tips and tricks using Scrivener/MMD to produce LaTeX output, but before I annoy you with my possibly very peculiar little problems, let me ask first:

Is there a concise documentation/manual about using Scrivener to get LaTeX code? The question is only scratched in the Scrivener manual.

Thanks!
Heribert

There is nothing concise yet. The MMD section of the manual has been somewhat behind the curve with the rest of the manual due mainly to bandwidth and a priority scale. More advanced topics are further down on the pole. But it’s my intention to work over that section more thoroughly for this upcoming release, in large part because MMD3 is out now, and we still embed MMD2—so some documentation on how to get Scrivener working with the new system is necessary. The problem with documenting a concise path to LaTeX, or any of the other MMD methods is that they depend upon knowing what to do in terms of composing a document with MMD syntax, in addition to any knowledge needed for the compilation process. Producing a LaTeX file is by itself very easy, you just compile using “MultiMarkdown->LaTeX” and it should work out of the box—that’s all there is to it—but of course that doesn’t work to well if you toss a bunch of RTF formatted text at it, and then you get into discussing all of the things that MMD mean, in terms of how one writes within the system (like double-spacing between paragraphs) and it starts to stray from being concise. :slight_smile:

Thanks for your help, Amber.

So please allow me to ask a first (stupid?) question: I tried to compile a first testfile, and more or less everything worked, but the special latex characters were masked, e.g., in Scrivener I write a formula $a+b$, and in my latex document I get $a+b$. Somewhere in this forum I found that I have to replace clean-text.xslt.

So I installed the Multimarkdown files (V3.0.1), but in ~Library/Application Support/MultiMarkdown/XSLT I only find 6 files, one of them clean-text.xslt, but nowhere clean-text-allow-latex.xslt. And even worse, not compiling in Scrivener to LaTeX does not produce a file with a correct preamble. The latex file immediately starts with something like

\def\mytitle{Tutorial}
\def\myauthor{Anonymous}
\def\format{complete}
\part{Part 1: Basics}
\label{part1:basics}

Somewhere I read there are problems with MMD3. Are my problems due to that?

Thanks,
Heribert

That’s a pretty good question actually, because things have changed in MMD3. In the past it would always by default produce a .tex file you could immediately typeset. Now it doesn’t, unless you read a little documentation and install some support packages. This is an aspect of the new version we are looking to make a bit friendly; more on that below.

Okay, so for now though, if you are using MMD3 you’ll need to read up a bit on how it works with LaTeX. The old method worked great if you were happy with a vanilla looking document, but customising it was a chore. You had to learn enough of XSLT to make a wrapper script, find the right parameters to overwrite in order to set your own preamble—all nothing terribly difficult, but if all you wanted to do was change the font to 12pt or use a4paper, it was a lot of work, and I suspect most people just resorted to fixing their .tex files by hand every time they compiled. So the new system doesn’t even use XSLT—in fact everything in that XSLT folder is dormant and won’t be used unless you run the special mmd2tex-xslt script in the bin folder. The standard script, mmd2LaTeX.pl that Scrivener runs via the compile system uses the new MMD3 system which just goes straight to LaTeX. No converting to HTML and then parsing it as an XML file with a complicated and slow technology for doing so.

The missing ingredient is in the new LaTeX methods. Since it goes straight to syntax, there is no “middle-spot” where you can easily customise and add in the a4paper or what have you. To address that, it has a built-in mechanism for utilising \input{} commands. You can either compile into a support folder that has the .tex pieces ready to include, or put your boilerplate headers and footers into the your ~/Library/texmf/tex/latex/mmd folder (which you’ll probably need to create). The default installer does not supply you with sample boilerplates, but they are easy to install. To use them, you’ll need meta-data that looks like this:

Latex Input: mmd-memoir-header ((YOUR META-DATA GOES HERE)) Latex Mode: memoir Latex Input: mmd-memoir-begin-doc Latex Footer: mmd-memoir-footer

As you can see from the link above, -memoir- is just one choice you can make. The LaTeX Input meta-data key does just what you’d expect. It inserts an \input{} command, supplying the provided filename. It is then up to LaTeX to include the support files during typesetting. The order of these are important. They will be inserted into the final document in the order you place them in the Meta-Data block. Right now this is a little clumsy in the compiler because if you mess up you can’t move a meta-data key to another position in the table. We’ll have a beta build out soon which will fix this.

Among other things, one of the notions we are addressing for Scrivener’s full MMD3 integration package is a front-end to all of this. A drop-down that lets you pick the LaTeX document class, or type in your own boilerplate .tex code into some header/begin/footer documents, right in the compiler—and then have the compiler insert the necessary meta-data for you. Right now you’ll need to read up on how to use MMD3—but it is perfectly compatible with Scrivener. I used it to produce a nice little customised .tex document for the integration design ideas.

The XSLT method and its alternate scripts to use it are retained because the new system, while it makes it much easier to make simple changes to the .tex container text, strips away all ability to customise the actual syntax. With XSLT, you could intercept the default for including a graphic and change it to emit icons of butterflies all over the page, if you wanted. So if you need that kind of flexibility, the system is still there as an optional method.

A postscript on how anaemic the MMD folder looks in your Library—that’s because its sole function is to operate as a compatibility layer between programs like Scrivener which assume MMD2, and the new system which is binary, and installed into the UNIX sub-system. That’s why there is not much note there. It also serves as a platform for customised XSLT usage. If you do need a bit more than just preamble and other document container adjustments, then installing the Support package will give you the infrastructure necessary to do so with minimum work.

Thanks a lot Amber. I successfully compiled a test file to LaTeX. I agree with you, it is very simple to use indeed.

Again I agree, fiddling with xslt files is not always fun. The new method is very flexible and convenient, in particular if you have two decades of LaTeX experience.

That will be very welcome! In particular since right now, Scrivener seems to (wrongly) re-arrange the order of the meta-data.

One question still: How do I get special latex characters (such as $, , _) into my latex file? With MMD2 I had to use some form of clean-text.xslt. How do I do it now?

–Heribert