Help with LaTeX snippet workflow

Scrivener Markdown & LaTeX
1

Dear Colleagues,
I could use some advice in restoring my Scrivener-MMD-LaTeX workflow, which got messed up when I recently updated to Scrivener 2.3 and MMD3.

Under my old arrangement (earlier version of Scriv. and MMD2), I wrote my LaTeX content files in Scrivener, using a fair amount of LaTeX code. If I wanted to code a table, figure, or other big chunk of code (usually several lines), I would enclose it in <!- -->. When compiling, I added the LaTeX XSLT line to the metadata, referring it to latex-snippet.xslt. The result of the compilation process is that it would produce a clean content.tex file, wherein all my LaTeX code (including natbib \citep{} commands, etc.) came through cleanly and all Scrivener footnotes were converted correctly into LaTeX footnotes. Then I would set up a LaTeX structure file which contained the preamble and everything else, while calling my Scrivener-MMD compiled content.tex file using the \include{content} command. This allowed me to enjoy Scrivener as my writing environment while being able to tweak LaTeX formatting without having to re-compile all the time.

That happy bubble burst when I updated Scrivener and MMD after some time away from writing. Now when I try to compile in Scrivener, all of the LaTeX commands in my content file get messed up, as commands like \citep{Kitschelt:1992bg} get converted to \textbackslash{}citep{Kitschelt:1992bg}.

My first solution was to enclose the whole document in <!- --> to protect the LaTeX commands within. That solved the problem of converting \ to \textbackslash but came at the cost of the footnotes, which were no longer processed correctly. I suppose I could go through and manually recode the footnotes using \footnote{} rather than Scrivener’s footnote tool, but I’d rather not have to.

So, wondering if anyone could offer some advice in getting me back to where I was before things went haywire: cleanly passing snippets of unmolested LaTeX code using Scrivener & MMD. Any help would be GREATLY appreciated!

For the record, I’m using Mac OS 10.6.8, Scrivener 2.3.1, and MMD 3.7 (including the additional MMD 3.7 support files)

Thanks,
Rob

2

First let’s see if we can’t get things working with the new system, because MMD3 is the present and the future, and MMD2 is the distant past at this point.

My initial question would be: why are you trying to hardcode the citation markings in the first place? MMD already has a good syntax for this that is easier to read: [b][][#Kitschelt:1992bg][/b]. Wherever that code appears is where MMD will generate [b]~\citep{Kitchelt:1992bg}[/b] for you. No need to use HTML escapes and all that.

But, if you already have hundreds of these typed in, just go to Replacements in the compile pane and have it fix these all up for you. No sense in typing in HTML comments around each and every one of these yourself! A search for, [b]\citep{$@}[/b] replaced with, [b]<!--\citep{$@}-->[/b] should work. The [b]$@[/b] lets you store anything between the string of text before and after it, and then print the contents of that variable in the replacement field.

The second point you have is to achieve a snippet mode. This is actually automatic now. MMD will switch to full document mode if any document-based meta-data is declared. So if you give something a title, or an attribution, then it considers the source file a “document” and runs in full output mode. If there is no document meta-data, then it uses snippet mode.

So it sounds to me as though you just need to visit the Meta-Data compile option pane and clear out any meta-data you don’t actually need.

You may also be interested in the “Custom” setting in the LaTeX Options compile pane. This lets you type the preamble, document beginner and footers into your compile settings. Scrivener will generate these three fields as .tex files, and link them up appropriately to the main content file. It may be a useful trick as you can then save this as a compile preset, and that preamble will always be included as an integral part of the format when you select it for future projects.

If these tips don’t help you out, we can see about installing an old legacy version of MMD2. You’ll never be able to automate straight out of Scrivener any more, but you can compile to a plain MultiMarkdown file and then use the MMD scripts to convert the document to .tex.

3

Thanks, AmberV - you’re the best! In fact, I think you were the one that helped me get set up with my old workflow a few years ago.

With some of your suggestions, I’d say we’re about 80-90% of the way to a good working solution here. To answer your first question, I have a dissertation’s worth (and then some) of \citep{} and similar commands that aren’t easily converted to the MMD syntax. So I’d prefer to stick with those if possible.

I tried your suggestion of replacing with That worked for all my \citep{} commands, but then I realized that I have all sorts of other LaTeX commands that needed similar replacements. So then I set up a bunch of replacements to get me through the commands:

replace \ with
replace { with
replace } with

That seemed to work most of the time, especially when I had more complex commands with multiple embedded commands like \textbf {Table \ref {tb:ECONdv } about here.}

OK, not bad. Then I realized that there’s a few more symbols out there (mostly in tables) that need replacing too: @, &, >, _, etc. Replacements seemed to work OK.

But then I realized that there are a couple of important symbols that are dual-purpose. For example, sometimes I use % to mean “percent,” and sometimes I use it to separate columns in the table environment. Similarly, sometimes $ means “dollars” and sometimes I need it to signal the math environment. The replacement strategy will only get it right part of the time, depending on usage.

So for compiling of basic LaTeX commands, the replacement strategy I outlined above works (and I cleared out all the metadata as suggested). But it doesn’t work so well for more complex things like tables. I use to envelope the entire table coding in a single set of but like I said, that doesn’t work anymore.

Not sure if there’s a viable way around this. I suppose the alternative is to go back through all my writing and hard-code all the footnotes into \footnote{} and then simply compile as a text document. But if you can think of anything, I’d be most appreciative…

4

Have you considered the MMD2 option? I still use an old MMD2 installation for the Scrivener user manual, because I haven’t had time to port all of my old post-processing methods to the newer system. As with my case, yours strikes me as being one of those times were it makes less sense to spend fifteen plus hours working around limitations in an upgrade with complex replacements and such, when the alternative is a couple of extra steps to use MMD2 instead of the built-in scripts single-click scripts.

5

I suspect you may be right. Two questions in that case:

  1. Do you know where I can download MMD2? I couldn’t seem to find it on Fletcher’s site. Are there any tricks to installing it and/or uninstalling MMD3?

  2. If I go back to MMD2, do I also need to go back to an earlier version of Scrivener to make it work? I seem to recall that most of my troubles started when I updated my Scrivener. I guess that’s a good warning to keep in mind: if it ain’t broke, don’t fix it…

Thanks,
Rob

6

There is no problem running a new version of Scrivener and using MMD2, it just won’t be integrated, so you have to use the plain “MultiMarkdown” choice. When that is selected, a “Compatibility” compile option pane will appear, and you’ll want to check the box in there to use MMD2 output. This will cause the output to be nearly the same as it used to be, before the 2.3 update to Scrivener.

So after you compile you’ll have a text file. Just use the provided command-line tools to convert this to .tex.

You can download MMD2 here:

github.com/fletcher/MultiMarkdown

It’s best to install that in a ~/Library/Application Support/MultiMarkdown folder.

7

Hmm…

  1. I downloaded the legacy version of MM2 put it in ~/Library/Application Support/MultiMarkdown and removed the MultiMarkdown foldter with MMD3.

  2. Exported from Scrivener using the MMD2 option in the compatibility pane. I also added LaTeX XSLT to the metadata, pointing it toward latex-snippet.xslt.

  3. This produced an MMD text file that had LaTeX XSLT: latex-snippet.xslt at the top, followed by my text. At the bottom are all the [^fn1]:, etc.

  4. Ran it in the terminal using the following command: multimarkdown -t latex content.txt > content.tex

  5. This produced a .tex file that had some funky stuff going on:
    a) The top line is \def\latexxslt{latex-snippet.xslt}
    b) all the \ have been converted to \textbackslash, etc. In other words, the snippet part seems to have failed to do its duty.
    c) but the footnotes were converted correctly into LaTeX

One last strange thing I noticed: when I checked the version of multimarkdown using multimarkdown -v, it came back with “peg-multimarkdown version 3.7” Is it possible that I didn’t uninstall MMD3 correctly?

Unless there are any bright ideas, my last resort is to encase the whole document in and convert all the Scrivener footnotes to hard-coded LaTeX footnotes. The Scrivener becomes a fancy text editor, but at least it’s a pretty one!

Regardless, just want to say how much I appreciate all the time and assistance you’ve put into this basketcase. Wish I could return the favor!

-Rob

8

Oh no problem, that’s what I’m here for. :slight_smile:

You will have MMD3 installed on your system still, that is fine. It’s a little confusing because Fletcher put out a support package for MMD3 which is strictly optional. It is there for older software that still tries to execute MMD via the Application Support/bin folder (as Scrivener did for a while), and it is also there for those that would like an XSLT workflow. If neither of those apply, it is thoroughly unnecessary. The actual MMD3 is binary, which is why it is so much faster, and installed into standard the UNIX utility location, /usr/local/bin.

So when you type in ‘multimarkdown -t …’ you are executing the binary version. What you need to do is run the scripts in the Application Support folder you just extracted.

Easiest way to do that is to just get your shell session in that bin folder, and then run something like:

./mmd2LaTeX.pl content.txt

Wait a bit, and then you’ll see a ‘content.tex’ file appear in that bin folder.

If you’d rather not use the bin folder as your workspace, another route is to locate your shell session in the work folder, then drag and drop the mmd2LaTeX.pl script from a Finder window into the Terminal window, and then finish typing in ‘content.txt’.

So in summary, you’re still running against MMD3 at the moment, which is why things are haywire.

9

Alas, the fates must not be aligned :confused:

Did as you suggested. After compiling to MMD in Scrivener and using latex-snippet.xslt in the metadata, the following comes out in the content.txt file (as an example):

\subsection{The Effect of Economics: Measuring the Collapse}

Running it through the terminal using mmd2LaTeX.pl converted it as follows:

$\backslash$subsection{The Effect of Economics: Measuring the Collapse}

I tried using clean-text-allow-latex.xslt just for kicks. That passed through the latex commands fine but messed up all the quotation marks and apostrophes. So then I tried replacements in Scrivener with `` ‘’ and the like, but no luck.

Not sure whether it’s time to throw in the towel on this. Grasping at straws here, but is there any way to purge my system entirely of MMD3 and go back to an older version of Scrivener? I still have a copy of 2.2 on my system, and my original workflow worked with that. Needless to say, I rue the day when I clicked “update!”

10

I’m not sure how you got converted headings like that, prior to post-processing. They should look like:

#### The Effect of Economics: Measuring the Collapse ####

For a sub-section with no base header level offset. Are all of your headings typed in by hand as well? Normally Scrivener would generate the above syntax for you based on a file located in level 4 of the outline, so like this:

# Top level folder; treated as Part #

## Treated as chapter ##

### Treated as a section ###

#### Subsection ####

For a binder outline that looked like:

Draft
   Top level folder; treated as Part
        Treated as chapter
            Treated as a section
                Subsection

Do you have things set up differently?

If you are compiling with “Compile For” set to “MultiMarkdown”, with the MMD2 compatibility flag checked, and then run that output file manually through the MMD2 Perl scripts in a shell, you have effectively already purged MMD3 from your workflow. I do this same thing myself several times whenever there is a user manual PDF. Scrivener initiates no post-processing when using the plain “MultiMarkdown” compile format. It just makes a text file and generates a little MMD where asked to, such as in header titles, footnotes and images.

But you wouldn’t have to completely remove MMD3 from your computer, because 2.2 is entirely unaware of it. Just make sure to rename any Application Support/MultiMarkdown folders so they won’t be used, and it will go back to using the built-in version in 2.2. I would recommend backing up your project first, before opening it in an old version. It should be okay, but since it has been a long time since that version was released, it would be wise to make sure you have a pristine copy last opened in 2.4.