Potential incompatibility between Scrivener and MMD3

I am experimenting with MultiMarkdown 3, which has a much more direct translation to LaTeX, bypassing XSLT altogether, which is nice. However, MMD3 has also changed policy on metadata in LaTeX translation: the new policy is that “Metadata is used in order, so the order and placement of the LaTeX Input metadata fields is important”. This results in a potential problem when using MMD3 with Scrivener.

I set up the Meta-Data fields in Scrivener as directed in the MMD3 documentation. When I compile to MMD, I get the following metadata block:

latex input: mmd-article-header
Title: Whatever Title You Like
Base Header Level: 2
LaTeX Mode: memoir
latex input: mmd-article-begin-doc
latex footer: mmd-memoir-footer
Format: complete

Which reflects the order in which I manually placed the fields in the Meta-Data dialogue, and everything works fine. However, when I go to recompile the next time, and open the Meta-Data dialog, the fields have been rearranged into alphabetical order by Scrivener. The compile to MMD now looks like this:

latex input: mmd-article-header
Base Header Level: 2
LaTeX Mode: memoir
latex input: mmd-article-begin-doc
latex footer: mmd-memoir-footer
Title: Whatever Title You Like
Format: complete

I do not know why the metadata comes out in the particular order it does in the resulting MMD file, I just know it’s a different order. The result is that the Title information is in the wrong position in the LaTeX file (notice it now follows footer in the metatdata), and is not picked up for inclusion by the \maketitle command. I can easily imagine further problems arising if MMD3 is counting on a particular metadata order, and Scrivener is rearranging it.

I guess the simple question is, is there a way to lock the order of Meta-Data fields in Scrivener? Or is this rearrangement an inevitable result of using a .plist to store this data?

If the fields cannot be locked, I guess I’ll take this to the MultiMarkdown community for comment.

And as usual, since I’m so new to this, maybe I’m doing something weird, or not understanding properly… Always a possibility! At this point I’d just like to know that someone can reproduce the problem.

I don’t think there has been much MMD3 chat on this forum yet, but since this is the way that MMD seems to be going, it might be useful to start a thread or two.

Cheers,
Mike

[snip] It does seem to me that the issue is at the Scrivener end, as it’s just a simple matter of letting the user lock (and freely rearrange) the items in the metadata list.

Yes, I don’t recommend using the Meta-Data pane in the compile interface with MMD3 right at this time. Copy and paste the meta-data in its correct order into a file at the very top of your Draft (or relevant compile group—it needs to be the first file compiled) and call it “Meta-Data”. When such a file exists, Scrivener will use it to supply meta data. It will also not automatically add the Format:complete bit, which you don’t need. The way it works is by tacking on the data to the end of any meta-data in the Compile meta-data pane, so for this to work correctly you’ll need to delete the keys from that table. Because it just puts in the contents of the top text file named “Meta-Data” it will simply be whatever you typed in.

We are working on MMD3 integration and compatibility right now, and fixing up the Meta-Data pane to support the new requirements of version 3 is already on the list of things to address (such as drag-and-drop to reorder keys and to always preserve their order). So this will only be a temporary work-around.

That’s great to hear! I can only imagine how difficult it is to react to innovations in companion packages, while retaining back-compatibility etc. But I can’t express enough how wonderful the Scrivener → MMD → LaTeX workflow is for me. Many thanks for the continuing support…

Cheers!
Mike

Regarding MMD 3 integration and compatibility…

As I understand it, MMD3 uses the “latex input” metadata to find LaTeX support files like “mmd-memoir-begin-doc” which are stored in ~/Library/texmf/tex/latex/mmd, and then adds them to the final tex output using the /input{} command.

Is there a way to include the required LaTeX support files directly in the Scrivener document?

Since the support files control the final appearance of the compiled document and I customize them on a per-document basis, I would like them to travel with the Scrivener file. That way the Scrivener file would be self-contained if I needed to give it to someone else or move it to a new computer.

Perhaps there is a way to achieve this now which I am not aware of, otherwise consider this a feature request!

Thanks

Ioa? :slight_smile:

Yes. :slight_smile:

Mike B: You’re welcome! Keeping Scrivener up to date with MMD is a point of interest to myself as well, as I use the system extensively for nearly everything that I write. So I’ll continue to do my best to keep abreast of changes and likewise to not alienate those who can’t upgrade yet for whatever reason (because I’m one of those as well, some of my projects have heavily customised MMD workflows and it would just take a lot of work to switch them over to new system—but I’m adopting it for new projects here on out).

whaynes: There isn’t a way to do this right now from within Scrivener, as you note. One of the interface designs we are working on includes a way of addressing this and making it easy to work with. This is all very tentative, so I hesitate to say anything concretely, but it would be nice if you could:

  1. Easily specify a starting class using the samples provided by Fletcher. From a drop-down menu in Compile, select “Memoir” or “Article”, and not even have to bother with any of the meta-data keys required to set these up. This would make upgrading a snap, and keep the system approachable for new years while also being convenient for experienced users. If you want to change the document class, just flip a switch rather than changing three or more meta-data fields. This would place the boilerplate files into the compile folder for you, which will make it easier for people who are not familiar with customising LaTeX in the support folder.
  2. Make it easy to add your custom input files. I’m thinking here of having the interface designed so that you can type into three text fields in the Compiler to provide a header, post meta-data block (like mmd-memoir-begin-doc), and a footer. Anything you type into these will be placed into three corresponding .tex files when you compile, right into the compile folder, then dynamically linked and set up in the meta-data block, so that you can typeset them immediately after compiling.

In both cases this will help keep clutter out of the meta-data table. Scrivener will assemble and position the appropriate keys for you, leaving only the primary document-specific meta-data for you to define.

On the last point, I did toy with the notion of making it so that you could put these support files into the Binder somewhere, and then specify those files in Compile—but I like the idea of having it in the Compiler, because that means they can then be saved as a compile preset, and applied to future projects with a single action. If we went with putting the custom files in the project and linking to them from the binder, it would create compile presets that “broke” if you tried to use them on a new project without those support files in place. A compiler preset shouldn’t heavily depend upon the content of the project. They are meant to transcend it.

But like I say, standard caveat, this is all still in the early design phase. If you have any comments or see any flaws in the above, now is a good time to point them out. :slight_smile:

Hi, I notice the order of metadata bug is still around in v. 2.1. Is there an ETA on this being fixed?

Could you provide an example of what you mean? I’m not sure what the metadata bug is.

Just what was discussed above: that if I order the elements in the “Compile”->Metadata list, so that e.g. the Title element comes before my Latex Header element, this order is not preserved in the next call of the compile command. Which is a problem because in MMD3 the \title is referenced in the Latex Header and must therefore be defined first in the Tex file.

I’ve used the workaround of a Meta-Data document so it isn’t a showstopper for me.

Hmm, I can’t reproduce this. I did a test with Title, Author, and Base Header Level, shuffling them around and then compiling and each time the resulting MMD file uses the precise order I had set in the meta-data pane. Does this happen for you in a test blank project, or just in one particular project that you are working in?

I reproduced it on a blank project with Scrivener 2.1. Select MMD->Latex, add a new custom entry to metadata called e.g. Latex input. Add it between Base … and Author. Compile. Then try again, this time dragging the new entry to the end of the list. Compile. Then the next time you go to compile, Title and Author are at the end instead of the custom entry.

Oh, if I understand you correctly, this shouldn’t be a problem. The important thing is that the meta-data keys are in the correct order in the base MMD file itself. That is why I tested using plain MMD compile. That is what the MMD engine will use to construct the .tex file; what it does with the keys within the .tex file afterward is its own domain—so if you are running into a case where keys are in the right order compiling to a plain MMD, and messing things up in the .tex file after post-processing, than that would likely be an MMD bug you should report to the github area. But like I say, I don’t think it matters if the [b]\def\mytitle{Stuff}[/b] lines are shuffled in the output .tex.

MMD files just parse metadata in order. That makes sense to me from a separation of concerns POV: MMD shouldn’t be trying to guess what should be done and when. I think Scrivener should output the metadata fields in a user-specified order.

See https://github.com/fletcher/peg-multimarkdown/wiki/How-do-I-create-a-MultiMarkdown-document%3F - the Latex input section.

Here is the problem I have:

MMD3 allows you to specify a Latex Header/Footer. It does this using the \input command and finding that .tex file. This just ‘pastes’ the header.tex file into the generated doc.tex file.

Now if I have Title after the Header, then if my header says \title{\mytitle}, it will crash because \title isn’t defined.

Like I said, it isn’t a big deal, and your solution for a Meta-Data file is probably a better way to do it. But I thought I should explain what I was seeing.

In other words, Title must come first. Now, in Scrivener 2.0 it would preserve the ordering, and no problemo. But in 2.1 for some reason the ordering doesn’t stick, so I have to keep moving the elements around.

Ah ha, I see what you mean. I was testing just the output, not the meta-data pane itself. I can definitely reproduce the problem where Title and Author jump to the end of the list. Thanks, I’ll write this up for Keith. And yup, for now stick with the Meta-Data file in the Draft. Do note that once this is fixed you’ll be able to quickly move that data from file format to the compile pane with copy and paste.

Just to note that I’ve fixed this for 2.1.1. The problem was that Scrivener always ensures the title and author keys stay the same even when you switch compile formats, so as not to lose project-specific data, but the way it was preserving them was resulting in them moving to the bottom of the list every time Compile was opened. Thanks for bringing it to my attention.

All the best,
Keith

Well, I hope that before it is fixed I have handed in my thesis and don’t need to look at (the very excellent) Scrivener for a good long while. :slight_smile:

How about now? :slight_smile: Grab the beta version from that thread and give it a spin. I haven’t tested this particular problem yet, but the release note state it should be resolved.