Scrivener hangs when compiling for MultiMarkDown

[jjs357]

I am using @AmberV 's suggestion to make an index. I have a long document (over 300 compiled PDF pages) that I can compile to docx and PDF formats without issues. When I switch to the MultiMarkdown → odt compile setting using the Scrivener MMD OpenOffice Document format the compile hangs and I have to force quit to recover on my Mac (running latest Sonoma and Scrivener 3.4).

Even when I compile a single chapter some of them work and some also hang the compile. When I remove the last section from a chapter that doesn’t work in my selected sections to include and just compile that chapter, the compile will often succeed. There is nothing in this last section that appears odd. It is a Headed Section with some sub-Heading2 formats in it but no sub-Heading1 formats. I might want to identify index terms to include in the index in this non-working section so this is a pain!

Can anyone suggest something to try or to look for?

[November_Sierra]

What happens if you compile only to MultiMarkdown? Let’s say a chapter you know is causing trouble.

And if that works, try the second step via Pandoc (don’t know it that’s too much of a hassle, I have it installed anyways). At least to get an idea / error message if there’s something fundamentally wrong with whatever is coming out of Scrivener.

That’s what I would do.

[AmberV]

If it is hanging, at what stage is it hanging? The progress bar will fill up, which represents Scrivener’s work, and then switch over to a stateless animation that indicates it has handed the file over to MultiMarkdown and is waiting for it to finish.

That aside, I would only add one troubleshooting steps to the above, and that is to take the .md file you compiled without conversion (assuming you can), and run MultiMarkdown on that using Terminal, to see if it hangs. In theory it should, if the compiler halts during the second phase.

And yeah, at that point maybe Pandoc-DOCX will be a better route for you to take. It has the indexing additions as well (the simple one, not the more complicated hierarchical indexing I added to ODT).

[jjs357]

Thanks for the suggestion @November_Sierra. Both straight MMD and Pandoc compiles don’t hang Scrivener (no errors are reported) but I don’t have an output form from which to create the index. At least Amber’s suggestions point to an OpenOffice doc format being the output format where index creation is possible. But maybe I am missing something.

[jjs357]

Thanks @AmberV – I have the md created. On a Mac, what is the process to “run” MultiMarkdown. I am familiar with homebrew to install something if needed. There are a bunch of figures that are part of the document: pngs and jpegs. They appear in the folder that contains the md file. Is there a way to attempt to an odt conversion on the folder.

I do see the stateless animation and that’s where things hang. So maybe I can learn to compile the compile as it were.

[jjs357]

Is there info on how to produce an index inside a pandoc-produced docx file? The pandoc compile does not hang scrivener on the chapter that gives me the hang for multimarkdown. My interest in this is to harvest index terms that I markup inside of my Scrivener source file.

[jjs357]

@AmberV

If it is hanging, at what stage is it hanging? The progress bar will fill up, which represents Scrivener’s work, and then switch over to a stateless animation that indicates it has handed the file over to MultiMarkdown and is waiting for it to finish.

The progress bar fills up partially sometimes and then I get the spinning beachball. Sometimes the entire progress bar seems to be at the end but then spinning beachball.

[jjs357]

@jjs357

Is there info on how to produce an index inside a pandoc-produced docx file? The pandoc compile does not hang scrivener on the chapter that gives me the hang for multimarkdown. My interest in this is to harvest index terms that I markup inside of my Scrivener source file.

Well I am new to this but opening the pandoc-produced docx inside of LibreOffice allows me to create an index at the end like I was able to do when MMD doesn’t hang and produces an odt file. Can I get better formatting? I see this where the page numbers carry over to a second line. Can I get the index entries to stay on a single line? I want to pass the terms (page numbers are irrelevant since the book will be typeset by the publisher) out as a file with one index term per line.

I just want to save the terms and ignore the formatting altogether.

[AmberV]

For future reference, what you can do is double-click on the special Word/OpenOffice compile Format to edit it, then click the above the left sidebar to make it available to plain MMD as well.

But if you’re happy with the Pandoc approach, and that isn’t hanging at all, then I’d stick with that. It is overall a little nicer, more flexible than MMD. For instance…

Can I get better formatting? I see this where the page numbers carry over to a second line.

In LibreOffice this mostly comes down to styles, and the settings you use when inserting an index (honestly don’t know why you have two lines, but word processors commonly jump lines if a tab stop isn’t set up right).

Setting up styles to make the document look the way you want is how you’re going to be doing all of your formatting. The basic way of handling that is to design things as you go with the compiled output, and if you need to compile again to update content, clear the file’s contents (except for maybe the ToC and index) and import the compiled .odt / .docx into the designed copy, with the Insert ▸ Text from File... menu command. This will match style names and thus conform the compiled output to your ongoing design efforts.

You can also make a template for Pandoc, which is just about as easy as that, only you save the cleared file somewhere as .odt, and use the plain MultiMarkdown compile option to gain access to the Processing compile format option pane, where you can stipulate the command line used to convert. Something like, pandoc -o <$compilename>.odt --reference-doc=/path/to/your/template.odt <$inputfile>. The bracket codes are Scrivener’s, the first is what you put into the file save dialogue box, the second is the location to the temporary .md file it creates.

Both are the same to be clear. One saves you a little time every time you compile, the other takes a little setup initially—so which to use depends on if you intend to compile once or twice and move on, or dozens of times yet.

…(page numbers are irrelevant since the book will be typeset by the publisher)…

You should ask, but they may also be handling the index formatting as well in that case. The hard work is the stuff you’ve already done with styles to mark the phrases. Getting the automated listing tidy may be more their job than yours.

---

The progress bar fills up partially sometimes and then I get the spinning beachball. Sometimes the entire progress bar seems to be at the end but then spinning beachball.

It would be nice to figure out what is going on. It almost sounds like a memory problem though, given the sporadic chaos of it. If you can nail it down to one or two sections that always cause it, I’m always happy to take a look and see if there is a bug we can squash. PM me for instructions.

[AmberV]

@jjs357

On a Mac, what is the process to “run” MultiMarkdown. I am familiar with homebrew to install something if needed. There are a bunch of figures that are part of the document: pngs and jpegs. They appear in the folder that contains the md file. Is there a way to attempt to an odt conversion on the folder.

Sorry, I missed this one before. You can install it with brew install multimarkdown, and then from Terminal, cd into the compile folder, and run:

multimarkdown -t odt -o filename.odt compiled.md

The images will be detected and integrated into the .odt file.

[jjs357]

@AmberV

Sorry, I missed this one before. You can install it with brew install multimarkdown, and then from Terminal, cd into the compile folder, and run:

multimarkdown -t odt -o filename.odt compiled.md

The images will be detected and integrated into the .odt file.

Thanks for the tip. I tried it and it works and I get a nice odt file. BUT…

There are no index terms when I choose the LibreOffice option to generate an index. For chapters I compile using the full MMD + odt options, I do get index terms when I use the LibreOffice option to generate the Alphabetic index.

Is there a multimarkdown command line option I need to use?

[jjs357]

@AmberV

For future reference, what you can do is double-click on the special Word/OpenOffice compile Format to edit it, then click the above the left sidebar to make it available to plain MMD as well.

I commented elsewhere in this thread that I was not seeing index terms when I asked for an index after first compiling for plain MMD. Doing what you describe in the quote above and then using that format when I compile for plain MMD and then converting on the command line to an odt file works for index creation. I now get the index items I expect to see.

[AmberV]

So to step back a bit, since it is useful to know about how it all works: MMD/Pandoc have no idea what we are doing here. There is no official Markdown syntax for marking index terms. However both of these tools have ways of passing raw syntax into the target format, whatever that may be. If we are making an RTF file we could insert RTF codes, or HTML, or OpenOffice or Word XML.

So if you load the compile Format that makes these and look at the Styles pane, you’ll find the indexing styles you’ve been using listed here. Copy and paste the prefix and suffix values out of that and into a text editor. Around the whole thing will be the Markdown syntax we use to tell it when to insert this raw into the final output, and everything in between those backticks is the XML Word or Libre will recognise as indexing codes.

When you insert an index, it goes through the whole document looking for those codes, and builds the list.

And this is how we can add “features” to the compiler that neither Scrivener nor Markdown support. In a sense this is how all Markdown works, in that you use a simple interface (text markings) to generate raw word processing formatting. We’re just using styles for that, since Scrivener makes that so easy.

So that’s why you need these two formats, the ones for ODT and DOCX, because only they have the necessary XML for the styles. The “Basic MultiMarkdown” format you were likely using before does not have those, because it is meant to be used for all kinds of files. The LaTeX compile Formats do have indexing for these styles, but they of course insert the raw LaTeX syntax for doing so.

[jjs357]

@AmberV

And this is how we can add “features” to the compiler that neither Scrivener nor Markdown support. In a sense this is how all Markdown works, in that you use a simple interface (text markings) to generate raw word processing formatting. We’re just using styles for that, since Scrivener makes that so easy.

So that’s why you need these two formats, the ones for ODT and DOCX, because only they have the necessary XML for the styles. The “Basic MultiMarkdown” format you were likely using before does not have those, because it is meant to be used for all kinds of files. The LaTeX compile Formats do have indexing for these styles, but they of course insert the raw LaTeX syntax for doing so.

Thanks for the longer explanation. But using the MMD/ODT format does the trick when I make MMD and then the command line call to multimarkdown produces the actual odt file in which I can make the index entries using LibreOffice. I like this workflow better than the Pandoc one. And the index is cleaner-looking in the odt document than in the pandoc docx document:

[nontroppo]

Sounds like you have a working option , but for completeness I did make a Pandoc filter for indexes that is an alternative workflow:

Making an index?

I’ve created a Pandoc filter that basically does what @AmberV’s compile replacements do for ODT, but with the added flexibility of supporting Typst + ODT + DOCX + LaTeX. It depends on the same two styles AmberV specifies: Index Key and Index Term — these should get compiled to \index{ancestor!parent!keyword} and \indext{ancestor!parent!keyword} (you need to wire these up in your compile format); the ancestor and parent are optional. The difference between \index and \indext are based on whether …

This steps the filter entry one step back, the markdown still contains the \index{keyword} and the filter adds the correct low-level markup when generating the output. This makes the Scrivener compile format more generic and the markdown document more compatible, you can use the same compiled markdown for ODT/DOCX/Typst/LaTeX.