Compile to markdown: 'do not escape special characters' potentially bugged when 'convert richtext to markdown' is also selected

Scrivener Scrivener for macOS
, ,
1

Description

I have a document written largely in markdown, with the occasional bit of scrivener’s richtext formatting. I am thus trying to use the ‘hybrid’ support for Scrivener and Markdown as discussed in s 21.4 of the Manual, where I make use of some richtext capacity but also write largely in markdown.

However, if one selects the ‘Convert richtext to markdown’ compile option, even if you do not select the option to ‘Escape special characters’ Scrivener will still escape certain key markdown characters.

In particular, the compiler automatically escapes all ‘*’ and ‘_’ characters, meaning that it is not possible to italicise or make bold text simply by writing *italics* or **bold** in the scrivener document. When I examine the .md file that the compiler is creating – and which I am then passing to pandoc – all such characters are escaped (ie, with a '\*' prefix and suffix), meaning that pandoc does not treat them as markdown when processing.

It is only these characters that are affected. Eg, markdown notation for a quote ‘>’, or a footnote ‘[]’ all work fine in compile and are not escaped.

Is this a bug? If not how would it be possible to make use of some amount of richtext features while writing markdown for the most part?

Reproduce

Create a scrivener document which includes markdown notation using ‘*’ and ‘_’ characters. At compile (to .md), select ‘Convert richtext to markdown’ but do not select ‘escape special characters’. The output nevertheless escapes these particular markdown characters.

2

You can use Character Styles to wrap the RTF inline (including binding ⌘i and ⌘b to generate the Style so you work almost identically to RTF alone), then use the style instead of plain RTF + convert RTF. . Note that Hybrid mode in §21.4 mentions: Scrivener’s powerful stylesheet system is a natural augment to Markdown style writing, and refers to lists, links, tables, footnotes etc but doesn’t explicitly mention bold and italic. I don’t remember whether * and _ were ever considered in-scope for the escaping feature when we discussed adding this (during the beta?), @AmberV will probably know…

3

As a general point, the global RTF to MD conversion switch is going to be overkill (and in the way) for anyone using a fair amount of Markdown in the source. As noted in the manual, it’s really meant more for those that are not at all fans of using the method for writing, but just want access to one or two of our more advanced technical outputs, like LaTeX or DocBook. It is meant to take a source that is oblivious to Markdown and make it look as typed—and so thus escaping just about everything that might be confused with syntax.

The secondary checkbox you refer to isn’t meant to be a “let me use Markdown anyway” flag. It might allow some to pass thru, but that’s not it’s intention. It’s meant more for people who must put some other types of markup into their text, such as citation markers for post-processing in Zotero or whatever, without interference.

That said, I do agree there are some things we could probably do better with this flag off, and I have noted some issues with both the flag on and off, where some things that should be escaped aren’t, like asterisks at the beginning of the line, which a writer might use as a casual footnote, and end up getting a bullet list instead.

Is this a bug? If not how would it be possible to make use of some amount of richtext features while writing markdown for the most part?

Here is an older topic on this matter: Turning on “Convert rich text to MultiMarkdown” breaks MMD. There is no one right best way though! Scrivener gives one a wide variety of tools from which they can build a workflow that matches their preferences. You can go from using RTF conversion with a little markup here and there via styles set to pass-thru as raw markup. You can make heavy use of styles and section layouts to do a lot of the heavy-lifting with syntax for you, to the point of pretty much not having any in the editor at all. You can mix and match automatic generation with manually typed in stuff.

There is no right way to do this, and exploring the option in Chapter 21 and picking which are right for you is really the best I can suggest. For use cases, check here on the forum, we’ve got lots of ground covered in terms of how the software can help author Markdown texts.

Also be sure to check out the user manual source itself, which is itself firmly in the “hybrid Markdown” arena. I have since gone back to using Markdown lists (because lists rich text lists sound good on paper but in my experience with them, it’s a major point of regret in using them in that project), but otherwise it’s mainly cross-references where you’ll find syntax, and that’s because I strongly prefer internal links as a writing tool rather than a shortcut for generating syntax. It really does not bother me one bit to throw a couple of square brackets around a link I mean for the reader to have access to as well.

Overall my advice would be to turn this checkbox off entirely. And start from this from the perspective of: okay, I’ve got this formatting in my editor, how can I tell the compiler to convert it for me (or is this maybe just easier to type in). Over time you’ll build up a library of techniques that you can apply to future projects.

1 Like