Getting Markdown into Scrivener while preserving formatting

I know that Scrivener is a sophisticated Markdown editor. So I guessed it should be simple to take notes from Obsidian (which is all MD) and put them into Scrivener (keeping, of course, formatting like Mailto: and URLs and italic and bold and so on.

Alas, I can’t figure out how to do it. Straight cut and paste preserves no formatting. Importing File does not do so either. Have I missed some box in Settings that you tick to tell Scrivener to convert Markdown code to WYSIWYG ?

Wondering what people do. In Obsidian, I have some pandoc options, but none seem a good fit. I feel sure there is some obvious solution that I’ve missed.

Thanks to whomever tells me what it is!

David

Markdown is inherently a plain text format. It has no formatting. Obsidian has an engine for rendering the Markdown, but that’s not part of the underlying file.

2 Likes

Right. I get that. What I want is for Scrivener, which is Markdown-savvy, to take the plain-text file and translate its markings into the same kind of file I see when I create a Scrivener doc. I guess I thought that if Scrivener can create a MD file from scratch, it could also interpret MD coming from elsewhere.

Is the problem that Obsidian’s markup is different? Ie, that this is italic in one app but not in the other?

Anyway, my workaround is to use Pandoc to export the Obsidian note into a docx file, then paste that into Scrivener. It seems a long way around, but I guess there is no simpler way?

David

I think Scrivener may be a different kind of Markdown-savvy than what you’re referring to. In fact what you’re describing sounds more like a program that doesn’t do Markdown at all, but instead has an import filter that ditches it entirely and migrates one to a rich text workflow. That doesn’t sound terribly appealing to me, but I’m one of those people that uses Markdown to write because I don’t care for rich text editing in the first place.

But, if that’s all you want, use Pandoc to convert your source to .docx. Just be aware you’re no longer using any of Scrivener’s Markdown stuff at that point, and it will be much more like writing in LibreOffice or Word.

If you do in fact want to use Scrivener to author Markdown content, then this post may help you out.

I think maybe the fundamental point of confusion may be that some editors do a lot of cosmetic formatting of the text itself while you write, while Scrivener does not. The end result is identical though, and it also uses Pandoc (as well as MultiMarkdown) to generate documents from your Markdown source. So asterisks do the same exact thing as they do everywhere else.

It can be a lot more heavy-duty than that if you want, which may be what you’ve heard about, Chapter 21 in the user manual should help you out there, and this other post kind of breaks down some of what makes Scrivener different than most offerings. It does use some rich text conventions to generate MD, but it’s more of an opt-in thing than the standard and default way of using the software.

@AmberV — it does seem that Scrivener could do more when importing markdown to convert to RTF? Pandoc could be used in the background if it was installed for example?

@dberreby — you can use pandoc to translate a folder full of documents all in one go, so in this case whatever obsidian does to make its markdown expotable to pandoc, then pandoc to RTF/DOCX…

1 Like

That’s all true, but IMHO not the same writing experience. E.g. I find “non syntax-highlighted” (or “auto-formatted”, you know what I mean) Markdown text harder to read than it needs to be. I know what the asterisks etc. do and it doesn’t matter for the final result, but come on…

Editing Markdown in Scrivener is a little bit, dare I say it, kind of “worst of both worlds”: Using a rich text editor without the rich text formatting to write in “enhanced plain text” without the visual feedback you’d expect coming from Markdown editors.

(Sure, one could apply both to the text to achieve this, but that’s not going to survive external folder sync in both directions, which is likely the prime use case for attempting something like that.)

That only applies if you don’t use any styles (insisting to enter raw markup). If you use styles, then you get the “best of both worlds”, clear in-editor formatting and clean markdown on compilation :star_struck: In fact it is the best of multiple world, because you can also compile without markdown and still get some of the visual identification from the editor to the end document via the RTF.

Honestly, using the RTF engine as a pimped up semantic grappling hook is why Scrivener, as a markdown editor, rules the markup multiverse :stuck_out_tongue_winking_eye:

3 Likes

Sure, as I already addressed in both of the threads that I linked to, some people do not care for writing in Markdown without software changing how that process works in some fashion (syntax highlighting, typing aids, etc.). And I do get that, even I’m not myself as adamant about it.

@November_Sierra : (Sure, one could apply both to the text to achieve this, but that’s not going to survive external folder sync in both directions, which is likely the prime use case for attempting something like that.)

And you noted, how Scrivener works need not be a hindrance to adopting Scrivener into your workflow, rather than replacing the workflow entirely. One of the (massive) advantages of using plain-text markup to write, instead of word processors (or their conventions), is that it lowers the friction between software to zero. I could hop on an AlphaSmart for some portable writing, then offload that to Scrivener, then fire up Sublime alongside it do some editing, without any lossy conversion needing to be done. And of course once I’m done with that, I can use Pandoc seamlessly to produce high-quality output, or just produce raw Markdown and keep my output as agile as it always has been.

The premise of the OP seems to be to migrate from Obsidian to Scrivener. They have their motivations for doing so, and I won’t question them, but that isn’t something one must do, thanks to Markdown being capable of effortlessly crossing all software boundaries.

There are many writers who use who tools like Obsidian and Scrivener together, precisely because they make for a really good workflow; they augment each other with their own unique strengths. For example, it is a good system if you like to write with colour-coded text but want a heavy-duty text assembly platform around that. Other people use other editors with Scrivener seamlessly (even Ulysses!)—it’s not hard to do, and if you really feel super strongly about your asterisks being bright red or grey or whatever, instead of black, there you go.

It is an option, one needn’t work that way, they can do everything in Scrivener’s text editor as well. That is surely what I do most often. I only set up a sync folder setup so I can use additional tools for certain kinds of writing projects, where bringing those to bear against the text is beneficial.

Now for the more subjective argument, as for where I am coming from, the wording in my first paragraph is a big clue…

…I’m one of those people that uses Markdown to write because I don’t care for rich text editing in the first place.

It’s not the Markdown itself that is the primary impetus to use markup while writing, it’s how inadequate I feel a rich text writing environment is for writing as a core philosophy. For design and layout, sure okay, but trying to mix that with writing has always been puzzling to me, and I don’t understand the appeal. That was my first reaction to Word, with all of its bluster about how “no codes” was somehow an improvement.

So in other words, if Markdown did not exist, I would probably be writing the way I was before the simplified markup concept came along: raw LaTeX… with very little syntax highlighting or typing aids at that!

Perhaps these examples will put into perspective why I myself don’t care if Scrivener does much or nothing with Markdown in the editor:

How AmberV wrote in 2001
\chapter{Name of Chapter}

Whik gronk; thung \emph{epp rintax whik} jince dwint srung sernag nix la quolt sernag brul jince. Twock, quolt whik tharn dri cree gen...

\begin{itemize}
  \item Prinquis nix delm velar rhull korsa ti epp su rintax lydran irpsa, kurnap re menardis.
  \item Ma ozlint ju wynlarce gronk ma.
  \item Cree clum la wex frimba zeuhl.
\end{itemize}

Velar menardis, wynlarce furng berot furng gen. Thung er wynlarce wex tolaspa, srung morvit galph. Gen athran morvit... korsa, morvit menardis kurnap rintax velar.

To me, that’s a fine and workable writing environment—LaTeX was after all designed as a tool for writers. So when the following came along in the mid-2000’s, I only saw it as an upgrade to that basic idea of writing with clear markup, only without the extra keystrokes and unnecessary space spent on markup:

How AmberV wrote after finding Markdown
# Name of Chapter

Whik gronk; thung *epp rintax whik* jince dwint srung sernag nix la quolt sernag brul jince. Twock, quolt whik tharn dri cree gen...

* Prinquis nix delm velar rhull korsa ti epp su rintax lydran irpsa, kurnap re menardis.
* Ma ozlint ju wynlarce gronk ma.
* Cree clum la wex frimba zeuhl.

Velar menardis, wynlarce furng berot furng gen. Thung er wynlarce wex tolaspa, srung morvit galph. Gen athran morvit... korsa, morvit menardis kurnap rintax velar.

@nontroppo : Honestly, using the RTF engine as a pimped up semantic grappling hook is why Scrivener, as a markdown editor, rules the markup multiverse.

Although I think we take that concept to different degrees, I can’t disagree with that. I do use styles here and there, where they benefit me, but to be perfectly honest I see that more as a second-best than what the best solution would be. For myself anyway, the best solution would be a program that lets you design your markup on the fly.

Pandoc comes very close to that (especially if you can program a bit), but its base solution is still a bit bulkier than I feel Markdown’s ethos strives toward—it tends a bit more toward the raw LaTeX verbosity. For example, I could do something [like this]{.menu-command} to expand Markdown’s basic semantics into a work-specific local syntax, but is that actually any better than \menuCommand{like this}? Debateable! What would be ideal is if I could type in something {{like this}} and tell the software that when exporting to LaTeX it should produce the second example, and when exporting to Markdown the first example, and when exporting to HTML, <span class="menu-command">like this</span> and so on.

Instead we have styles, which produce that same effect in the end, but doing so using the conventions of rich text writing, which for me isn’t ideal. I would rather visible markup than “menu command” being a different colour or something, and having to click around into differently coloured phrases and squint at toolbars to fathom the overall semantics of the text five years later.

So while I do use styles in Scrivener to expand markup effortlessly, it’s a second-best solution for me, and although it is possible, I wouldn’t dream of replacing all markup with styles. That’s just me though, and my strong preference for *visible* markup being as important to the reading and writing experience as the text it applies structure to—as an ethos.

That’s why I don’t really “get” Markdown editors that try so hard to hide the Markdown, too. :slight_smile: For what advantages I choose plain-text markup for, it counters the very essence of that choice—and doubtlessly colours how I tend toward using Scrivener more as a pure Markdown editor.

2 Likes

Yes, that’s why I wrote:

Raw markup (or -down) is the only formatting that survives the back and forth in this case. The asterisks stay, the paint comes off.

@AmberV

I agree. My reason for writing this was specifically:

I also don’t want the Markup to be hidden, quite the opposite: I want it to stand out more.

When I started coding, there was’t even color (well, technically there was color for some billion years, but only one on the screen). I don’t need colors to make a program work. But I’m glad syntax highlighting is a thing now. It’s easier to read. That’s how my brain works for some reason.

You can load a program into different code editors and get the same visual representation. Out of the box. But you can’t load a Markdown file in a Markdown editor and Scrivener and get the same visual representation.

That being said, I don’t think it’s worth throwing more resources at solving this problem (inconvenience). It’s too niche, and probably non-existent for most users once Scrivener for Android hits the market.

2 Likes

To me, that speaks to the root of the two sides of this discussion… what you prefer depends on your background. I may be someone who has never done any coding, but I do appreciate the concept of semantic structuring and markup; in fact, I came to realise that for years I have used Scrivener in just that way, compiling to a standard RTF document and using a Nisus Writer Pro macro to create the final formatting. But, not being a coder, I find having the markup hidden behind styles much easier to read than having it all exposed, leave alone the question of having to write it.

:grin:
Mark

1 Like

Exactly, and the schemes I use in Sublime and whatnot are geared toward making the markup brighter and more obvious. But the main reason I am compelled to take text out of Scrivener through its sync folder mechanism is for the typing aids, like table clean-up, better listing environments and auto-complete that takes the content of the text as a source, like suggesting headings when you open a bracket pair. The syntax highlighting, while nice, isn’t enough of a reason for me to set up a sync folder.

As to what @xiamenese is saying, that’s definitely an interesting thing! Scrivener in its own way sits in a peculiar space between hard word processing (Word; LibreOffice; NWP) and markup, in that it treats rich text formatting more like Pandoc treats Markdown, if that makes sense. The compile approach to generating documents promotes thinking about the text in a more structural manner, rather than spending a bunch of time making it look right as you write (and that’s the part I find a major time sink in word processors). That such an environment exists is part of what makes it such a good markup platform, too.

Oh, there was one thing I forgot to say anything about, and that is that Scrivener is indeed a program that lets you make up your own syntax on the fly. From my previous example, I can in fact write {{something like this}} and have to converted to standard markup or extended markup when compiling, via Replacements. It’s a bit limited when compared to something like what you can do with a Lua filter in Pandoc for sure, but with regular expressions in the mix, you can actually do quite a lot with them. It’s worth noting there that the original implementation of Markdown, and MultiMarkdown as well for that matter, was a bunch of regular expressions. So you can do a lot with them, but at the same time it is also worth considering that Markdown conversion has progressed a lot since then and none of the major conversion tools at this time use RegEx to power markup conversion. Even back then, with MultiMarkdown, it filled in the gaps of what RegEx couldn’t do with its XSLT-based output filters.

Styles on the other hand are even more limited because they operate purely on surrounding marked text with markup. You can do a lot with that idea, but you can’t, with styles alone, do some of the more interesting things you might like to do. Thankfully we get both, with Scrivener. :slight_smile:

Can you do these things in other programs? Generally no, unless have some programming skill. Obsidian, for example, could probably be coerced into doing some of the above, but you’d have to learn how to code in Javascript, and know enough HTML/CSS to make your own plug-ins. Can you defer it all to Pandoc? Yes, but to really do what styles+replacements can do, you’re going to want to learn some Lua scripting. So it’s not that Scrivener is bringing something utterly novel to the table, it’s that it is making these capabilities more accessible, and thus the kind of thing anyone can start thinking about as possibilities for increasing their writing and document production efficiency.

Yeah, I tried to achieve something different back then: Basically using external folder sync with Markdown editors as a substitute for the non-existing “Scrivener for Android”. As a writing tool on the go. Which then looked better (IMHO, e.g. syntax highlighting) than the main program (Scrivener) – that it was supposed to “just” complement – when using simple markup for emphasis and such. If that makes sense.

Got it. Thank you for the pointers! I will check out those resources.

I do think I have been confused. When I look inside a Scriv package, the individual bits of writing are .rtf files. I don’t think I ever quite got how it worked with Markdown.

Nor did I much care. I came to Scrivener 15 years ago because it was writer-friendly and intuitive. It still is. And I liked that the WYSIWYG aspect – I wanted italics to look like this, and not be the same typeface with some code in front of it and behind it. For me, code is distracting.

David

Um, not quite. Obsidian is definitely a Markdown application, but it lets me toggle between Markdown-visible and Formatted modes. So for instance if something wonky is happening with highlighting, I can switch to markdown and move the == marker for highlighting, then switch back and see highlighted words. Easy-peasy. Not an import/export system. Just a switch.

I like being able to edit in rich text (while still being able to access markdown). Rich text saves space, and makes identification of the structural parts easier.

Writing this is in italics instead of *this is in italics* is much faster, and creates less clutter. Better yet, you could even use character styles showing something like this is a reserved word and then translating into [this is a reserved word]{.reserved-style}, without having to actually write all the controlling code.

Also, not having to end a paragraph with two spaces is something I like a lot. It makes reusing a long text a lot easier.

Paolo

2 Likes

I mean, I understand there are people that like writing in rich text, I am not oblivious to that. I was just stating that it never made any sense to me, personally. That means I have never written that way. I have never pressed the return key once between each paragraph, so that habit means nothing to me, and doesn’t seem in any way an advantage, save for whether it is already your habit (not that one has to write with fully spaced paragraphs in Scrivener if they prefer not to, which is also true of some Markdown programs, like Typinator).

How is holding down the Ctrl key and then pressing the i key actually faster than holding down the Shift key and pressing the 8 key? Again, that seems like something that is a habit rather than a fact, that the very same fingers pressing keys a centimetre away from the other is slower. And that is for the things there are shortcuts for. I effectively have “shortcuts” for everything because all formatting can be done at the same speed as writing, if that is how you’ve always done it.

As to whether it is “clutter” is again merely a matter of opinion and what you are used to. A phrase in asterisks is to me no more cluttered that a phrase in quotes or parentheses. That one looks like clutter to you while the eye slips right over and understands literary punctuation should be proof enough that minimal markup can become equally wired into reading and writing, without conscious thought or awareness of the grammatical or formatting syntax.

So again, I am not stating any of my preferences as though they are somehow better than other people’s preferences, or actually faster or slower or more or less cluttered than the visual noise produced by a multitude of font variations. All I was stating is that Scrivener can be used to write in Markdown (or LaTeX, or HTML, or whatever), to a minimal or full degree, and anywhere in between, as the OP seemed to be under the impression that converting their Markdown files to DOCX files before importing them was a necessity for using Scrivener.

One might do so, but since the expressed desire was to tap into Scrivener as a Markdown tool, I wanted to make sure it was clear what their options were.

4 Likes

I believe I referred to it in one of those posts I linked you to before, but maybe not: if you’re on a Mac, have a look at Marked. It’s a general purpose tool that does what you are describing, only it’s not stuck in one program, but rather can augment many programs with a live preview. It does this by previewing any file on the disk as it saves—and interesting to Scrivener users, it does that for Scrivener projects, too. It is able to read the .scriv format, and will preview your Draft as you write it.

2 Likes

@AmberV – I just wanted to ping this feature request as I think it got lost in the interesting philosophical meditations. In an ideal world, Scrivener could convert markdown to styles (where > blah becomes a block quote etc.) or direct RTF formatting[1]. This could be an option on import.

I can imagine someone who uses markdown in a notes app could want the semantics of that markup to be expressed optimally in a Scrivener document.


[1] I realised for RTF a user could manually convert MD to RTF before import. The tricky bit is RTF to styles…

Yeah, I suppose we could add something more sophisticated like that as an integration, and have an entry in the Import menu for it, but it always struck me as a bit odd to do too much along those lines, mainly because of the styles problem. But I guess I was always thinking of it in RTF terms as well, while maybe having Pandoc convert to DOCX would be better. Of course you’d have to somehow know what it uses for style names for them to import correctly, but that is the case for regular rich text imports too.

Where this (hypothetically) could work really well would be some kind of “plain” mode for certain documents. Still using the same RTF files under the hood, but allowing just Markdown, either by import or entering it manually. Otherwise this will always be a “lossy” translation in terms of external sync. The most important word in this reply is “hypothetically”.