Scrivener has no “use last settings” option for a quick compile (@AmberV may know of a workaround, I’m sure we’ve discussed this before), but its possible to use GUI scripting to automate this (I never bothered, I’ve just internalised the 4 keypresses required).
I was afraid of that. That would be a very useful thing to have. because then I could author in scrivener and routinely see the Jekyll engine refresh a preview webpage for me exactly as how it will be rendered, But if its more the a single one-click or one key command, then it will much more annoying that way.
I notice marked is able to somehow monitor the scriv file and sense changes and update its GUI when you change something in the scriv document, it can even cause a “compile” to take place when you change the scriv document, though I don’t know which compile its using and probably its putting the results into its own gui only, not into the filesystem other then some cached location. but anyway, its possible for a third party to monitor the status of things changed in the scriv document…so…maybe keyboard maestro or something could do something similar but I don’t know how it would easily fire off the compile with hard coded options and destination, which Marked2 is able to do somehow…
The overwhelming flexibility of Scrivener definitely makes it difficult to know how best to use it but since my final destination will be Jekyll I will have to think pretty hard about whether the juice is worth the squeeze of compiler complexity.
If your writing is “atomic” (each piece doesn’t much depend on any other), then the clear merits of Scrivener’s project/idea management becomes somewhat moot. We are biased here, I do everything in Scrivener (including writing letters), and I personally wouldn’t hesitate to use it if I was going to do what you want. But my preferred workflow will be different to you.
Because a Scrivener project is parseable (the binder is XML, the content is plain text RTF, which while not really human-readable, is machine intepretable), Marked can do pretty cool live rendering, but it cannot understand all the details that may be contained in styles and section formats that an “elegant” Scrivener project utilises. Thats why you must try it for yourself.
You can automate Compiler dialog, I just tested using BetterTouchTool and ⌥⌘E ⇨ ↵ ⇨ ↵ ⇨ ⌘R (4 keypresses are needed including accepting to overwrite an existing file) can automatically click the dialog keys without any intervention.
In my case, they are not completely atomic. They will be related articles. ON the website there will be next and prev buttons to advance through them as if they are chapters in a book. There are definitely advantages to using Scrivener for organizing them as I write.
Also, I just realized that I will need to be able to generate separate article markdown files, Scrivener seems to want by default to compile the whole thing into one massive end format document. I see that I can right click on a note in the binder to just compile that one into its own separate destination text markdown file…but anyway…doesn’t seem like people here are really doing this, so I will have to figure it out later if that will be possible without headache to be able to use keyboard maestro or something to conveniently recompile very often.
It doesn’t (well, I haven’t used it for a while, but it certainly didn’t before). It reads the scrivener project and does a “live” conversion itself, separate to the Scrivener compiler. That is what I mentioned above, for example if I use a style, Block Math for maths and this style injects $$ … $$ markup on compile, Marked 2 doesn’t know about this and so equtions do not get rendered using MathML… The more you use Scrivener’s styles and section types, the less well Marked 2 is able to process your Scrivener file…
I often compile single documents in a much bigger project, this isn’t an issue unless you want to use the same automation as a generic compile. But you can run different automations based on selection / manuscript etc…
EDIT: also, you can use a script to split your Scrivener compiled project into separate markdown files, this is not an insurmountable issue…
Something I would add to the mix is the notion of using the external folder sync feature for all matters that involve integration with a larger Markdown-based toolset. In essence, this approach would sit right in between your first two stated simplest approaches.
Here, the discussion was primarily about linking together Obsidian’s fantastic note-taking capabilities with Scrivener’s long-form writing platform to create a greater whole than either could provide on their own. But the basic concept, of having Scrivener dump .md files out into a folder somewhere, files that it keeps itself synced with, making easy integration with other tools that are adept with loose files.
Definitely consider throwing a shortcut onto the File ▸ Sync ▸ with External Folder Now menu command. That will be your “compile”, if you will, your one-shot update—though in this case it can be a two-way update if you want it to!
That’s certainly not a necessity though, particularly for some projects, and particularly in the early phases where pure drafting is being done. Then, I would argue, there are good arguments for eschewing much of the “fancy stuff” and sticking to CommonMark in the editor so that you can use other editors freely, which nobody will debate, do a better job of helping one write Markdown.
If you’re thinking of all the window management that might incur, it doesn’t have to be that way. Scrivener’s project window is extremely flexible, to the point of becoming a very narrow outliner that can be set alongside another program’s editor+preview window. You develop the outline on the left in Scrivener’s window, and do the writing on the right. It’s really not that different from having multiple split editors open in Scrivener. As you noted in your original post, at this level you’re using it more for its organisational capabilities than anything, and that’s a lot of benefit all by itself. There isn’t much out there that works to this capacity like Scrivener does.
Here we have the Scrivener project window pared down to an Outliner view (more on why in a bit). This narrow window can be placed alongside anything else. I’ve never been much for live preview myself, but if I really wanted it in Sublime Text (which is on the right), it is a plug-in away. What you put on the right side is up to taste. I like editors such Vim, VS Code and Sublime because they are as agile as Scrivener is when it comes to keyboard management and navigation, but most importantly are, at their roots, extremely powerful text editors—a thing many Markdown editors decidedly are not. I like Obsidian, don’t get me wrong, but I’d honestly rather write Markdown in Scrivener. I do use that tool, but not at all for its editor, that pretty much stays in read-only preview mode.
It’s a little hard to see in the screenshot, but I’m actually editing three separate text chunks, with the view split vertically and two tabs in the lower split. I can effortlessly jump between sections via its project-wide searching and opening tool (the project, in this case, is Scrivener’s sync folder, but given how these editors work, I could easily add multiple sync folders to one project).
In my opinion, the agility of such text editors really mitigates some of what might seem up front concerns with sync folder usage: the granularity of Scrivener’s outline. It’s blurred out, but you can see I’m editing a small chunk of text in the lower split. While it has no “Scrivenings view”, the rest of how it is used does not feel substantially different from using Scrivener’s project window—both are very keyboard shortcut driven and make jumping between sections by name easy.
And hey, if I need Scrivenings, that’s easy. I just double-click on any section’s icon in the left window to hoist the outliner view to that area (or with it already selected, ⌥⌘O / Alt+Shift+O), and hit Ctrl+1. Done. When I’m finished getting a bigger look at the combined text I hit Ctrl+[ to jump back to the full outline. While this setup makes Scrivener look deceptively simple in a screenshot, it’s important to keep in mind that little narrow window there is full blown text editor view, with all of the many things it can do (full project navigation, collection management, filtering, brainstorming, snapshot revision review &c). That’s why we are using an outliner view here instead of a binder (well, that and you can’t hide the editor, so you might as well use it).
In my opinion, all of this above is an answer to the analysis paralysis problem. If you aren’t sure where to go with Scrivener yet, that doesn’t have to be an impediment to getting started with the basics. Use it like a text editor. Use it with a text editor. Treat its outliner as a file manager at first, so you can bail out if you don’t like the program, and take your sync folder with you (but do start outlining as soon as you feel it, because that’s where the power is—anything can throw .md files around in a list). Don’t bother with the contingencies and branching flexiblity—until such a day arrives that you hit a wall, and maybe one of those things is your answer. Then you learn that, introduce it into your process, and keep working otherwise as you did.
More than any other way of using Scrivener, Markdown users have a more natural capability to just get in there and use 1% of Scrivener to get real solid work down, and gradually build up from there in a way that doesn’t require extensive revamping as you learn new things. The Markdown you type in is 100% identical to the Markdown Scrivener can generate, or be “programmed” to generate, in the end. It all looks the same in the .md file, right alongside each other. You hardly ever need to go back and “upgrade” your markup unless you really feel the burning desire. And that fact, by itself, should help in just diving in and starting to use it.
On the matter of “one shot” compile, I made a script for that ages ago, which you’ll find in this post. Two caveats: the actual macro file attached to the post was made ages ago, I don’t know if it will still drop in as functional in a modern copy of Keyboard Maestro, but if not the basic formula described in the post should suffice to recreate it. The second caveat would be that in the years since then Apple has messed up file dialogues so that they are slow even on fast systems, and sometimes no longer buffer keystrokes. So that warning about maybe needing another pause may be more necessary even on a screaming fast machine.
With that method I could setup Scrivener’s styles to kind of pseudo match markdown formatting or not…doesn’t matter, the compile process would chuck it into jekyll for me and I’d see the preview there. I’m kind of leaning towards this option.
I do a little of that, but mostly only for the things that really do benefit from it, like hanging indents on bullet lists, block quotes and code blocks (mainly for an alternate tab stop setup). It may seem a bit silly, but it actually looks better in my opinion than what a lot of text editors can do anyway and with shortcuts on the few I use regularly, it’s really not a bother since writing with Markdown itself is so simple and effortless.
Most of what I use Scrivener’s rich text for is editing marking though, as I had to say over here:
I only use styles to generate markup when I really need to, or when it’s a bit too cluttered for my taste. But most of my projects do not need more than Pandoc+CommonMark. That actually covers a lot of ground with nothing more than what can be done at full writing speed.
Also, I just realized that I will need to be able to generate separate article markdown files, Scrivener seems to want by default to compile the whole thing into one massive end format document.
Two things there:
In the Contents tab on the right side of the compile overview window, there is a dropdown at the very top (probably set to Draft). Set that to “Current Selection”. Now with everything else set up the way you want, your export process becomes selecting the parent outline item that represents the article and hit the KM macro. If you also keep all of your YAML metadata (or whatever Jekyll uses) in the outline instead of the compile settings, then you probably can manage with generic settings that get reused for entirely different pieces of content, each with their own title, pub date and so on.
Hint: if you are hitting limits with “Current Selection”, set it back to Draft, then click the funnel icon to its right and enable filtering. This also has a “Current Selection” filter, only this one extracts the selection from the original outline instead of treating it as a flat list, and also grants access to the front/back matter features as you are now doing a “full” compile (that most of the draft is filtered out is irrelevant).
But I am wondering if I should get into the compiler and consider setting up the right set of styles, so that markdown is specifically NOT written into scrivener (will be compiled later based on styles and such). If I do it that way I have more options down the road of switching to a different variant of markdown or whatever, or perhaps publishing to PDF etc.
While on the surface that is true, and I can see why some might want that, on the other hand if one already has a foot and a half into the Markdown realm then in my opinion there is very little above the Markdown section of the Compile-For dropdown that is compelling enough to sacrifice a workflow over. With Pandoc and Quarto in the mix, everything you can do with Scrivener can be done lots better elsewhere. DOCX? Yeah, Scrivener’s output can’t hold a candle, and particularly if you try to use it to format the text, which ostensibly is the only reason you would want to. I.e. the best way to use Scrivener’s native word processing output is to struggle to turn everything it does off so that you end up with something almost as (but not quite as) clean as what Pandoc gives you, so that you can, like Pandoc, dump the raw styled output into a template. So why bother? The only reason to bother is if you don’t at all use Markdown.
One-click output with Pandoc...
This is straight out of the compiler, via Pandoc’s template system (well I did regen the ToC), and represents a collage of page layouts rather than the actual output (which includes blank pages to force recto layout for major breaks, but makes for a weird screenshot).
You can’t do most of what you’re seeing here with Scrivener—at least not a in way that makes for a good, well-formed document where what you see here is 100% a product of the stylesheet, all the way down to the numbering, page flow header/footer switch-ups, drop caps and so on.
ePub? Again you will spend hours to get output as clean as Pandoc’s ePub output. Now here Scrivener does have some theoretical advantages, but mainly for people who are not comfortable enough with CSS to design a book with it—it’s great for that. Even then I would say there are plenty of tools that can help you generate decent CSS from GUI tools that can, with a few tweaks, be dropped into your Pandoc compile settings—such as Scrivener.
Quick and dirty PDF is probably the best argument for what you’re describing. Most Markdown tools will be deferring to LaTeX for that, which can be quick and dirty if you already know it, but if you don’t, it’s anything but. That leaves ODT/DOCX and using one of those word processors to pump out a PDF. Not a bad solution, really, and just a few extra clicks. There is nothing in Scrivener that will give you a better result for less effort than that, as demonstrated above.
If you still aren’t sure though, use styles, why not. There are many Markdown-only writers around here that work that way, without a drop of “syntax” in the editor, that have no intention of using Scrivener for anything other than its Markdown output. I’m not saying with the above that one shouldn’t use Scrivener’s writing kit if they are a Markdown writer, I just wouldn’t want to sacrifice external folder sync and having a whole phalanx of tools to work on my material with, for that reason alone, because I personally never use Scrivener’s rich text oriented output for anything at all other than QA testing. A styles-only workflow would be, for me, a mere matter of aesthetics or to avoid excessive LaTeX or other final syntax in the editor (as the Scrivener user manual would be). But, we’re way beyond the basics at that point.
Amazing post! Thanks for putting so much thought, time and effort into writing that!!! definitely I may refer back to this later.
I have more or less decided as of this morning to use obsidian rather then Scrivener for this particular task…blog/article jekyll hosting. its a closer fit. If I had a lot of stuff in scrivener already and was used to working that way it might be different, but I’m kind of starting out and I don’t think Scrivener is really the right tool for this job, despite all the myriad of ways it appears it could be shoe horned into the task. Obsidian is a much closer fit really for the following reasons:
obsidian “vaults” are basically dirs full of files. So you can basically point it at a jekyll dir with a bunch of markdown files and obsidian will be ready to display and edit them, appearing in obsidian as a “vault”.
Obsidian has plugin support with hundreds of plugins, some of which will let me closely replicate the jekyll GFM markdown, including mermaid and a few things like that which I will definitely be using. End result is that I can skip all the stuff I was saying above about trying to preview the formatted doc in jekyll, obsidian alone will provide a good enough preview of the documents as they should appear, that I can just stay there and write and see it formatted properly as I write.
Obsidian is basically free unless I need to sync with my ipad
Its editing mode is closer to real markdown, which means whatever I write will translate well to jekyll.
no complications about compile or not.
its true that I won’t have Scriveners ability to keep research ideas around in a nice organizational place, but actually that can be done with Obsidian too, its just a matter of putting that kind of research into actual files under the obsidian vault in an organized way. what scrivener particularly makes easy is reorganizing how different ideas will be structured into a final longer form. i think its brilliant for that and when I get ready to write my first novel I will definitely use it.
Well maybe i will use obsidian. I might be back to Scrivener. There is going to be more overhead setting up scrivener, especially in light of the compile multiple md file complexity, not undoable, but some initial thought and setup and learning about scrivener more before i can even begin. The main attractions I see to Scrivener for this task are as follows:
by using the compiler, I can write the content WITHOUT MARKUP. This is very key because then I will have a core initial version of my writing that is not littered with any markup at all. The compiler can add all the markup. Then later I can change jekyll themes or change to a completely different publishing format and deal with it in the compiler, but I like that I would be saving the core writing without markup embedded in it.
while Scrivener would not really be providing WYSIWYG preview of the formatting, the RTF capabilities of Scrivener will allow me to format the core writing in some way that will make sense in that job of doing the writing, maybe something close approximation to final form in some ways, maybe not. What matters is I setup the right set of styles so that the compiler can turn it all into the right markup. Not sure how the compiler will deal with character formatting though.
I do like the idea that this work will be all inside some kind of DB, such as Scriveners, so that all the writing I will do will not be scattered around some folders checked into github, but rather in some kind of write-centric form, rather then a bunch of markdown files. Also in scrivener I can use the commenting and inline annotations features to make reminders for myself as I’m working on it, etc. and save various images and tidbits also in the Research folder. Also I like that i can break up documents into subdocuments and able to rearrange later easily.
So these are all strong reasons to use Scrivener presuming I can learn about the compiler and setup the right set of styles to support it and able to generate separate md files per chapter, easily, and able to go quick one press compiles also… Once I work through those initial technical challenges, then I think it would be in many ways a very nice way to work, but just a lot of setup in advance.
On the other hand, if I do not use the compiler and try to just use markup inside the documents, etc…I think i’d rather just use Obsidian in order to edit the md files in place in the file system.
But anyway, a little torn on it right now, will probably try both ways and see where it goes.
While it’s true that the Compile command can be complex, it’s important to remember that you really only need to set it up once. That’s a small price to pay for all the other things that the Scrivener environment gives you, especially for large projects.
oh I get that. The learning curve is a bit steep for me right now, but once I learn it and set it up I’m sure it will be smooth once I get through the learning curve about the compiler and or depending on whether I decide to use pandoc could be more learning curve there too.
I am truly torn on the issue. But I do really like the idea of having by writing content isolated away from all formatting markdown. That represents my intellectual property in its purest form and what is cool is that Scrivener even has a way to translate styles and position in the outline to turn that into markdown or other published formats however desired now or different in the future, no problem, the core writing will be in a symbolic form inside Scrivener
Well I will have to spend some time with it and see how it goes.
While I fully agree with the common view of separating content from presentation, Markdown has become so ubiquitous, and Pandoc so powerful, that Markdown can almost be considered pure text for most purposes.
I believe the biggest challenge you will face is not getting the compiler to convert your semantic structure (once you understand the concept of style rules and section types), nor creating a one-click automation, but rather the tricky issue of compiling single documents while retaining cross-references. Compiling just a selection (as I often do for correspondence letters and small project grants) is trivial, but your situation requires referencing other compiled documents. My single-document compilations never reference other docs, but yours will. There is likely an elegant solution for managing cross-references, but it remains unclear to me… Do you plan to use GitHub and Jekyll for this?
That’s a very interesting point about links and that may be definitely become a problem. We’ll see.
basic markdown I agree more or less, but only to a point. Markdown is not the be all end all. If I decide I want to publish it as PDF instead, then what I have to go back and filter out all the markdown somehow, etc. or deal with it. In addition, I will be using additional IAL’s provided by Jekyll to make the output just a little more controlled then pure basic markdown, this is all provided by kramdown. So there will definitely be a bit more markdown in the document then typical simple markdown. I’d rather have the compiler add that for me if possible
Yes I will be hosting on GitHub pages, with Jekyll, using Actions and the minimal-mistakes theme.
Well, Pandoc has 11 different PDF engine options so the same markdown can build any manner of PDF in seconds , but your general point is taken (as being someone who loves Scrivener and keeps all their work in it but still uses markdown for all output to the big wide world)…
Now, kramdown seems to use GFM-flavour by default, which is directly supported by Pandoc, so at least the markdown output from Scrivener should all work as expected
kramdown uses GFM yes, but its also possible to use so called IAL’s and lots of css stylesheets associated with it to do extra things not included directly. It also uses the liquid templating language to do things as well on top of the kramdown. Well I only need to use a few of them, but they are beyond normal GFM and more a part of what is provided by Jekyll. For example
{% highlight javascript linenos %}
function doit(arg) {
console.log("do it");
}
{% end highlight %}
So anyway, it’s beyond just simple markdown to make it the best it can be in jekyll.
But I think we’re still back to the problem you mentioned about linking between my blog articles, which I would be doing a lot of sure. I’m trying to think of some way I can do that, it might involve having to create some custom meta data and a post script that can chop up one large markdown file into many small ones with all the links corrected…not sure right now…
These can still be generated by Scrivener’s compiler and protected during processing, so that is fine.
Don’t overthink it, it may be that relative links will just-work™ without any further fussing — HTML is pretty good at cross-linking. As long as the link names are consistent, it may not require any more thought. Without a basic test we won’t know…
Another thing to throw in the mix: Quarto, which is basically a turbo-charged Pandoc. It has direct integration to Github pages, and is used to generate blogs and websites alongside many other outputs. As it is basically Pandoc, it integrates with Scrivener in the same way.
The cross links will only work as you say if I encode them into the markdown using the destination final generated HTML paths as the link URLS…which I definitely don’t want to do. So no I think that is still going to be a challenge requiring some creativity.
today I experimented a little with linking in Scrivener. Created a small scrivener project with a few chapters (top level folders inside Drafts) and then each one having some documents underneath. By default this will generate a single large markdown file from compile.
I created some links in scrivener by dragging documents from inside chapter one, onto a document in chapter two. this resulted in creating a link in the document and while using scrivener, the link works as expected.
When I generate markdown there is no link markdown syntax added at all during compile. All link information is ignored in the markdown version.
Is this something I need to use Scrivener a different way to create links that will translate to markdown links?
Ultimately I will need to change these to external links also since the destination will end up a separate markdown file for each chapter…but that will be a separate secondary challenge, first I have to see how we can have scrivener compile markdown links.
It definitely compiles some kind of link oriented markdown, but so far I can’t find a single markdown editor that can actually make any sense out of what it is currently generating. Has anyone else ever actually used this feature?
it seems to change the headers of linked-to things to include a generated reference string…which destroys the look of the document and also tries to link to that, but doesn’t work. I don’t think doing what it should be doing.
For example, I linked one section (the entire section) by dragging that scrivener document onto the page of another. The link works in scrivener. After compile, I went up with markdown link like this
[part 1][]
and
[part 1[ref1]
The word ref1 was not written by me, totally generated by the compiler, but where to find it? later on it added a header section that looks like this
## part 1 [ref1] ##
I guess that header was added as part of the section processing to add chapter titles automatically, I could theoretically remove that, but then the link wouldn’t work and actually it doesn’t work anyway…so really this is just not correct markdown being generated as far as I can see.
But anyway, in my case, I will have to run this through a post script…so maybe I can fix the output so that it works after going through Jekyll, I can try to write a script that goes through looking for auto generated names like e"ref1" to try to make sense out of it, but this is already getting into nonsensicalness to me… I might be barking up the wrong tree here…
Here is a fresh test project (blank template, minimal changes, Multimarkdown compile option, though I tweaked it to run Pandoc/MMD to generate HTML to see the results):
Cupcake ipsum dolor. *See also* section "[The Moon][]". Sit amet marshmallow topping cheesecake muffin (detailed in the [Discussion][]). Halvah croissant candy canes bonbon candy ([but note otherwise][The Sun]). Apple pie jelly beans topping carrot cake danish tart cake cheesecake.
...
# The Moon
...
# The Sun
...
# Discussion
...
Note there are no IDs added to the headings. Both MMD and Pandoc auto-generate links. Here is how Pandoc transforms the markdown generated by Scrivener:
<p>Cupcake ipsum dolor. <em>See also</em> section “<a href="#the-moon">The Moon</a>”. Sit amet marshmallow topping
cheesecake muffin (detailed in the <a href="#discussion">Discussion</a>). Halvah croissant candy canes bonbon
candy (<a href="#the-sun">but note otherwise</a>). Apple pie jelly beans topping carrot cake danish tart cake cheesecake.</p>
...
<h1 id="the-moon">The Moon</h1>
...
<h1 id="the-sun">The Sun</h1>
...
<h1 id="discussion">Discussion</h1>
All links work fine.
MMD or Pandoc will add the correct #ID references by default (the rules for generating auto-ids are fairly straightforwards: Pandoc - Pandoc User’s Guide). I’m not sure how you got ref1, you can share your project to see where you went wrong.
so the same markdown can build any manner of PDF in seconds , but your general point is taken (as being someone who loves Scrivener and keeps all their work in it but still uses markdown for all output to the big wide world)…
