New Scrivener user here. I have started writing a technical book related to computer programming. Most likely I will need to do formatting beyond what appears to be possible with Scrivener, such as flow-around-images, sidebars, etc. I’m trying to figure out what additional tool I should use in order to do the page layout and it sounds like the main contender is InDesign(expensive), Scribus(free), or possibly using LaTex as is done with the Scrivener manual.
I am wondering what people are doing to streamline the transfer of text from Scrivener into InDesign or Scribus. Is there any way to automate things so that I can go back to Scrivener iteratively to make changes and bring the new text back into the page layout software somehow?
What is the best work flow?
I’d like to use a lot of same kind of formatting through out the book for sidebars and other things, so I’m wondering which formatting method will avoid a lot of manual work. Seems like Latex might be good that way for me, but I’m not sure whether the output from Latex will be appropriate for eventually selling print or amazon ebooks, etc.
Appologies if these questions are stilly stupid, this will be my first writing project… Just want to try to use the right tools and integration strategy from the get go if possible.
The best workflow is going to be the one that involves the fewest iterations between writing and design. That is, you want to have the text as close to finished as you can manage before you try to build the page layout. That’s especially true if you’re dealing with sidebars, images, and other elements, too.
My strong recommendation would be to work with a professional designer, and give them the component files in the format they ask for.
You should seriously consider Pandoc. You write using Styles in Scrivener (must be version 3+), then set up mappings from styles to markup during compile. Pandoc takes this structure and transforms it into multiple possible formats like ICML (InDesign intermediate), HTML (you can then use something like PrinceXML for final layout using book-specific CSS), and LaTeX. Scrivener styles + Pandoc abstracts away the complexity of manually copying and pasting. You can use arbitrary block and inline containers so you can transform any Scrivener Style into a structured INFO-box or special inline markup. You are not locked into one output format, and Pandoc can run filters during conversion to automate complex transformations etc.
It might make you happy enough if the alternatives involve either paying a lot to have it done professionally or having to learn a lot about Pandoc to do it yourself.
Word can do a lot of advanced design. Swift 5 is another alternative.
Write the book first and worry about layout when you are done.
Scribus supports several standard text import formats such as ODT, DOC and HTML, but I don’t know how good their conversion is, but Pandoc generates very clean source files.
As for the Scrivener manual, AmberV uses many of new features in the Scrivener 3 compile system using the Multimarkdown (MMD) engine built-in to Scrivener to generate the LaTeX source. Pandoc is more powerful and extensible overall, but as you can see from the high quality of the Scrivener manual, you can achieve a lot with the built-in MMD! The Scrivener source is going to be identical so you can start with MMD then add in Pandoc if you want along the way. Just make sure you write using styles and you can build up your compile format as you go. I chose Pandoc over MMD because I regularly output several different formats, need a bibliography always, and don’t want to fiddle after I compile. Pandoc filters and templates handle a lot of stuff that MMD alone cannot.
So what I have been reading in a lot of places is that writing and page layout should be handled separately, InDesign, for example, is not a great tool to write in, spell checking sucks, etc. Word can be fine for that, but page layout in Word sucks. etc. So choose a dedicated writing up for writing and a dedicated page layout mechanism for that, if you need it.
Scrivener is obviously very interesting for writing and organizing my ideas. I have other books I want to do later, this is kind of a trial run for me. So whatever process I put in place will be useful rolling forward. I like the idea of using Scrivener to write the content, and organize my ideas as far as I can go with that, attempting to do as much formatting as possible entirely in Scrivener…but since layout is limited, finding a way to automate some further formatting from there. I also like the idea of using Styles in Scrivener and transforming them into whatever I want in the layout world…because that is automation…ultimately can result in less manual work as long as I can come up with a layout that is fine for my purposes.
As you said, the Scrivener manual is actually quite, not bad at all! With a bigger budget I could see myself getting a designer to design something in InDesign, so that everything looks really cool, especially for a technical book that has lots of diagrams, side notes, etc.
This first thing has a very limited audience. I will probably not make much money, if anything on it. I have not decided yet how to distribute it. Using the built in MMD and latex obviously can create a perfectly fine PDF output. If I wanted to publish print versions or distribute through Amazon, maybe not this book, but some other book down the road, then what is involved exactly? Can these self publishing providers print from PDF as long as it conforms to certain layout rules or do they require the source to be in InDesign format or something of that nature?
That’s not what I am saying, but the same issues apply pretty much regardless of the transfer method. Page design can be very finicky: simply adding a few lines in the middle of a chapter can require re-flowing the entire chapter from that point forward. Adding, deleting, or moving images and sidebars can require even more dramatic layout changes. It’s a pain in the neck, and you don’t want to do it more often than you have to.
Scrivener is IMO bar none the best writing environment for building your text. It does not focus on page layout (although you’d be surprised how powerful the standard compiler is, and many users use just that for successful submissions to publishers). With V3’s addition of styles and an amazing compiler that can trigger post-processing tools it became a superb tool for automation for final output.
With styles you can transform blocks or spans of text into custom “holders”. So for example, I create an “Allegory” style in Scrivener applied to a paragraph “All animals are equal but a few are more equal than others”. Then in Compile, I set it up so it automatically gets wrapped in a custom style allegory markdown block:
::: custom-style=Allegory :::
All animals are equal but a few are more equal than others
::::::
The compiler takes each allegory style and generates this format. Now this has a semantic tag, Pandoc can then create, for each output, the correct “holder” for this custom style. For example, for EPUB/HTML, it would create a styled
, and for LaTeX it would create a named environment. With CSS in HTML or instructions in LaTeX, you then style this block as you want (floating in a margin, in a box etc.). The IMPORTANT point is not that you have a PDF at the end you hope a future publisher can accept, but that you can generate any output if needed now or at a future date. The other important point for both Pandoc and MMD is that they are designed to pass through RAW content, so you can always embed raw LaTeX or raw HTML as required. If you style these then you can use the compiler to only include the blocks that are needed for a particular format, which can be very powerful.
For the Scrivener manual, I think AmberV uses both styles, replacements and raw pass-through extensively. With Pandoc, I prefer to use Pandoc filters for replacements, but the important point is you have choices that do not depend on your source material (or even the choice of MMD or Pandoc), you are not “locked in” like you would be with Word or InDesign.
As a retired developer myself, I am irresistibly drawn to using MMD/LaTex rather then a GUI layout tool like InDesign. So I will start there and see where I get.
Just reading up a lot of old forum posts about MMD and LaTex, it seems a lot of it is out of date compared to what S3 can now do.
One thing I do not entirely understand is why anyone would put any MMD markup directly into Scrivener documents. Are there certain aspects of MMD markup which Scrivener editor does not currently represent generically? My understanding is that I can just use the formatting tools in Scrivener to make things bold, create tables, etc…and MMD will be created at compile time. but wait, if my end goal is to get to PDF or LateX, or Latex to PDF…then why would I need to go through MMD at all?
I totally get what you’re saying about using styles to drive dynamic markup at compile time, both with or without Pandoc I think? You have provided some great insights about using styles to exclude certain hard code markup as well, very handy…
I can see where it might make sense to hard code into Scrivener document some LateX markup…because Latex obviously goes way beyond Scrivener’s formatting or MMD formatting,
But what are use cases for hard coding MMD markup (unless of course you actually NEED an MMD document as a final result. What’s the point of using MMD as an intermediary?
Scrivener can’t compile to LaTeX directly. You can hand code the LaTeX markup you need and compile to plain text, but why would you want to when automated tools exist?
So basically we write in Scrivener, it has its own native formatting and structuring. We have these options for compile:
We can compile that directly to PDF, word, rtf, epub, etc… and it looks pretty good, using whatever formatting and structuring are native to Scrivener and mostly WYSIWYG, compile options control and add a lot of stuff too.
We can compile to MMD or pandoc, which opens up many other formatting possibilities, once it goes to MMD or pandoc, it can be converted to all the above formats, and also Latex, using a variety of tools…and it sounds like Scrivener can be configured to call MMD or pandoc in order to run that Latex compilation step.
Fine so far?
Further to that:
If we’re doing the 2nd option above, then in addition to the normal compile options, we can use styles to drive extra markup inserted at compile time before sending it on to MMD or pandoc
We can hard code MMD markup directly verbatim into our Scrivener docs and it will be passed through to MMD. If we stylize it, we can selectively include/exclude this hard coded MMD markup.
we can hard code Latex escaped as HTML comments. We can stylize it to selectively include/exclude it from compile formats
Does that about cover it?
But again…what MMD features are not covered by Scrivener’s formatting that I might want to hard code as MMD markup in my document? I’m no MMD expert. Its easy to see many reasons to hard code some Latex for sophisticated page layout…but MMD is not much different then what Scrivener does already with its own native formatting…so far that I can see…what is missing from scrivener’s native formatting that a little MMD markup would be useful?
Yes, you have the general gist correct. An obvious example where you still need to use Markdown/LaTeX directly in Scrivener is for equations. You use raw LaTeX maths and style these as “Maths” blocks, then Scrivener wraps this in $$ … $$ maths delimiters, and MMD or Pandoc then takes it from there.
Another example of where you need to hardcode Markdown: Scrivener footnotes are limited in that you cannot use Styles within them, only character formatting. Therefore if you want emphasis/bold etc. in a footnote, it must be done using plain markdown.
And if you want to store more advanced Pandoc metadata front-matter in Scrivener (I do, as I can then control every parameter by editing this file), then this should be written in YAML, and must be passed through without any processing through the compiler (trivial to do in the compiler).
Possibly more advanced Table layouts are better dealt with using Markdown?
But that is about it, style transformations and regex replacements can handle almost anything! Even properly captioned figures for which I used to use Markdown for (), can easily be generated using linked images and a Scrivener Caption style automatically.
Thanks nontroppo, I’ve seen various videos around the net with people putting some MMD markup into the Scrivener editor, and I guess this is either due to limitations from S2, or else they didn’t want to get into the daunting aspects of massaging the compiler to do what they want. I’m definitely on the same page as you in terms of wanting to use styles and metadata to control everything as much as possible rather then hard coding markup into the visual editor.
I read through chapter 21 (and a bunch of others) last night. Its not easy reading. I can’t really come up with a use case so far that I would need to use MMD markup hard coded into the editor, but one thing I’m not sure about how I will accomplish for this project is inserting syntax highlighted javascript code into the book. That might be where I need to go beyond what even MMD can do and probably embed some LaTeX into it, which I also will have a lot to learn about it sounds like.
I also experimented last night with MMD->PDF, and the font information from the compile format was lost. So learning about how to get Latex to recognize all the same pretty fonts that Scrivener can use itself is another thing I have to dig into. Just using one of the factory formats “Modern” did not pass the font out, ended up with some old school looking thing in LaTeX.