What is the best way to compile to AsciiDoc?

[Dewdman42]

So I have been learning about asciidoc lately and I’m quite thrilled to find out its supported by Hugo, gitlab, GitHub and basically all the places I was needing to publish to, with more features than markdown. Pandoc comes close with its own markdown, but I have to say asciidoc is just much better thought through and I’m surprised more people aren’t adopting it in some way. If not, then why not?

I am curious what people have found is the best way to compile scrivener to asciidoc rather then MMD or pandoc? is this a matter of post-script translation in order to do it? where is the best place in the toolchain to generate asciidoc markup?

what I really like about Scriv is the prospect of organizing my ideas and easily being able to restructure documents and write a lot of stuff for a while that will eventually be restructured into different web articles…or combined into chapters of a book, etc… but anyway the main point of Scrivener to me is about that… Maybe I should just write in asciidoc markup right in scrivener to avoid having to worry so much about compile complexities, I don’t know. Still pondering that. I also want to be able to easily re-structure things into chapters of a PDF or ePub…so…I don’t know if that will make sense to always go through ascii doc (in which case the markup in scriv would be fine…or if it would be better at that point to use pandoc or whatever to LaTeX, etc… just so many options, its hard to know. But anyway at the moment I’m impressed with asciidoc and in ht process of converting my gitlab wiki’s over to it. Just thinking about the future and whether it makes any sense to try to use scriv for asciidoc workflows.

[nontroppo]

There are three options:

1. You can use plain text compile and write the markup in the document.
2. You can use plain text compile and use styles / section types to build an asciidoc markup compile format.
3. Use a markdown workflow and Pandoc to convert from markdown to asciidoc in post-processing, any asciidoc features not supported in Pandoc’s AST are output via compiler as asciidoc markup directly.

I think Scrivener is the perfect writing environment for any of these markup-based tools like aciidoc, where the compiler really shines. Which route is best for you depends on you, some people like writing markup directly, others (like me) think styles and section types are the best way to seperate your written content and presentation ensuring their work is not “tainted” by format specific details.

I’ve worked with an asciidoc output once, and I used Pandoc ((3) above no surprise), in my mind this guarantees more flexibility. Specifically for me, academic referencing is essential, for which asciidoc has no direct support so Pandoc can build the bibliography as text then generate the asciidoc. This also illustrates some problems with formats like asciidoc. As it is not as hevily used, asciidoc 3rd-party plugins are not updated very frequently and bugs linger. I did find a bibliography plugin but it always crashed when compiling to PDF. Pandoc is heavily developed and the developer is amazinlgly productive, with bugs or issues often being fixed in days. The environment is mature, development is active, and workarounds for issues are easy to build given Pandoc’s underlying architecture.

@Dewdman42

I’m surprised more people aren’t adopting it in some way. If not, then why not?

Yeah, asciidoc is great, with most of the core features needed for technical writing. But once you are locked into a single format it limits your future flexibility and for less used format puts people off (limited extensions, long-standing bugs etc). There are also just so many more tools for markdown out there. and I suppose there are other options like Quarto which overlap most of the features and design-intentionality of asciidoc but with the benefits that markdown bring.

Honestly, as you can use Pandoc to convert asciidoc anyway, I tink it is a a good format to work with if it covers your final outputs now and you don’t need other targets in the future. As I love Ruby, I also like the association of the asciidoc[tor] tooling all being written in Ruby

[Dewdman42]

asciidoc provides much more capability and flexibility in terms of structure and eventual formatting then MMD or pandoc’s own markdown so if I go through MMD or pandoc on the way to asciidoc, I lose that additional structure capability which completely defeats the purpose. Something more has to happen besides trying to send through mmd or pandoc, which is a lowest common denominator.

The question is how to compile directly from scrivener to asciidoc and gain the extra structural capabilities of asciidoc. ascii doc provides admonitions, sidebars, examples, and number of other capabilities…including also nested blocks and other things…all of which scrivener could potentially handle itself I think in some way from the perspective of using styles, metadata and section types, etc…to drive a compilation…but its not clear to me that MMD and pandoc won’t get in the way unless some work around has been devised. I’m asking for people who may have actually done this already. Unfortunately it will be quite some time before I will be able to figure out how to write a compile from scratch, so maybe this a non-starter…but I can definitely say…if it has to be translated to mmd or pandoc markdown on the way to asciidoc…then it defeats the purpose. I’m looking for a way to compile from scrivener to asciidoc…

The reason for asciidoc is in order to have nicer looking HTML based blog posts… Hugo and other static site generators support asciidoc. Event gitlab supports it. And its much more capable then any of the other common markdown forms…unless they have been heavily extended…in which case there is no standardization. So for publishing through the web…asciidoc just makes a lot of sense…but not if it has to be watered down through mmd or pandoc first, that is pointless.

Asciidoctor can also produce PDF and other formats… So it may or may not be needed to go through pandoc to produce those…but my understanding is that pandoc does only a so so job of translating asciidoc into HTML. Using Asciidoctor is probably how that should be done… The only reason for using pandoc would be in order to perhaps do some kind of asciidoc to LaTex thing, or to docx or something…which is definitely interesting also…but in any case… the fundmental question is how to make use of asciidocs superior structural capabilities…not how to hard code pass through HTML or pass through latex, etc. I want to write in scriv, and compile asciidoc which has the full structural capabilities of asciidoc. Then from there most likely will use Asciidoctor to generate HTML and/or potentially pandoc to generate book form eventually.

It doesn’t sound like many people here have actually spent that much time with asciidoc, but I’m a little baffled that scrivener isn’t already all over it…its far superior to mmd and pandoc markdown.

[Dewdman42]

unless perhaps there is a workflow where pandoc custom fences can be used to basically describe all the structural aspects covered by asciidoc…and then use pandoc with probably some Lua filters or something to finally generate ascii doc with that additional structure? Just thinking out loud…but already its sounding like way over my head in terms of being able to implement it, so I will probably resort back to using VSCode and Asciidoctor for my blog work…

[Dewdman42]

I did find this just now. It’s 11 years old, but I will have a look at it to see what they did.

https://github.com/jorhett/scrivener-asciidoc

[nontroppo]

@Dewdman42

asciidoc provides much more capability and flexibility in terms of structure and eventual formatting then MMD or pandoc’s own markdown so if I go through MMD or pandoc on the way to asciidoc, I lose that additional structure capability which completely defeats the purpose.

Correct, asciidoc has deliberate markup for things like admonitions etc. You nevertheless can pass through asciidoc for features that are not supported. For example, lets take a warning admonition:

WARNING: Wolpertingers are known to nest in server racks.  
Enter at your own risk.

You can use a paragrpah (block) style to markup “Wolpertingers are known to nest in server racks. Enter at your own risk.” — this style will add WARNING: to turn this into an asciidoc admonition. You use plain text compile and this style rule will allow Scrivener text to generate a warning admonition. A different compile format can turn it into a GFM alert without any of your intervention:

> [!WARNING]  
> Wolpertingers are known to nest in server racks. 
> Enter at your own risk.

This is one example, you use styles, get the compiler to inject the markup and this gives you a clean document in Scrivener and the compiler can output asciidoc directly.

@Dewdman42

but its not clear to me that MMD and pandoc won’t get in the way unless some work around has been devised

Yes, a standard workaround exists, you can can erap inline and block markup so that they ignore the text and pass through clean. This is used a lot by LaTeX compile format for example. Scrivener compiler injects this markup, again your editor text remians clean and “pure”:

This is **pandoc** markdown.

```{=ascidoc}
WARNING: This is an asciidoc admonition
```

More *markdown*. 

You see that you can “embed” raw asciidoc in a markdown document Pandoc - Pandoc User’s Guide. So in this case Pandoc would convert the markdown but leave the asciidoc intact. This lets you mix and match formats. For example, my Typst sample project injects native Typst in some places and uses Pandoc elsewhere.

I am not telling you you should use markdown to write asciidoc, I am telling you you can mix the two without issue, using both formats as needed you can pass any extension or feature through. It isn’t any harder, but this may provide a higher cognitive load for you (given you are learning to build a workflow, I understand each bit makes understanding the whole harder): if so don’t use it.

Don’t give up on Scrivener. As I said above the 3 options, option (2) sounds the best route for you as you appear happy to only output asciidoc.

@Dewdman42

…perhaps there is a workflow where pandoc custom fences can be used to basically describe all the structural aspects covered by asciidoc…and then use pandoc with probably some Lua filters or something to finally generate ascii doc with that additional structure?

Yes, that is also doable, if you can code in Lua and you can output multiple formats too. But that is definately not a route for you.

@Dewdman42

I did find this just now. It’s 11 years old, but I will have a look at it to see what they did.

https://github.com/jorhett/scrivener-asciidoc

Looks pretty basic, my feeling is you are better off building your own Compile format from scratch. What asciidoc features are essential to you? I can make you a basic compile format to outout plain asciidoc projects.

[nontroppo]

OK, I made a quick Scrivener project that contains TWO compile workflows, native asciidoc via plain text and asciidoc via pandoc:

Asciidoc.scriv.zip (501.3 KB)

Sample outputs for both workflows in the binder. Both compile formats use the same Styles, so the editor is identical:

The styles get converted to either native markup directly OR markdown with raw blocks for native asciidoc content. It shows native adminitions, lead paragraph, collapsible blocks. It supports Scrivener’s links and images in both outputs. For Pandoc it supports Scrivener’s lists and tables which pandoc converts to asciidoc.

Here is sample output:

Note the leading paragraph has larger font, there is a warning admonition, inline formatting and the table and list (thanks to pandoc).

I hit a small number of issues, but nothing that can’t be worked around.

[Dewdman42]

Thank you I will definitely have a look at this later tonight. How would you approach situations such as source listing nested within sidebar?

I’m guessing all the character styles are automatically handled by pandoc but basically there is bold, italic, mono, mark, underline, over line, superscript, subscript, small caps, big caps

How do we control the asciidoc metadata from scrivener?

Asciidoc tables are significantly different from pandoc syntactically but have many of same features such as spanning columns and rows, but also have more features not in pandoc I believe. Perhaps those should be marked up in scriv though rather then try to convert from scriv tables to pandoc tables and hope pandoc and convert to asciidoc but what are your thoughts on that?

In asciidoc tables are so flexible that I can create what looks like multi column page layout with asciidoc blocks nested inside each column. I can control the widths of each column. Hide or show table borders, etc I made what I consider nicer looking description lists in a table with hidden lines for example.

Image handling in asciidoc lets me control many aspects about floating image to left or right and text wraps around it, sizing, etc. I guess this is metadata issue so again what would be the right way to get that from scrivener through to the asciidoc output?

Callouts I think would have to be marked up in scriv no matter what.

Footnotes I guess pandoc probably translates.

Endnotes?

Well off the top of my head most of the other features of asciidoc I will not use initially but if I can get something going along these lines I can probably figure out how to add and extend it later.

I’m actually not afraid of lua or other scripting at all or anything related to manipulating pandoc’s process. I can program those things, but the mysteries of scriv’s compiling and how to navigate the compile settings ui to get it to do whatever it needs to do is where I’m clueless.

[nontroppo]

I think most of that is doable either route. The one trick I didn’t include but probably will be helpful to you is custom metadata. Scrivener can control custom metadata on a per-document level. So for example you could set the image alignment as a custom metadata entry, and when it comes time to compile, a metadata placeholder is used to contruct the asciidoc markup. This lets you easily edit the values using the Inspector. I use this in my Quarto workflow:

I use a tag ««AB»» with a replacement that injects the metadata at that location required by Quarto.

[nontroppo]

A nice example of how actively pandoc is maintained: Yesterday I found that Pandoc’s markup for highlights [word]{.mark} was being converted into asciidoc with a role [.mark]#word#, whereas the asciidoctor spec suggests it should just be #word#, I posted an issue yesterday and john fixed it several hours later: Asciidoc highlighting not working from Pandoc's highlighting / mark class · Issue #10511 · jgm/pandoc · GitHub !!!

[Dewdman42]

Good to hear. I’m just getting home late tonight, will look through all your responses after I eat something… By the way didn’t [.mark]#word# also work? #word# is the simple version but I think when you put [.mark] in front of it, that assumes the same role…its an unnecessary markup in this case its true. But for example, underline is handled as [.underline]#word# I believe. and small caps and big caps are also handled that way…and you can actually dive into other custom roles as I understand it… But the default case with just #word# is to use “mark”. At least I thought so…maybe not.

[nontroppo]

@Dewdman42

By the way didn’t [.mark]#word# also work?

It didn’t highlight the words in the HTML output for some reason (VSCode asciidoctor extension, HTML preview on the right):