Compiling to BBCode for Forum Posts

I know similar features have been asked about before, but most of the posts are fairly old and I feel like I’m missing something on the more recent ones, so I’d like to try one more time.

A lot of what I write gets posted to forums and copying large blocks of text from Scrivener’s output can be hit or miss when it comes to formatting. Those forums also support bbcode, which would allow for far more consistent results. Given everything Scrivener’s compile function can do, it seems like it should be easy to arrange things so it’s out put replaces rich text formatting with the appropriate tags. Unfortunately, after checking the manual and google, I can’t find a good way to handle it. So far, I’ve seen four potential workarounds, but each one has it’s own, fairly significant, flaws.

Copy as BBCode: By far the simplest of the solutions; this would just require selecting the text, in scrivener, and then clicking copy special>copy as BBCode, at which point it can be directly pasted into the forum’s input box. Unfortunately, my writing tends to be highly fragmented, with pieces spread across multiple files at multiple levels. If I understand things correctly, I’d need to manually select and copy the contents of each file, one at a time, which would drastically slow the transfer and generally forsake a large part of Scrivener’s utility. Also, this function may be going away?

Use Custom Style: I actually expected compile styles to be the solution when I first started looking into this and the prefix/suffix functionality would certainly work for most common items. However, it seems like they require using character styles for formatting, instead of the normal functions. Unfortunately, the styles are a lot more cumbersome to set-up and use, when compared to the normal formatting buttons, which which would distract from the actual act of writing.

Compile Into a Close-Enough MMD Style and Then Use Find+Replace to Fix the Tags: This is probably the closest to working, but it adds an annoying extra step and introduces more opportunities for errors, if for no other reason then it’s fairly easy to mistype a find+replace string, miss something you need to convert, or accidentally convert something you shouldn’t.

Pandoc output filter?: I’ll admit, the main problem here might just be that I don’t fully understand what’s being suggested. In an earlier post, AmberV gave this solution:

After poking at the MMD and pandoc options, and checking the manual, I didn’t see anything that would let me set it up to produce BBCode. I think they were suggesting a more automated version of the find+replace solution, using external software, but I’m not actually sure. If that is the case, it would mostly have the same problems, with the added complication of needing to set-up and run an additional tool.


It feels like it should be possible to have a more robust solution in Scrivener, especially given the fact that it already can convert into BBCode. I’d like to suggest three styles of solutions that seem like they would work well, in the hopes that one of them is either already possible or a feasible addition. (Or they inspire an actually good solution from the professionals. :stuck_out_tongue: )

Compile to BBCode: This is the most straight forward solution (on the userside). Just add an extra option that spits out a plain text document that swaps rtf for BBCode tags. The main problems, that I can see, are that it relies on Scrivener’s ability to generate BBCode and it has the potential to leave some people out in the cold if they need a different formatting style, just like MMD currently does for people who need BBCode.

Allow Users to Create More Comprehensive Modifications to the MMD Styles: Compiling the the MMD or pandoc formats already requires Scrivener to parse formatting and turn it into tags. If a user could manually specify a different style of tag, that should be enough to solve the problem and allow for new formats without changing scrivener. Even if it requires creating an external style sheet to make the changes, each user would only have to do it once (or get a copy from someone who’s already done it) and it shouldn’t be too hard to set up, assuming a reasonable syntax is used.

Allow Formatting Options to Act as Pseudo Styles: This is what I initially expected to do. (That, or set one of the page wide styles to handle formatting options differently.) Like I already said, the prefix/suffix setup that styles already have would solve most of the basic cases and, combined with an option to escape special characters, this would make it pretty easy to compile most documents into an arbitrarily tagged format. It could also allow for interesting, if niche, uses, like replacing italics with a different typeface or bold with larger text.

I’d suggest having a look at Section 21.5 in the Scrivener 3 manual. The same functionality that supports Markdown output can be repurposed to create whatever markup you like.

Katherine

Firstly, you’re going to need the beta version for this. If you don’t want to take that step, then just know this stuff is coming, for now.

The approach I would take, were I to hypothetically restrict myself into using one single tool, is create a custom plain-text format, using the provided tools for doing so. The central compile tool for doing this is the Markup pane, available to “Plain Text (.txt)” alone, and as such the documentation for that pane also serves as a central hub that points you to other features that could be used to create syntax (you’ll need to reference the Mac documentation for that at the moment).

As noted in the yellow tip box on the following page, one of our project templates is a live example of how these various tools can be put to use. It turns Scrivener into a native LaTeX syntax generator, which fundamentally is not too different from BBCode in terms of the rules you follow. You just need markers inserted around the text and that sort of thing.

It would be a fun project, and I do believe Scrivener could support most basic Bbcode, and handle some advanced stuff with styles.

Just to mention, for Pandoc at least, you can use a custom writer that does the conversion for you:

github.com/iandol/dotpandoc/blo … bbcode.lua

Original here: github.com/tajmone/2bbcode

I use this with Alfred where I have a script that takes markdown written in a web forum and converts it to BBCode (I did it for this post 8) ), the script copies text to clipboard, runs it through pandoc using the custom bbcode writer then pastes it back in (it also expands references so I can get a bibliography if needed!):

echo $argv | pandoc –csl ~/.local/share/pandoc/csl/apa.csl –bibliography ~/.local/share/pandoc/Core.json –filter pandoc-citeproc -t bbcode.lua

This could easily be rolled into a compile format (Scrivener 3+)

The original poster has supplied an interesting constraint of only using one single tool for the production of BBCode. I’m not sure how far that is meant to be taken—when I copy BBCode out of Scrivener, it is done through a keyboard shortcut that is intercepted by a macro utility (Keyboard Maestro), which uses the shell to execute Pandoc against a custom Lua filter I wrote (I took some ideas from the filter you posted).

So while technically I am using four (five if we count the shell itself?) different tools here (Scrivener, macro tool, Pandoc, and a tool for translating data between the shell and the system clipboard)—I don’t really see how much that matters to myself at this point. I press ⇧⌃C and then paste BBCode over in the web browser. The only difference between that and regular Copy is one modifier key. So if from my perspective that is the only difference, does it matter if the computer is using one or fifteen tools behind the scenes?

If one does lift the constraint, but doesn’t want the macro overhead, then they could make use of the Processing compile format pane along with Pandoc to achieve a very similar result (and similar lack of daily awareness of there being multiple tools in place—much like, for instance, DOCX exporting in Scrivener isn’t something people consider to be a multi-tool workflow, but absolutely is).

Of course all of this does also require one to have at least one toe dipped into the Markdown waters. The original concept of building one’s own compile format would allow for a more “word processor” oriented approach to things.

Wow. Thank you for all of the quick replies and help.

I think I might be missing something. All I saw in that chapter was a section on how compile folders work, a brief bit about the formats that Scrivener already compiles to, and information on how to add meta-data to the top of a file. Is there a specific part I should be looking at or could you walk me through how you’d change bold (or whatever tags Markdown uses for bold) into the bbcode equivalent.

Already got it and it’s even up to date. :mrgreen:

Thank you for pointing me towards the plain text section and the mac manual, it does seem to have more information. But, again, I feel like I’m missing something.

The Markup pane itself only really helps with italics and links, since it doesn’t seem to touch other types of formating. Of the other panes mentioned in the section, only styles, replacements, and post processing seem relevant. Styles have the same problem I mentioned earlier and transformations seem like they’re focused on replacing strings of characters (unless there’s a way to use regex to detect things like bold text). Processing could maybe do something, but I’m less sure how to approach it.

As for the example project, if you were refering to the “General Non-Fiction(LaTeX)” template, it was definitely interesting to look at and it gave me a better idea of how styles and placeholder functions work. However, it also seems to rely on formatting via the styles dropdown, rather than the normal formatting commands, unless I missed something that enables them? For example, it has Emphasis and Strong styles, which to me implies you use those over italic and bold formatting.

I should probably clarify my reasoning behind that constraint, both for sanity checking and because realistically not all tools are the same. I’m primarily concerned by four factors:

  1. Additional overhead, both in maintaining the extra tools and in spending resources to keep them running in the background.
  2. Potential to interfere with other software or activities (like a macro intercepting a key press in other software).
  3. Extra time and steps taken to launch external tools (if they’re not already running in the background).
  4. Losing the ability to handle special cases through Scrivener. For example, if Scrivener was doing the compile, it could be set to leave the text in a code block as is, if you wanted to demonstrate something like markdown formatting in your post. I don’t know if that can carry over into an external tool.

A macro utility trips almost all of those concerns, but running a command in powershell or a script in the processing pane (which I think would just pass it to the powershell or command line?) only trips the last one. Running a file through something like the the latex engine would also be relatively OK, particularly if it was included with scrivener, since that would also remove the need to maintain it separately.

That said, it would be nice if this could be done purely through Scrivener’s normal interface. There are so many features that come close to doing the trick without actually doing it. I swear the markup page is taunting me with it’s italics, and only italics, wrapper settings. (Ok, it can handle links too, but what about bold and underline?)

I’m actually not familiar with pandoc. I was assuming that it functioned similarly to HTML, as a language that some programs happen to interpret at runtime to produce human readable content, rather than something like latex, where you’d write a document in it and then use a dedicated engine to process it into the final form that gets distributed. Now I’m thinking it’s more the latter and I’d need to set it up separately if I wanted to do anything with it, right?

I haven’t used lua before either, but it seemed fairly easy to interpret in those two custom writers. My main concerns are that I’m not sure where/when those functions are being called, nor do I understand a few of the choices that were made, like converting strike-throughs to underlines or not supporting tables. I wouldn’t normally worry too much about individual choices like that, but I don’t know if they were made due to limitations in pandoc or something else.

Sorry, I didn’t realize you’re using Windows. I’m referring to the Mac Scrivener manual, which you can download here:
literatureandlatte.com/lear … ser-guides

As Ioa said, the LaTeX template would be a good place to start.

Unfortunately, our documentation assumes that you’re already familiar with Markdown, which it sounds like you might not be? But, briefly, you can pass raw markup through the Compile command untouched, then apply whatever post-processing you want.

Katherine

Pandoc is not a language (though it supports an extended variant of markdown), it is a document converter. It basically has an “abstract syntax tree” of components of a document, and allows transformation from one type of componenet to another (e.g. text or \textit{text} are html and latex componenets for the abstract idea of emphasised text). Lots of documents formats can go in and out, as long as the intermediate abstract componenet exists it should get converted (and things like underlines, which don’t have a representation, get lost).

Specific to your question, you simply write in Scrivener using styles (V3 allows translation of styles to markdown), and compile as markdown, and in the post-processing pane set up pandoc to convert the compiled markdown to BBCode.

Not all components in markdown have equivalents in BBCode and visa versa, for example I modified the writer so inline code just gets turned into dark red bolded text. Some BBCode editors allow [font] settings, but others don’t so I modified it to be more generic. These are details, and the point is the lua code is easy to tweak to your preference. BBcode is not really a standard, and so it is hard to know which syntax works on which forums…

Scrivener 3 is super flexible and so it can do the conversion itself (as AmberV’s LaTeX compile format demonstrates). If you don’t want to depend on Pandoc, a Scrivener native solution should work. Indeed because of the great flexibility of using Styles in Scrivener, you can just swap compile formats and with the same document source use either a native or pandoc compile.

Did I mention, I love Scrivener 3 Style+Compile? 8)

I probably should have mentioned that at the start, sorry. Fortunately, both versions of that section seem pretty comparable.

Markdown’s the formatting language Reddit uses, right? That’s about the extend of my interaction with it, but I think I grasp the concept, if not the full list of capabilities. I understood that you can get Scrivener to spit out documents formatted with markdown (or html or LaTeX), the issue was figuring out how to get it from that formatting style to bbcode. Does markup have a special advantage in that regard, beyond being relatively simple?

Good to know.

Is it possible/feasible to add intermediate abstract components for things like underlines or are you better off sticking with the stuff it does support.

I was mostly trying to avoid using styles for the more basic things, since I’d like to preserve functionality like ctrl+[u/i/b] and not have to rely on drop downs, That said, if I was going to use styles, I’m not sure what the advantage of going through markdown or pandoc would be, rather than just having scrivener add the appropriate prefix and suffix. That’s what I was going to do for things like quoteblocks, anyways.

Are you Tristano Ajmone then? Thank you for the explanation. I was beginning to suspect that BBCode might not be entirely standardized, since xenforo’s version does support things like inline code and there’d be no reason to replace it with colored text if you could reliably use it. You’re right, though, that lua file would be pretty easy to tailor to my needs.

I will admit that Scrivener’s style system does seem quite flexible and well designed. I just wish it could apply to unstyled formatting (or that the features that handle unstyled italics could also apply to bold, underline, and strike-through). grump

You can’t modify the abstract syntax tree itself, but Pandoc has the concept of generic inline and block “containers” that can be named, and these can be used for custom conversions. So for example, you can use generic warning/info/alert blocks and this can get converted into the correct syntax for HTML / LaTeX admonition boxes etc. In your case, you would use an underline style in Scrivener, this would convert to [text]{.underline} in the markdown and that could be converted to underlines in BBCode. But this requires you to know a bit about how Pandoc filters or Pandoc writers work. The Pandoc Google group is very helpful, but there is a certain amount of geek-knowledge needed if you want to create custom solutions…

At least in macOs, you can easily remap [ctrl][c] etc. to trigger the correct style, not sure if Windows can do it natively but I’m sure there is some tool to do that.

Yes, if you use Styles, then you can get Scrivener to do the transformation directly, and you’ll learn about the compile system. Pandoc is way more powerful, but just for BBCode output only, it isn’t necessary. I was just offering you an explanation of what it was and how you could use it, not suggesting it was the only way to go 8)

No I’m not, but my version modified his. I never learnt Lua, but have created several Pandoc filters using it as it is similar enough to other scripting languages I do know.

BBcode is a mess, every forum software makes its own extensions, nobody ever tried to standardise it. Forums should switch over to Markdown, as it is more elegant to write, standardised, and with a much bigger ecosystem making it more interoperable…

As I mentioned above, at least on macOS, it already does :wink:

Eh. I am generally capable of geekery, so that’s not really a problem in and of itself. The bigger question is how much effort I want to put in to streamline forum posting. If I actually decided to go with pandoc, what you’re describing doesn’t sound like that much extra work, but the whole things seems like the kind of overkill that’s liable to have me haring down rabbit holes, rather than actually getting things done.

If I understand correctly, that’s reliant on how macOs handles hotkeys, and it’s ability to create custom hotkeys for individual programs, rather than anything inherent to Scrivener. Window’s doesn’t have a similar function. (It would also force you to include bold, italic, and underline styles in every project, but that’s less of an issue.)

It would be neat if Scrivener could let you assign shortcuts for different styles on it’s own, but the windows version, apparently, doesn’t even have shortcuts for every normal function yet, so I wouldn’t expect that addition any time soon.

Huh? I thought the feature you mentioned above was the ability to remap the shortcut key, not applying styles to unstyled formatting or having options for more than just unstyled italics?

Oh, i may have misunderstood what you meant be “unstyled formatting”, I thought you meant just being able to use the standard unstyled format toggles transparently (i.e. remapping keys), sorry.

But Scrivener can convert unstyled formatting to markdown which would get you what you’d want (use RTF formatting and it outputs as markdown that can be transformed). But that wouldn’t work if you used the native Scrivener route, whereas styles work with every possible solution…