semantic styles

Scrivener Wish List
1

Searching through the forums, I found a bunch of nice tips by AmberV regarding styles, including defining your own.

I am interested in defining styles by semantic meaning (the whole dividing content and style thing). Working in Mellel, for example, I have a style for book titles, and another for transliterated words (in LaTeX, I’ve defined these in the preamble as well).

If I was to define styles in the Cocoa-included editor’s style sheet (cmd-r, then “other” under the ‘styles’ tab on the left), would there then be a way to have these translated upon export? From my rather rudimentary experimenting, it doesn’t appear so: exporting as RTF, then opening the RTF file in a text editor (Subethaedit) showed that there was no mention of my semantically-titled “style” in the markup.

So, if that is the case…here comes my “wish” (we are in the wish list forum):

A user-definable style palette; or (semantic) style sheet; or (semantic) highlighting. Scrivener is already actually already half-way there, already has a few features that do kind of do this. Let me explain.

Yes, I know Scrivener is not a word processor. I don’t want it to be either. But when working with very long, structured documents (which, after all the card-shuffling, Scrivener excels in), I need to keep tabs of all my semantically-induced styling. And with Scrivener’s built-in ability to export to so many file types (either directly, such as RTF, or through MMD, such as HTML, LaTeX, etc.) it would seem to me that a semantic-based style definition would make most sense.

I mean, in a way, this is already there with footnotes and annotations: when exporting, both the RTF exporter and the MMD know that a footnote is a footnote, and translate/export it correctly.

Let’s say that we were able to define the annotation to do something, to be something else. Or the highlights: where a blue highlight would mean “transliterated word”, a green one “book title” (both of which are usually italicized). Then, when export comes around, the exporters (such as the MMD one, or the built-in exporters) could be tweaked to translate and preserve these meanings, just as Scrivener currently does with the “highlighting” that is the graphic footnotes field.

This would all assist in Scrivener being the app for long, structured writing of the academic and technical variety. I know this wasn’t the original intention of Scrivener, but the writing environment it currently has seems so well suited for for academic and technical writing with such an addition. (And, no, I don’t mean for 1.2).

Any thoughts?

2

I’d love this as well. I think I’m finding the same issue. All I want is to do all my writing in scrivener and be able to choose h1-h6, strong/bold, emphasis/italic, bullets, footnotes etc. and then to be able to export to word or another program and remember not the formatting, but the actual style, which I can then adjust on a whole of document basis. If I’m missing this and it’s there, I’d also like to hear about it.

I know Scrivener’s not a layout program, but there are some areas where formatting is a part of the writing process for me…

3

For simple h1-6, strong, emphasis, and so on – MMD → XHTML is really the route you want to go. The HTML exporter that comes with Apple’s system is really a waste of time if you want proper semantics. It does a good job of emulating the appearance of an RTF file, and that is about it. As you’ve noticed, none of the other exporters understand meaning either. It is all appearance.

The primary drawback to MMD → XHTML is that it is woefully ignorant of footnotes. Sure, they get formatted so that they look all right, but they are not footnotes – just anchored links – and no word processor will know what to do with them upon import.

This is a major crack in the current system (as talazem has discovered). You either get your proper footnotes but absolute no other semantics, or everything else gets semantics but footnotes are junked. The only exception to this is LaTeX – which has a very minimal audience, much much less than those interested in conveying proper style semantics to their favourite word processor. That crowd is huge in comparison.

From what I have gathered, there is one potential solution to this. Mellel has a fairly accessible XML format. MultiMarkdown, if you are not familiar, produces a clean enough XHTML file to make it exceptionally useful in conjunction an XSL transformer script. The extant LaTeX exporter is nothing other than an XHTML → LaTeX XML conversion script. While I have not actually looked at Mellel’s XML format yet, I don’t see any theoretical reason for there being an obstacle between MMD’s XHTML and Mellel. Yes, Mellel can already read XHTML, but remember – those footnotes!

Anyway, if this goal were attained, one could supply Scrivener with the correct XSLT, and essentially export straight to Mellel with full semantic transfer. I did see someone’s project creating a Mellel → HTML converter. Such a project would no doubt be a good place to start any research on the matter.

This would definitely be possible. Already, there are two footnote streams that MMD → LaTeX is aware of: Glossary and commentary. It would not be difficult to create new streams, and depending upon how difficult it is to assign them as such in Mellel’s XML format, these separate footnote streams could be retained upon export.

So, in a way, Scrivener is actually much closer to providing all of this than you might have realised. For basic full semantic export, it is only one XSLT file away from attaining that ability. The nice thing about that is that these XSLTs can be distributed and installed onto your system in such a way that Scrivener will give preference to them, instead of its built-in MMD version. Scrivener is, in a phrase, third-party modular when it comes to MMD exporting. That is one of its strengths that has yet to be tapped.

4

AmberV,

For my very personal purposes, allowing this to happen with LaTeX would suffice. But that is besides the point: I imagine tagging the words with the semantic styles in Scrivener, and this being translated into any of the exporting choices given.

You mention that this is already close for LaTeX, that one could create new streams. Is there a practical way for this to be done now, or do you mean that this would be a simple matter of programming for Those Who Program, if they decided to do so? As for Mellel, it would require knowledge of the XML format; would it also go through MMD?

In short: where do we start? This would be most wonderful, for the LaTeX people (who are probably more than you think), and the Mellel users.

5

For my purposes, I am quite content with the current system as it is. LaTeX works just fine for me, and I have yet to have need both RTF/DOC + footnotes, so the MMD → RTF is fine.

The current MMD system distributed with Scrivener has two footnote streams (I mistakenly classed it as a type of annotation earlier). There are standard footnotes, and footnotes which start with the phrase, "Glossary: ". These get handled as you would expect, using LaTeX’s glossary features.

But let me clarify, I did not mean adding semantics to LaTeX at this point. Most of my post was conjecture about creating a Mellel exporter. To answer your question briefly, it would take a programmer, though not an expert programmer. I am guessing it could all be done with the XSLT, though I am not positive. That would certainly be the most ideal, as any other method would essentially fork MultiMarkdown and require a more complicated installation.

What sort of LaTeX enhancements were you thinking of? What is not already offered that would be of use? Things that I have wished for, and are on my list of things to research, are manually set labels for cross referencing, and indexing. Cross references are probably the most important to me right now.

6

As discussed in another post, by making additions to the “cleancites.pl” file included in the MMD package, one can manually write in LaTeX commands into Scrivener, and have them be exported correctly through the MMD–>LaTeX exporter. But this clutters up the text pane with all these LaTeX commands; furthermore, you are trapped in LaTeX now, since exporting to RTF would also export all those manually typed in LaTeX commands.

What I was getting at with the original post – and would resolve the various issues that have been raised in this thread – would be some type of extensible style palette, or a “styles settings” under one of the menus, that allows the individual user to define semantic styles. The semantic meaning of these styles would then be understood by the xslt files, which would then convert it to the desired output (RTF, LaTeX, HTML, etc.).

So, just as I can now highlight a word with the Footnote marker, and it gets correctly interpreted as such by all the exporters, I would like to have such an ability that is user-definable and extensible.

To give an example from my current usage, I have a fair share of transliterated words in my papers. As it currently stands, I type \translit{}, with the transliterated word between the curly brackets. I have hacked the “Cleancites.pl” of MMD to recognize \translit{}, and not parse it out incorrectly (i.e. not to read it as individual symbols, but rather as a LaTeX command). So when I export, my resulting LaTeX sheet shows it correctly.

In my example, I’m defining it directly as a LaTeX command. The problem, like I’ve said, is that I can no longer export to RTF (without a lot of work of going back and changing everything). What I would like would be to have user-definable semantic styles/tags that have a GUI element – for example, instead of typing “\translit{}”, I would simply highlight the word in a blue highlight similar to the red one used now for annotations (for example). Even user-definable markup would be fine: the most important thing is that the user could define the semantic meaning (“transliteration”) of that markup, and then the various exporters would then know what to do with it.

It’s that last part that I imagine would be tricky; a thousand people could want a thousand semantic style definitions. The key here, again, would be to have a user settings under one of the menus to tell the various converters what to do. So, for example, let’s assume that we can export to RTF, LaTeX, and HTML. We would pop open the relevant menu, set “blue” highlight to semantically mean “Transliteration”, and then define “Transliteration” for each of the three exporters:

  • for RTF, perhaps simply place it in italics;
  • for LaTeX, we would make the LaTeX definition: \newcommand{\translit}[1]{\emph{#1}}
  • for HTML, likewise the relevant HTML tags…
  • If someone were to develop a Mellel XSLT converter, all that would matter would be to preserve the semantic style names, without even defining any meaning (i.e. italics, etc.); if imported to into Mellel, the user would then define the meaning of these semantic style-names in Mellel’s style options.

The XSLT’s would then parse out this information to the relevant XSLT files, just as Scrivener+MMD currently does now with the MMD metadata that one manually enters in the “Multimarkdown Settings…” under the “File” menu.

In other words, you leave all the “word-processing” to the word processors, just as Scrivener’s philosophy is now.

From what I can tell (in my admittedly limited understanding), the only changes that would be required from things as they are now would be:

  • a GUI for creating more “highlights” akin to the annotation and footnote highlights currently available (ex. blue=transliteration),
  • a corresponding GUI for defining these highlights in the particular exporters Scrivener has (ex. transliteration: RTF=italics, MMD–>LaTeX=\newcommand{\translit}[1]{\emph{#1}}, RTF–>Mellel=verbatim, etc…)
  • the creation or modification of a number of XSLT files to allow for these dynamic changes to occur in the XSLTs, and then export the relevant output.

Yes, I imagine this would be a “power-users’” feature (though no more than defining your own MMD metadata is now). It would best be pitched at academic and technical writers. But it would make Scrivener the one-stop writing environment that would allow any user to export to a myriad of word processors in a way even more powerful than now, and that would then save the end-user from having to mess around with XSLT files and the like (like LaTeX users are doing now), or to have to manually mess around in Mellel to fix all the styles upon conversion from Scriv–>RTF into Mellel, all the while preserving the Semantic Styles that the user wants. Thus, if at any point, the user decides that s/he would like to change all the transliterated words from italics to underline, this could be done globally and easily from within any of the word processors (like Mellel and Word) that allow for styles…or even from within Scrivener (by means of the proposed Style Menu) if it is used as the be-all-end-all by simply exporting to RTF.

7

I posted some thoughts on what this could look like from the GUI side in this thread. All of this is really high speculation. Such a massive over-haul of the current system would would be something entertained a year or two from now. While there is no doubt that this would be an ideal way of conveying structural meaning without looking too geeky, for now we’ll have to stick with methods that utilise the current system.

As it stands, you could use colour to mean things. The export engine is set up to pass colour information to the scripts, but honestly that would be tricky because of the way annotation colours are set. You can tell somebody to set their annotation to ‘blue’ to make it into a label – but which blue? There are thousands of blues you could select, and only one would be mathematically correct to trigger the appropriate script.

8

Of course I don’t mean for this to be done right away…next week would be just fine. :wink:

But seriously, I was posting the ideas that came to me as I have been using Scrivener intensively in such a way over the past week, and wanted to suggest them before I forget them. If there is any value in them, perhaps they could be contemplated in a future build.

I was just hoping that the post would be a stimulus for discussion about how such functionality could be implemented, if it were to be…kind of to see how people view it, or would use it.

9

Hi, woah, this is a long post and I don’t have time to read through it all right now. However, I have read the original post, and I have to say that this is something that I do not see making it into Scrivener any time soon, if ever. The whole text system is based on the default OS X one, and I have no ambitions on making any major changes to this at all, and the MMD stuff is really there for structured writing so does anything MMD can. Anything much further than that is really beyond the scope of Scrivener - beyond its original remit, as it were.
All the best,
Keith

10

The recently-announced Nisus Writer Pro will include attribute-sensitive search and replace. So presumably you could do things like using a “font” for bulleted lists, another for block quotes and then after importing into Nisus do a search for the “font” and replace with the appropriate style format.

Dave

11

@Dafu: thanks for the tip; I think (as someone has posted elsewhere) Mellel can do that as well. But what I’m getting at is having such “meta-styles” in the writing environment which is Scrivener, and then being able to easiily translate them/convey them into any of a host of “word processors/document processors” for them to do what they are best at – producing the final document – while retaing the user-defined semantic meaning of the styles.

@Keith: Of course, it is your project, and your writing needs that inspired its raison d’etre. It’s just that the writing environment of Scrivener is superb; it’s the best I (and apparently many others) have come across. And mind you, it’s working for me quite well right now.

But there is a need for a good writing tool for technical and academic writing, that joins between the creative, non-linear environment and style-vs.-content structure. If I was to create my own app (which I can’t), it would be exactly what Scrivener is now as the writing environment, with something like LaTeX as the formatter. Scrivener with all its tools as what the user works in, LaTeX with its precision as the engine behind the scene. The ability to write and then to export out to any variety of formats.

Sound familiar? It’s because that’s what Scrivener (with MMD)already (just about) is. I really think that the semantic style-concept would finish it out, and some tweaking to the way MMD currently is integrated with Scrivener would round it out just perfectly.

Of course, it’s all just thoughts. I’m writing my academic essays/reviews with what I you’ve already given us quite well. And I realize that its a slightly different trajectory than what led you to produce Scrivener. Just give it some thought down the road, as I don’t think it would take that much to allow Scrivener to be the technical/academic writing enviornment par excellence, as well as the creative one it is now.

12

First off, “The rumors of my death have been greatly exaggerated.” While not dead, I have been really busy, and I have not had time to keep up with the Scrivener forums. I do check email and my wiki, so I can be reached. And I welcome questions, and comments about MMD. But there is a high chance I won’t see them here without some prompting…

That said, Talal mentioned this thread, and I wanted to mention my $.02.

I’ve said it before, and I’ll say it again - I think very carefully before each addition to the MultiMarkdown syntax. I think John Gruber did a fantastic job with Markdown, and his concept of keeping things as markup free as possible. When possible, I try to keep to this principle — adding new markup syntax is something I try to avoid whenever possible.

As for user-definable semantic styles - I wonder whether this will “cross that line”. I am always trying to avoid unnecessary “bloat” that could end up discouraging people from trying out Markdown. Already, the math syntax is really pushing this (but I felt strongly about a syntax for math support…)

I am open to suggestions for a Markdown syntax for user definable styles, provided that:

  • the syntax is unobtrusive and easily understood when read in plain text
  • this feature is reasonably useful for a significant portion of MMD users (again, this is subjective)
  • I can be convinced that I might have a use for the feature someday (in order to motivate me to code and maintain the feature :slight_smile:

In the case of \translit, for example, you mention that in RTF it should be italics. What does it look like in LaTeX? What is the benefit of defining a new feature, rather than simply using italics in MMD, RTF, and LaTeX? Whenever possible, I try to re-use syntax. For example, to add epigraphs, I simply modified the XSLT to convert any blockquote that starts a chapter into an epigraph. I did not have to design a whole new syntax just for an epigraph. Can this be done in the case of \translit, and possibly other semantic styles? I’m just asking, not rejecting this possible feature…

I hope this makes sense. :slight_smile:

Fletcher

13

To clarify my thoughts on this issue, because I think some of this is getting misunderstood–

The primary “gap” that exists today is that you cannot get true footnotes into a standard word processor if you go down the MMD road; and you cannot get any form of semantics if you go down the RTF road. So you either have a document that is missing footnotes, or you have a document that is nothing but font changes.

I think there are a lot of people who could use both. So, my idea to create a Mellel XSLT requires:* No new syntax features of MultiMarkdown. This would merely bridge an implementation gap in the existing syntax.

  • No new Scrivener functionality. Again, the tools to create a document with structure and footnotes already exist in the form of LaTeX output, it is just missing for those who need RTF to satisfy an editor or publisher with Word.doc requirements.
    Now, it just so happens that Mellel handles multiple footnote streams. Great, this can probably be done using the sub-string searching function XSLT and nothing more complex than putting "Stream A: " in front of the footnote, as one would do with Glossary. It would be nice if some day Scrivener had flexible notation systems, derived from the current system – a formalisation of colour selection and possible collection of note types and assigning function to them. This is all pipe smoke at the moment though. The key thing is that the footnote gap can be closed with a single XSLT file with no new Scrivener features and no new MultiMarkdown syntax. It is just a matter of building the glue between features that already exist, and are not properly conveyed with Apple’s “quick and dirty” RTF generator that MMD uses.
14

Amber’s last post was about an even smoother export to Mellel. I’m all for that.

But so that the original argument doesn’t get lost: SEMANTIC STYLES. We can already export to Mellel; Amber’s wishes, to be best realized, are dependent upon user-definable, semantic styles. Then when you export, you carry across the meaning (and not just the format) of the style you applied, so that – if you desire – you can modify across the board. (The Nissus Writer Pro feature mentioned above does look interesting, though).

But my point was greater, and I feel that maybe even Fletcher missed it. My “wish” is for (1) semantic styles, with a UI that allows for the each style to be defined (whether palette, menu, or otherwise) as to (a) meaning and (b) how they are represented visually within Scrivener (in the text pane, whether as highlights or otherwise), and (2) the ability for the user to define how each style then gets exported through the various exporters (RTF, MMD, LaTeX).

I was not arguing for a new syntax called “trans”. I wasn’t arguing for new syntax or tags at all. It is much more universal than that. As things stand now, I can open up Scrivener’s package, open the “cleancites.pl” file in a text editor, and include LaTeX escapes (markup) that, when written into Scrivener’s text pane and then exported through MMD, will be retained, and thus understood by LaTeX. I’ve extended it to use syntax I need. What I’m suggesting, is a way to do this from within Scrivener, and instead of my writing \translit{x}, or \booktitle{x}, or what have you into the text pane (the main writing area of Scrivener), I can set a list of styles (just as I can do in Mellel, Nissus, or Word), and then highlight the word that I want that style applied to (just like we can currently “highlight” a phrase as being a footnote, or an annotation). Then, come export time, the RTF exporter or the MMD exporter or the LaTeX exporter would known how to translate it into their syntaxes, because the user will have defined that in the style menu (or whatever UI it takes), as I’ve described in one of the posts above.

I hope my idea is clear, now. I apologize if I’ve gone on long enough about this idea, and I’ll be quiet now. :wink: But as a side note, let me mention that Scrivener has been mentioned repeatedly on various LaTeX associated forums and boards (such as that of BibDesk), and the first question that is asked is: does it support Styles? So I think there is a big interest for something like this in the academic and techinical-writing community.