rich text driven "code hiliting" for markdown in editor?

Scrivener Wish List
1

howdy!

been writing a lot in the new scrivener iOS and am just loving it. thank you thank you thank you thank you!

i really love the way scrivener handles everything - and in the ios, the manual syncing with dropbox is just sooooo perfectly well implemented (as opposed to automated [sounds good on paper] constant syncing in the new ulysses ios) and also the fact that you pretty much gave us a scrivenings mode!

so amazing stuff!

but as i write, i am reminded of how much i (still) dislike rich text. :slight_smile: sorry sorry… it’s a personal flaw, i admit it. i’ve written about some of the quirks that i find distressing in other posts asking about best practices in rtf but my heart still does long to have all the sweet features of scrivener and yet be looking and working in the editor like i would in editorial or ulysses - that is, having the markdown codes being previewed in the editor in real time.

currently, there is a way to use a menu operation to apply rich text on the tags but that is not a real time operation and would be a clunky way to write.

is there any chance that we can get that kind of real-time “code hiliting” in scrivener using the rich text engine? in my head, it would completely rely on the rich text engine that is currently in use in scrivener and just be looking for tags so that if it type:

bolded text
italicized text

i would see in the editor:

bolded text
italicized text

(i think some programs show the final effect when you close the tag but i think it might be better to start showing it when you initiate)

and for the #heading ##heading tags, they’d invoke different font sizes. and you’d still have the tags visible in the editor (which is ideal imo) but you just get the rich text engine giving you a preview of the effect.

and this could be implemented as a MODE inside of scrivener. so you can have rich text word processor mode which is the way that scrivener currently works where you can hilite words and apply formatting to it that way and then a… i dunno what you’d call it… a markdown mode? where you can type structured text with tags and have the editor format the tags.

you can totally write markdown in scrivener now - just ignore all the formatting stuff and write your tags and have the compile engine turn all that into final form.

but as in code hiliting for programming languages, it’s really helpful for me to SEE some of the formatting in the editor. i’m writing fiction now and sometimes working with fountain screenwriting markdown but just like with code hiliting but with formatting instead of text color, i find that seeing the effects in the editor really helps me to break up walls of text and see what’s what.

also, because scrivener is rich text, it could be like the best markdown editor ever in that the user could conceivably make all the “effects” look exactly how he wants it to look even to the point of altering colors or making our own tags. heck depending on how involved the parser gets, it might even make scrivener a very viable CODE EDITOR!

and per the discussion here about darkmode

viewtopic.php?f=4&t=35115&p=217135&hilit=ulysses#p217135

i think this separate MODE of scrivener would be a great place to implement that as well - to override the rich text color attributes in order to achieve the desired end result… that may feel anathema to a traditional rich text editor but would be fine in this “markdown mode”.

so is such a dream worth nursing? imo, that would make scrivener like the ULTIMATE solution that is in my mind literally the BEST OF ALL WORLDS. as it is, i keep wishing for the lovechild of scrivener and ulysses but honestly i think this would be better.

(and hey, scrivener already compiles to latex, if you go with the ability to do actual code hiliting for programming languages, maybe you add scrivener compile modules to c++, python, etc… just make scrivener the go to for every conceivable use of text on a computer)

but that’s just me… if this is definitely not in the cards, shoot me down. i’ll get over it and learn to love rich text…

thanks!

jin

2

FWIW, I too would like to see a markdown editing mode added to Scrivener. I got spoiled in the Bad Old Days pre-iOS Scrivener – almost everything about trying to use Index Card sync or External Folder sync to work on iOS was a kludge, but I miss plain text editing. Seriously.

OTOH, I doubt that L&L will consider it. Keith has expressed his preference for rich text, and frankly as a retired developer it looks like Yet Another Feature in an already feature rich app. I like the Binder and Compile enough to tolerate rich text – especially as I can manipulate compile options so that I can use markdown syntax for bold and italic even though they aren’t displayed right. So if markdown editing never happens I’ll still use Scrivener :wink:

3

oh yeah, i’m sticking with scrivener too. it just has too much else to offer.

if this is a no go, i’m just going to have to reconcile myself with rich text once and for all…

jin

4

I thought the point of MD was that you did NOT see the final formatting while writing? If you want the editor to show the final “rich text” formatting, the only difference is essentially that you use different keyboard shortcuts. Or am I missing something about MD?

5

Well, yes, that is part of the theory, Lunk. But most of the MD oriented text editors show an inline preview of bold, italic, the various header levels, tables if it’s got them, and so forth. No font preview unless the editor offers an HTML preview service – but the text isn’t editable in an HTML preview.

Part of the point of MD is that the “keyboard shortcuts” are themselves plain text and can be edited. You can see exactly what they affect, including white space – no more struggle to find out if that paragraph break is italic or not. You change italics to bold by adding another set of surrounding asterisks and so forth. And of course the ultimate goal is that you can print the raw text, and have that be human-readable. (Ulysses fails that test, BTW.)

For me, the last is the most important point. An .rtf or a .docx file is absolutely not human-readable if you look at the raw file content. [rant]And my personal opinion is that the .rtf file format is an illegitimate child of Microsoft and should have been abandoned decades ago. I can’t imagine what possessed Apple to make it the native format of their text kit.[/rant]

Ahem. That said, there are formatting things (like fancy Roman numeral indented lists) that just can’t be done in MD. Image support is primitive. Tables are primitive as well. Those are things you give up when you move to MD – and I’m sure that many Scrivener users aren’t willing to abandon them, even though I am. Heck, they complain about iOS Scrivener’s limitations and it’s not nearly so restricted as MD.

So yeah, for human readable marked up text (with preview display) I’m willing to do without a lot of amenities. Just not without the Binder or Compile, thank you very much. And since I have much of it (without preview) I’m not inclined to complain on my own behalf – but I’ll second someone else’s request. YMMV.

6

right.

and to illustrate, consider:


http://i.stack.imgur.com/WurCQ.png

what’s happening on the left side is what i would love to see in scrivener’s editor. as you can see, all the tags are still there and visible… it’s just that you get bolded text instead of just bolded text .

what’s going on on the right is the final output that would happen upon compile and i would NOT want to see this in the editor.

so for me, having the syntax/tags lit up and bolded just allows me to use such things as organizational elements while i write… with the advantage of never having to take my hands off the keyboard in order to go back and hilite something and then apply formatting… (although i guess a way to emulate my desired workflow with rtf is to bind all the presets to keyboard shortcuts).

also do totally understand keith’s preference for rich text and am asking about this because it seems like this system wouldn’t necessarily supplant rich text… in fact, the display in the editor seems to be would be entirely DEPENDENT on the existing rich text engine… it’s just that there would be a mode in which you can cue the editor to display the rich text effects with tags (so it would be like a regular expression parser in editor?) instead of the current method of mouse selecting a button and/or applying to a selection.

jin

7

I understand and partly agree with the advantage of saving text as .txt instead of in various strange formats like .docx, but I do not agree that .txt is human-readable, unless your eyes do read electrical charges on a computer memory. The difference is that it requires less sofisticated software to read, but it still requires software. My university had (still has?) lots of data stored on big magnetic tapes, most of it in .txt format. None of it is readable because there is no band station with accompanying software that can handle the tapes, and the tapes sure as hell aint human-readable.

But okay, .txt files are more easily read than many other file formats. So with MD and MMD you can have “rich text” but saved in .txt format?

Okay, but Scrivener uses .rtf, so it is an rtf editor, right? So the request is for Scrivener to pretend to be a plain text editor, but using its rtf capacity to show the meaning of the plain text MD codes, and then saving all the project files as .rtf anyhow?

I think I am still missing some point…

8

… or simply use the existing keyboard shortcuts for bold, italics and underline? :slight_smile:

9

actually, for me, i’d need those plus setting up hot keys for formatting presets like BODY, CODE BLOCK, HEADING, etc. but yeah, as i mentioned myself in my parenthetical, that is indeed a way for me to get close to the way i’d like to work.

jin

10

it wouldn’t need to save as .rtf files…

it could be saved as .txt or .md. and in the scenario that i posit, this could be a project wide thing that’s triggered by entering into this “mode”.

and then, if scrivener supported the tag previews, it would simply display them properly with the rich text rendering when next it loads into the editor.

as for not understanding-

  • there are such things as markdown editors. i posted a screenshot from such an app. in such apps, people type simple plain ascii (or unicode i guess) text and the editor displays the kind of formatting as is our current topic of discussion. it would seem to me that they too would use rich text rendering to display this to the screen but i don’t know.
  • some people like working that way.
  • as i said, i personally don’t like rich text. rich text is not the thing that draws me to scrivener. but scrivener has an overwhelming amount of other stuff that i adore… that actually happens not to depend on rich text.

so for me, the request is a greedy attempt on my part to get the best of all worlds. get all the great stuff about scrivener combined with the ability to work as if in a markdown or even text/code editor.

but as i’ve noted, it’s not a make or break thing… just my personal bullet on the wish list. and if keith has no intention of going this way, i’m totally cool with it and will still use and love scrivener for what it is.

jin

11

But what your screenshot shows IS rich text, isn’t it? In the left pane you have rich text, plus the keyboard shortcuts used to produce the rich text, somewhat like what you get in Word if you turn on ‘Show field codes’.

I’m not criticizing, just trying to understand,

12

no problem,

i don’t mind trying to explain myself and i’m not taking it as criticism… :slight_smile:

it’s primarily a matter of workflow…

as you note and as i’m guessing, the left hand panel in my example image may be using rich text TO RENDER the editor area and the text. i don’t know for a fact… it may be using other rendering technique but i am guessing that. but whether it’s rich text rendering or not doesn’t really affect my request.

the key part of my request is how one works. someone composing something in a markdown editor gets the rich text feedback but they work differently than someone who works with a rich text editor.

so currently, without using hotkeys, in order for me to format the text that i see in the editor, i have to take my hands off the keyboard and click on buttons and go through drop down lists for the presets. what i would like to do is simply TYPE.

that is what people do in markdown… everything that is possible to do is simply done by typing.

in other rich text word processors, you CAN make the op codes display… but people don’t create the documents by typing in the opcodes (do they? i’ve never heard of that).

i like the simplicity of that. having to go through menus and buttons seems like a complication to me. other people love it. it’s just a different way of doing it.

one is not more powerful than the other… or if it is, it’s besides the point. it’s just a different way of working.

so in my request, as i said, i’m asking for the best of all worlds - being able to work as i prefer, as if in a text or markdown editor, getting the (richtext or whatever) previewing in the editor… but all i ever do is type.

jin

13

I almost exclusively use keyboard shortcuts, “hotkeys”, to format the text. When I forget the key combination, I just hold down the cmd key and the CheatSheet app shows me the available shortcuts. Very handy. :slight_smile:

[size=85](there are a lot more keyboard shortcuts than just bold, underline and italics)[/size]

But I guess it also depends on what kind of text you are writing. My output can be divided in three broad groups: manuscripts for scientific journals (usually Word format without any formatting except possibly italics), teaching material (usually Word format with some formatting), and e-book manuscripts (usually epub format), so I guess I don’t have much need at all for advanced formatting. All I need is Scriveners ability to handle text, aid me in my authoring.

And to tell the truth, I very seldom format anything at all while writing. I love the Compose mode, where I can focus on only the words. Like cooking. You begin with the major ingredients, and flavor the dish in the final stages. :slight_smile:

14

Like the idea of code highlighting for MD, so long as it can be toggled off (from the keyboard) if it gets to be too distracting.

On maOS, MD works so well in the combined hands of Scrivener and Marked.

…

15

Without wishing to “shoot anyone down” ( :slight_smile: ), there are definitely no plans for Markdown code syntax highlighting in Scrivener. There are several reasons for this:

  1. As noted, I’m a rich text person. I was overjoyed when I could finally use real italics in my emails rather than resorting to asterisks as I had to in the early 90s. :slight_smile: My heart just would not be in any Markdown editor implementation, and such an implementation would be a lot of work. You can bet that the developers of Ulysses, for instance, spent months on their editor alone implementing decent syntax highlighting.

  2. There’s no way to mix rich text and syntax highlighting (which has already been acknowledged by the fact you are asking for a mode, but it’s worth mentioning anyway). With rich text, the formatting is part of the data - part of the underlying file itself. The data contains both content and formatting. With Markdown syntax highlighting, only textual content is saved (as plain text), and the formatting is applied at the UI level. Essentially, Markdown editors use the rich text features of the TextKit, but they parse the plain text to convert it to rich text, and don’t save the rich text to file. These two modes cannot be mixed in a single file - you have to have one or the other.

  3. Having a “mode” in Scrivener would involve turning off all rich text features. The footnotes and comments pane of the Inspector would be useless for such files, because you couldn’t apply comments and footnotes to such text (these features are all based on rich text). You wouldn’t be able to create links between documents, have inline annotations or comments, or a number of other Scrivener-specific text-based features, because they are all based on rich text features. Whole reams of Scrivener’s menus would either become useless in this mode, or all those features would need completely rewriting so that every one of them could do things in two ways. (Insert image? In rich text mode, insert the image using this chunk of code; in MD mode, insert tags using this chunk of code and maybe place the image somewhere in the project. Bold? In rich text mode, call the standard formatting methods; in MD mode, add tags and check for syntax highlighting… And this would need doing for hundreds of features, many of which already have a lot of code.)

Ultimately, it does come down to a design decision. It’s not that a Markdown mode would be impossible, it’s just not something I see as part of Scrivener. It would entail a great deal of work and redesign (it would be nowhere as simple as just “adding a mode”), introduce further complications and involve a duplication of code for many formatting features. I could not implement all of this in good conscience, because my heart would not be in maintaining the MD side of the code.

As you note, you can still use Markdown in Scrivener - but you have to do it the old-fashioned way, without the syntax highlighting. :slight_smile:

As for SD’s comments on RTF, I don’t dislike RTF anywhere near as much. It’s a solid format that has been around long enough that it is nearly universal and can be read across all platforms. It’s also plain-text based, insomuch as it is just a single ASCII file containing text and markup, with even images embedded as hexadecimal data. That said, parts of it are a bit of a mess, and I would love to see a new, more modern standard take over. The .docx format is actually very nicely designed, but it has its oddities. I’d really like to see an XML version of RTF, with images stored inside it as hexadecimal data still. It would even be possible to make such a file human readable, if the formatting were stored separately from the content (although this might increase file size). This could easily be done with something like this:

<text>
	Here is some bold text, and here are some italics.
</text>
<formatting>
	<font range="0,50" family="Palatino" size="14"/>
	<textStyle range="13,4" value="bold">
	<textStyle range=42,7" value="italic">
</formatting>

I don’t think I quite have enough clout to invent such a format and make it universal, however. :slight_smile:

All the best,
Keith

16

For those that want it, it seems to me a certain amount of code-hinting might be reasonably achieved already. For MMD markup that has typed starting and ending markers, it seems to me a combination of character presets and keyboard macros could be designed that both typed the characters and hinted the code – the use of which would be no more costly in keystrokes and sometimes a savings. Even some markup that oe thinks of as having only a start marker fit in this category, for example, headings which are really a pairing of hashtag and return character.

gr

17

Exactly the answer I expected. :slight_smile: Be well.

18

Actually I used to read the ASCII punches in paper tape and translate the EBCDIC on punched cards. Laborious, and I preferred to use a reader, but I could do it and did when I had to…

19

Yes, that would serve to get the formatting in. It wouldn’t solve the problem of editing it – a system of macros would have to be very elaborate indeed for that :slight_smile:

The problem really happens on compile. Take the example of a header. If you’re compiling to not-MMD you’re going to have that pesky # string sitting at the front. If you’re compiling to MMD, the string of # will be doubled as Scrivener will dutifully translate the header preset to MMD for you. You could strip all the markdown codes before compile, I suppose… But that defeats the purpose, because you’d have to go back in and re-apply them to continue working.

Nope, I think living without preview is easier. :slight_smile:

20

thank you much keith for taking the time to issue such a detailed answer.

i can understand your position completely and it definitely DOES sound like a LOT of work. i didn’t expect the idea to be trivial to implement but i definitely didn’t see just how many things rich text touched in scrivener.

oki doke then. i will get cracking on making formatting presets and hotkeys and learning to love rich text, which, when it comes down to it is not much of a burden. i just need to adjust my mindset.

thanks again and as i’ve mentioned in the first post, i am absolutely loving the ios version and the seamless back and forth i have with my desktop. congratulations on an amazing implementation.

jin