Feature Request/UX suggestion for Markdown workflows

I’ve been a user of Scrivener since 2008 but over time have found myself using it to a far lesser extent primarily because I use multiple devices for writing when working on long-form writing projects; and the current plain text sync folder solution gets broken if you access the same ‘.scriv’ file from multiple OS X locations. It’s fine if you only write on a single OS X machine but as soon as you open that same ‘.scriv’ file on secondary OS X hardware (e.g. using both desktop and laptop installations) the sync folder gets broken and you have to delete the contents and re-sync again.

I understand why this ‘safety feature’ is built in but it’s very frustrating. I think in this case it would be better if the user was given a ‘health warning’ so they’re aware of potential database corruption risks but they can make that decision for themselves (rather than disabling access to the sync folder altogether).

I like the benefits of having my Markdown writings stored within RTF files in Scrivener (comments, external reference file links, highlighting etc) but whilst editing files I prefer to use a dedicated MultiMarkdown editor (primarily MultiMarkdown Composer). If I’m using a single OS X machine for writing my project I can edit the plain text sync files in MultiMarkdown composer and they’re seamlessly updated within Scrivener. I can of course also edit those plain text files from any device (IOS, Android, Windows, Linux or OS X) and everything is hunky dory. However once I access the ‘.scriv’ file from a second Scrivener installation the sync folder is no longer accessible (even by the installation that was originally used to set up the sync folder). This is a huge irritation as I have to set up the sync folder each time I use Scrivener on a different OS X installation. The sync folder workaround wouldn’t be necessary if Scrivener had an ‘open with’ function for text files or if Scrivener had an integrated Markdown editor for those that prefer Markdown workflows. Alas I’ve seen that Keith has no intention of integrating a Markdown editor within Scrivener so the sync folder would appear to be the most appropriate and flexible workaround.

Ulysses doesn’t come close to Scrivener as a writing tool but it’s getting better all the time (especially now that it’s external folder support has been significantly improved). Over time I can potentially see myself moving over to Ulysses if it continues to improve at it’s current rate (even if it lacks so many of the features that make Scrivener such a wonderful writing tool).

Keith has made a few comments in the past about Markdown’s popularity as being something that was amplified by early IOS shortcomings (lack or rich text capabilities) but I and many others have used Markdown long before the advent of IOS devices for a bunch of well documented reasons (and have no intention of stopping using it just because IOS now offers rich text support). It’s a pity that Scrivener doesn’t offer better Markdown support but Keith is free to prioritise the features he sees as being most important on his development roadmap. However there are some very simple fixes that could make the current product far easier to use for those of us that choose to make use of Markdown workflows.

Have you checked to make sure that this folder is in the same exact location on all computers? If any one of them has a different path to that folder (and all it can take for that to happen is a different user name on one machine in most cases) then Scrivener will detect that the sync folder is missing and disable the feature. That’s ordinarily all that is happening. Do note however that if you’re using the sandboxed version of Scrivener from the Mac App Store, then you won’t be able to use this capability—though you could switch to using the version of the software that uses the computer normally, by downloading the demo from our site. It will unlock itself if you have MAS receipts.

The disabling event itself isn’t a safety feature, just a natural case of cause and effect. In the case of the MAS version, it is actually Apple that is triggering the event because machine B has not been authorised to access that location yet (and you can’t do that because the only way to authorise a location is to use an official sandboxing-rated Open dialogue, which won’t allow you to select an existing folder). Otherwise the feature is simply disabled if the folder cannot be found, and that’s the right thing to do in that case.

On the other hand, if everything about the context is the same on all computers, the project and the sync folder are all “the same” thanks to something like Dropbox, then what you are asking for already works.

The safety feature aspect of this is that Scrivener won’t let you hook up the folder sync feature to some essentially random folder with files in it. This isn’t a “project health” thing that needs to be monitored, the project as a system would be fine even if you could try, the content may all be wrong however. That’s why it is simply disallowed, because there is no way for software to distinguish between 15 radical rewrites that are fully intentional and 15 incorrect files blowing away other files thanks to misaligned ID numbers.

Regarding the future, Scrivener will continue to evolve in interesting ways. There will not be a whole different editing mode or “open with”, but better tools in the editor itself, and better ways to take even more advantage of its rich text capabilities. I’ve been writing using plain-text formats, starting with raw LaTeX, for going on twenty years, and I personally don’t agree with Keith that vast numbers who adopted Markdown in the first place are going to stop using Markdown because their phones can now load RTF. Sure, many will, but I do think a lot of eyes have been opened to the fact that one doesn’t need hugely complicated formatting engines and desktop publishing features in order to just write. But, I am an advocate and ideologue when it comes to plain-text writing, I would want to think that. :slight_smile:

My Dropbox folder is on an external drive but sym-linked back to my home folder on my desktop machine (I don’t want 1TB of data clogging up my SSD). This works perfectly for other apps that rely on Dropbox residing in the home folder such as BBedit, (which syncs preferences across multiple installs via Dropbox). My home folder is named differently across my devices but BBedit handles this by using the tilde character to represent the home folder ~/Dropbox/Application Support/BBedit/ (a standard OS practise). I have no wish to change the home folder naming conventions across my devices as I have no problems with any other workflows I use that make use of Dropbox. This would seem an easy enough fix for Scrivener.

I look forward to seeing how Scrivener develops over time. It’s good to hear that there’s somebody on the team to influence Keith with regard to the continuing benefits of plain text workflows. The critical factor for me with plain text is the flexibility it provides with regard to output formats whilst maintaining absolute data portability.

Because of the shortfalls in both Scrivener & Ulysses (with regard to my personal workflow requirements) I’ve developed a set of multi application workflows that have become second nature and provide me with most of the features of Scrivener (but without the benefits of an integrated ‘IDE’). I’ll list a few core elements here - not to show off my ‘clever workflow’ but hopefully because some parts of it may inform Markdown/Plain Text workflows within a future version of Scrivener:

  • Using Moom (windows management application) makes the organisation of multiple panes really simple to manage so automatically organising reference material and editor window panes side by side is pain free.

  • Using MultiMarkdown’s file transclusion abilities negates the need to ‘compile’ individual files via Scrivener.

  • CriticMarkdown enables you to add notes, todo’s, links to reference files etc within your Markdown docs and these are safely ignored when ‘compiling’ your final document. Bret Terpstra’s Marked app works really well in tandem with CriticMarkup

  • Yosemite file tagging makes it really easy to associate reference files by tag to your project (wherever they live on your file system). Path Finder is my preferred method for adding Tags

  • Opening related reference files (via Tags) is very easy if you use HoudahSpot, Ammonite or Yep.

  • MultiMarkdown Composer is my preference for editing individual Markdown files but there are many advantages to managing the overall project in BBedit (text factories, grep, project wide search/replace etc, as well as overall project organisation).

  • TaskPaper formatted todo files (or iThoughts mind-maps) are a great inclusion within your Dropbox project folder as they can be utilised on IOS as well as OS X

This approach obviously works for me because I’m very happy with managing projects via multiple applications but it’s reminded me how much I dislike solutions that rely on file bundles to do their thing. It’s a pity that Scrivener forces you to import everything within it’s bundle rather than taking a true ‘IDE’ approach of referencing files. The main benefit of my current approach is that the results of any action I take to complete my project is also available to any program on my system (and in many cases on my IOS device too).

I am also a fan of the plain-text and Markdown approach – not because of the limitations of iOS but rather because of the simplicity and efficiency.

I would much rather concentrate on the writing than get distracted by the formatting.

And it is really annoying when I end up disliking the default formatting that was applied by Scrivener when a new section/page was created in the binder. I’d much rather set a global font for reading and for writing. The styling can be applied when the finished work is compiled and exported.

So my (very limited) “wish list” for Markdown would be the following:

  • an option to set a global font that applies across all pages in the binder (whether already existing or newly created).

  • inline emphasis for bold and italics

  • inline emphasis for #headers

Markdown and Multimarkdown do feature so much more, but even the above three things would be delightful.

Cheers!

There shouldn’t be a need to change account names or folder structures. Consider using symlinks to create an alternate path to your user folder. For example, on the laptop a symlink in /Users that uses the home machine’s account name to redirect to the laptop account name. Scrivener looks for /Users/homemachine/Dropbox/MySyncFolder and is silently redirected through /Users/laptopaccount/Dropbox/MySyncFolder. I used that technique myself for years, back when I had mismatching account names since so many Mac programs are simply not coded to work with UNIX shorthands. I’ve added a note to look into the tilde thing though—relative links in general are something we’re looking to support better in the future, while this isn’t strictly speaking a relative link, it fits in with that goal and would be a whole lot easier than having to explain to people how to make a symlink. :slight_smile:

Regarding “Moom”, I’m not sure if this is the same thing, but have you checked out Scrivener’s own layouts feature? It’s in the Window/Layouts/Manage Layouts… menu. It will not only save split positions (and existence), but all manner of settings if you wish. Just about everything you can do to a project window will be saved, even outliner columns and whether or not the secondary split has a header or footer bar, if the Binder is showing, etc. I have one that removes all UI elements and maximises the window, mainly I use it for freeform corkboards.

You can already establish links from your Binder to files on the drive (see File/Import/Research Files as Aliases—or just drag in your own aliases bulk, the alias itself will be archived just like normal files, but will also display content in the editor pane if the file type is one that can be). Granted that won’t accept links to text files to reduce confusion—but with text you already can expose the content to automation with the folder sync feature.

                    ⠂─────── ⟢⟡⟣ ─────── ⠂

That’s not really a Markdown thing though, right? If I understand you correctly you’re just talking about the default font settings for new documents in the software.

You may also be interested in this forum thread, which has some example settings you can apply to Scrivener which will make the main text editor look and act more like a plain-text editor. (Note you might want to save your existing preferences to a preset before trying them—there isn’t anything in them you couldn’t emulate in your settings of course). It’s also worth noting that the user interface is very customisable. You do not have to stare at font controls at the top of your editor if you don’t want to.

There are no plans to add syntax highlighting to the software. This concept is not compatible with how the text engine works. If you want a more thorough explanation there are threads in the forum where this has been discussed.

One comment I had though: you are aware that Scrivener can generate heading codes for you, right? It can calculate the number of hash marks based on its outline depth and print the title of the document between them. This is all done in the Formatting compile pane—of course you can just type in headings wherever you want in the editor, but I rarely ever do that with Scrivener. It’s one of the few forms of Markdown it can actually generate.

Indeed, it’s one of my favourite workflow features within Scrivener. The point I was attempting to make was that there are ways to recreate many of Scriveners best features whilst working directly with OS X with no need to import files into a bundle file format - which has the effect of making those files no longer accessible to other applications on ones system. If Scrivener could work more like BBedit where project management is handled within the program yet project assets remain part of the file system that would be my ideal situation.

I’m raising these points more as food for thought rather than definitive feature requests. Programs such as Scrivener, Curio & DEVONthink, which were once at the pinnacle of OS X innovation are beginning to look a little inflexible in this ‘cloud fist/mobile first’ world we now inhabit. And a nod towards plain text workflow principles is a perfect way to excel in this multi device, multi platform landscape.

Just a note to say I absolutely agree with you there Jon. I far prefer to have my files stored in a way that I can access them, and edit them, with applications of my choosing, and multiple applications when that best serves my workflow and options. 8)

Interesting point… as I am currently delving into Markdown because I face that very issue on Android. There is a significant lack of decent RTF file editors (that don’t add crap to the files, or mess with them), and that support settings one’s own folder location, etc. (required for folder syncing with Scriv, as opposed to Dropbox syncing, which many Android writings apps seem to favour exclusively). BUT now that I have got my head around Markdown (easier enough to do, I’d just felt the need to look at it until now) I actually prefer Markdown to RTF for the same reason you point it. It’s a much simpler way to write, with at least some basic formatting, and especially on touch screen devices.

I found your post whilst looking for past discussion on Markdown, how well Scriv supports it, and if not fully supported, whether or not Keith plans for it to. Sorry to hear he does not.
And I agree… Ulysses has come a long way. Checked out the current version last week. It’s greatly improved, and has excellent Markdown support.

I don’t plan to support Markdown because Scrivener already supports it - and very well. A lot of code has gone into the MultiMarkdown exporters so that you can write in MultiMarkdown inside Scrivener and then export to a range of formats - PDF, LaTeX, RTF. And in the future, you’ll also be able to install Pandoc and have Scrivener export your Markdown to .epub and other formats. Not only that, but I have plans to have options in Compile that will convert rich text to MultiMarkdown and vice versa, so that MMD writers can take advantage of all of the rich text export formats, and rich text writers can export to LaTeX and other MMD-only formats.

What Scrivener doesn’t do is convert Markdown typed in the editor to a rich text representation of how it would look when exported - because Scrivener is fundamentally a rich text editor, not a plain text one and that is how it will stay. We’ve always been very open about Scrivener’s design drive, and how it came about from my own requirements and wants, and I am a rich text writer - I personally would never dream of writing using 1990s-email-stlye asterisks and markup in my text (apart from when writing these replies, obviously…). However, Ioa does write that way, and I recognise that a small fraction of writers do, so we supply excellent export tools for such writers.

Ioa is no doubt right that some of those wrote using Markdown because of the limitations of mobile devices will continue to do so, but the fact remains that the vast majority of writers are used to Word and expect that sort of formatting (myself included).

Ulysses is a superb tool for plain text writers, though, and if you prefer it to Scrivener you should certainly use that instead. We’ve always been open about how Scrivener won’t be for everyone, because all writers are different and it’s impossible to cater to all tastes, which is why we provide a links page pointing people towards alternatives, including Ulysses.

As for the op’s problem with external folder sync - at least I think that was the problem - that is something that we’ll be less strict on in a future version. We’re adopting a different internal ID system for documents that will mean that choosing the wrong folder to sync with won’t cause data corruption, so we can be more lenient.

cavalierex - Ioa has already pointed you in the right direction, I believe. Just change the default formatting in the “Formatting” pane of the Preferences.

All the best,
Keith

Hi Keith, good to see you get involved in this thread. The Pandoc & Rich Text options you describe here sound very promising (if they manage the conversion glitch free).

To be clear, the points I originally raised had nothing to do with integrating a Markdown editor with preview capabilities within Scrivener but rather to beter facilitate integration with external editors.

I can’t speak for other people but the reason I embraced Markdown and continue to use it is because it:

  • Is a truly portable data format
  • Enables writers to focus on content and semantic meaning not formatting
  • Provides glitch free export to multiple online and offline formats
  • Fit’s seamlessly into cloud facilitated workflows (I can link to images/rich media rather than embedding them in my documents) and that rich content is rendered via Markdown preview technologies whilst working on my projects

By storing all my research in plain text in nvALT (same database indexed by DEVONthink), and storing all my rich media in the cloud (primarily Dropbox and OneDrive but using iCloud Drive too) I’m able to to access all of my report writing from the last 15 years or so on any device and without requiring a huge memory overhead (my iPad Mini and my HTC One phone are both 32Gb devices and are often used as part of my writing workflow).

I actually value the ability to use RTF in Scrivener and all of the associated benefits that RTF brings (markup, annotations etc) knowing that upon export the MultiMarkdown engine will strip out the RTF information without causing any complications or errors.

This is my biggest bugbear at the moment as it means that I have to reset my sync folder as I access my Scrivener files across different OS X installations. As was highlighted in an earlier part of this thread this could be remedied if Scrivener worked with relative file paths as well as absolute ones.

~/Dropbox rather than /Users/homefolder/Dropbox

I make use of Curio & BBedit across multiple OS X installations (via Dropbox) in this way (and each installation has a different naming convention for it’s home folder) so it would be great if Scrivener could follow this same methodology.

Scrivener as a far superior writers tool to Ulysses but Ulysses is slowly catching up. Ulysses is still not a tool I choose to use (even though I own both the OS X and IOS versions) because it’s a closed system and in many ways breaks many of the core benefits of using Markdown in the first place.

As I stated before, I raised these points more as food for thought rather than definitive feature requests. I believe in the strengths of Scrivener as a writers tool but question some of your assumptions for peoples reasons for using Markdown syntax. It’s likely that I’ll purchase the IOS version of Scrivener when it’s finally made available but overall I still believe that Scrivener could be far more friendly to cloud driven mobile workflows.

Hi Keith,

Should add Fade In to your script writing section of the links page you mentioned. Personally find it better than Final Draft (and cheaper).

Amazed to see a software producer confident enough to link to competitors, and no sly put downs or anything. Kudos!

Cheers
Rob

Thanks for clarifying Keith. I spent a good chunk of the day getting myself up to speed (well, heading in that direction) with Scrivener’s support of MultiMarkdown, and how imports, exports, and syncing is handled.

:question: One question: (for anyone who knows) Is there a way to prevent imports of MMD files from splitting the file into multiple documents? Currently they are automatically split at the #, ##, and ### headings ?

I shall continue digging into the ins and outs of MMD and Scrivener. I have no intention of jumping boat and going with Ulysses. It’s a beautiful app, refined, and clean. BUT (big but) it is still very much lacking in many key features I love Scrivener for. For me Scrivener is an indispensable part of my overall work-flow and book planning process.

Cheers,

Jonathan

Marked (marked2app.com) is invaluable as a companion app to preview (and more) MMD text written in Scrivener.

I use MMD for web writing, as it is the most reliable and cleanest way of creating HMTL content in one app and then moving it to another app that I have found so far.

Scrivener works so well as a platform for researching and writing before copying work to the web. Did find the process a little cumbersome until I learnt to partner Scrivener with Marked.

Looking forward to the developments listed. I’d love to be able to split the editor and load Marked in one part of the split. Though running the two apps side-by-side works well enough. :smiley:

This is a good point, actually. The changes I’ve made so far wouldn’t affect this - you’d still need to choose the folder afresh every time you moved to a new machine (but you would be allowed to do so). I’ve added a note to my development file to look at implementing this. One problem with have is Mac App Store support, where it’s not possible to store relative paths like this, because under the App Store’s sandboxed security system, the user has to have provided you with a full path in order for the app to have permission to access it. There are solutions for that, however (such as asking the user for permission if permission has not yet been given). So, I think this will be a workable solution.

~/Dropbox rather than /Users/homefolder/Dropbox

I disagree with that - I don’t think Ulysses is catching up (in the same way that we would never be able to “catch up” with them, either) because I don’t think the Soulmen are trying to do the same thing as us. The core features I built Scrivener for in the first place are not things Ulysses does or tries to do, although I can see they have made steps in the direction of some of them (images in the side bar, opening documents in different windows, multiple text view).

Scrivener automatically breaks up MMD files, but you can work around this by renaming the file so that its extension is something other than .markdown, .mmd or .md. If you change the extension to simply .txt, Scrivener will import it without interpreting it as MMD (and thus without splitting it up).

The next update of Scrivener will give it a more modern, polished UI. That is only a temporary stepping stone on the way to a much bigger overhaul that has already been in the works for two years, though, and which is probably going to take the best part of the rest of this year at least to finish. So, we’re not standing still, it’s just that some other companies have got their big overhauls out way ahead of us. :slight_smile:

Thanks and all the best,
Keith

@wisdom33

If I import a markdown file (extension MD) via the Import > Multimarkdown File option then I have the same experience as you (files separated by headers).

However, if I do the simpler Import > Files (command-shift-i) then the same Markdown file seems to be imported without any file separation by header.