Exit strategy: Does Scrivener negate MultiMarkdown?

The plight of a co-worker, who has about 4,000 files in a format that is now defunct, has highlighted for me the fact that it may not be enough for an application to be capable of exporting into common formats. Even if a batch conversion is possible, every file must still be touched to restore some of the fine tuning. I am therefore attracted to MultiMarkdown, since the files can be made pretty when required, but are otherwise stored in a presumably future-proof format. It seems, though, that this advantage exists only when the MM files are stored as plain text. I am attracted to Scrivener’s environment for the same reasons as other members of this forum, but wonder if saving MM in the Scrivener format doesn’t negate one of the main advantages of MM. Wouldn’t I be better served (regarding this issue) writing with a text editor? This isn’t intended as a disparagement of Scrivener. I’m not a code-writer, and am just trying to get my mind around this problem before I generate a lot of content. Obviously, there are experienced MM writers who are using Scrivener and don’t see a problem. I would appreciate any enlightenment they can provide.

Thanks,
ydkm

Like you, one of the things that really drew me to the MMD format was its “future-proof” format. Plain text with open source standard parsing engines. You really cannot go wrong with that combination.

As for Scrivener vs. a text editor: It definitely comes down to document size, in my opinion. There are plenty of MMD files that I create that never go in Scrivener, because they are small documents with simple organisation. But a large project, say a book, will be greatly served by an organisational application such as Scrivener, which can export to MMD plain format for archival once the project is complete. Besides keeping track of plot points and section status, the matter of document structure is made much easier with Scrivener as well, since it automatically generates headers when you compile. It is possible to adjust section order, create chapters, move chapters between parts of the book and so on—and all of the ‘#’ syntax will be taken care of for you.

Even with a project as “small” as an article can benefit from this sort of treatment. A user working in Scrivener with the intent of producing MMD documents can either choose to ignore all of the formatting features supplied, or they can use them as a form of meta-data that will be ignored upon export. Granted, some features simply do not work, such as list generation and tables. These will produce unreliable results in MMD, and standard plain-text methods are preferred.

Another advantage for individuals with a lot of figures is Scrivener’s ability to keep resources all together. You can link to images by dropping the file into the appropriate portion of the image syntax, and Scrivener will not only create the path for you upon export, but it will dump all referenced images into a convenient directory with the MMD file. This way you can avoid long absolute paths in your source document.

There are certain areas where a tuned text editor (such as TextMate with the MMD bundle installed) will make editing MMD files much simpler from a syntax standpoint, so it really comes down to, as said initially, the size of the project. Larger projects that will benefit from organisation can take a hit on the syntax level—but for smaller projects it is simply easier to use TextMate with all of its syntax aids and colouring. Another option here is to set up your system (a bit more tricky with Leopard, but still possible) to allow TextMate to interface with Scrivener as an external editor.

But in general, I feel the two compliment each other beautifully. Scrivener is by far the most beautiful interface I have encountered for large scale projects, and MMD is by far the most stable format in terms of longevity and flexibility. I find that for the most part, the lack of syntax colouring and so forth is mitigated by Scrivener’s interface—and frankly in an average project you don’t run across that much pure syntax. In a small help document, sure, but in something like a book—for the vast majority of the time you’ll just be typing like ordinary.

ydkm,

Scrivener documents are stored in plain text. They are rtf documents in a package, and rtf is plain text which, like multimarkdown, just uses a markup system to encode things like typeface, text-decoration, etc.

I suppose that the driving force behind Multimarkdown is not in its being future-proof. It began life with Markdown which was conceived as a low-mark way of producing HTML code. Since HTML code is already future-proof in the sense you are speaking of, future-proofing is no part of the going-in idea of the thing.

I agree that having the stuff in a simple, non-proprietary format is a great feature, but, as I see it, the central points of Multimarkdown are two: i) easy, low-mark coding, and ii) multiple translators for re-purposing the data.

Best,
Greg

P.S. If you really want to be nervously future-looking, it is well to remember that your “plain text” files are encoded, too. Just think of all those plain-ASCII files (1 byte per char) out there that are going to be kabonged when the conversion to Unicode (2 bytes per char) is really complete and software stops building in backward compatibility as a matter of course. Oh sure, something will convert them easily enough, but as you say, all bazillion files would still have to be touched to do it. Ironically, the newspaper on your front stoop may have more longevity than your plain text file!

Thanks to Amber, for pointing out that it’s not an either/or situation, and just because I have a hammer, it doesn’t necessarily follow that every job is a nail. Thanks to Greg for clarifying the purposes of MMD, and bursting my bubble about text, resulting in total paranoia. Our organization has data on paper going back nearly 60 years, all of it still completely usable, while digital data less than 15 years old are in jeopardy.

The rtf format of Scrivener is comforting at face value, but if you’ll bear with me a little further, let me probe this by providing a hypothetical scenario. Let’s say that you have developed 1,000 files in Scrivener. Contrary to his intent, circumstances force the developer to drop the Scrivener project. About a month later, Apple goes public with an OS upgrade – from Leopard to Alley Cat. For some reason, you are strongly motivated, perhaps even compelled, to upgrade to Alley Cat, but the word is that Scrivener won’t run in Alley Cat. How would you cope with this situation?

(I apologize if I have now moved the discussion in a direction inappropriate to the Markdown section of the forum. But I don’t know how to transfer it gracefully to another section.)

ydkm

It’s an interesting point. Charles Stross, who has been seen wandering the forums from time to time, brought up a concept in one of his books where the age we are living in will be referred to as a “dark age” in the distant future, because all of our data is stored in forms that degrade fairly rapidly, and in formats that go out of style in a manner of decades. In the distant future, nobody can read anything we’ve produced, and consequently are largely clueless as to what life was like.

My work here is done.

–greg

  1. Apple’s implementation of Time Machine will have been perfected in Alley Cat, and so your Scrivener processes would just be run in the distant Leopard past. If anything, this would be better, because Scrivener on Alley Cat via Time Machine would be able to deliver to you a zero-latency user experience. Now that’s what I call “Back to My Mac”!

  2. Running Alley Cat, one would just say, “Alley Cat, find all projects of type Scrivener and copy-convert each to current Beeblebrox format.” Or, “Alley Cat, hereafter, handle all projects of type .scriv in my Leopard shell.”

  3. Of course, since most of humanity would be using Scrivener by then, Alley Cat would never be released without Scrivener compatibility–or would be dumped like a three-legged mongrel dog, sending Apple into a well-deserved downward stock spiral.

  4. Besides being able to run the Scrivener compile or export functions on any pre-AlleyCat Mac, someone (Keith or someone channeling Keith’s enstellated spirit), responding to popular demand, would write a simple utility to export the files from Scrivener in a binder-structured and binder-named way. Couldn’t be too hard.

  5. The apocalyptic Alley Cat could not hose the OS X package system without ruining, well, most every third-party thing along with Scrivener projects. So, given that Alley Cat preserves the package structure, you are in luck even in the worst case scenario where you need to retrieve your text manually–opening a Scriv project to retrieve the rtf docs yourself is just a control-click away (not that anyone should be messing around in there before the apocalypse). And, of course, there would be no reason one could not batch the process.

The short answer is, I think Scrivener puts your data into as close to a millennial-safe form as you could have on a current computer system and that still enables you to do the job of writing well. It just doesn’t get any better than this.

–Greg

P.S. Of course, personally, I am printing all my stuff out in micro-barcode on acid-free newsprint with strict instructions that it all be launched into stable orbit upon my demise. Well, and then there are the stone tablets under the Rocky Mountains for back-up.

As a special parting gift, I once printed a student’s entire dissertation on a single sheet of paper–with very narrow margins and (something like) a 2 point font setting–and you could actually still read it without a loup, micro-endnotes and all. (And if I have thought to convert it to all caps beforehand, it would even make veritable easy reading.) I wouldn’t want to feed it to my current OCR software, mind you, but someday…

Okay, not probably the sort of gift most doctoral students would appreciate receiving at their defense party, but hey, I think it went over big time. Really.

–Greg

Man, you just get through making sure Scrivener is running perfectly on Leopard then people demand compatibility with imaginary Alley Cat operating systems - sheesh!

Actually, I don’t see why this situation would alarm you. This is the whole beauty of Scrivener’s format:

  1. The .scriv format is a package format. A package is just a folder that looks like a file on OS X, because OS X knows about packages. But if you copy a .scriv file to a Windows machine or a Mac that doesn’t have Scrivener installed, it will just look and act like a regular folder. So, say Alley Cat for some inscrutable reason did hose support for packages, all packages would just appear as regular folders, just as they do on Windows, so you would still be able to access their contents.
  2. Within the package/folder, all text files are stored as RTFD. RTFD is Apple’s own version of RTF, so it’s unlikely they’ll stop supporting it any time soon. But even if they did, they would just appear as regular folders with a TXT.rtf file inside (see 1, above).
  3. You could therefore open the RTF files in any editor that supported them. Even in the now really unlikely event that everyone suddenly stops supporting RTF - despite the fact that it is pretty much as standard as plain text - RTF is just a plain text file with markup. So you could still open all of those files in a plain text editor and extract the text. It might be painful, yes, but you could do it.

The real question here was, why should you use Scrivener over a regular text editor. The answer is simple: if you need all of Scrivener’s organisational features - the corkboard, viewing different types of document alongside one another, outlining, notes etc etc. If you don’t need any of these things then using Scrivener would be a bit like packing up the car with sleeping bags and food and then driving the kids next door and unpacking everything for their sleepover. Or something.

Best,
Keith

I’d not previously realized that I could open the Scrivener packages. I tried it with a few items and was able to read them fine in both TextEdit and Nisus. So, I feel better now. But that acid free paper in a hermetically sealed pouch stored in a bomb shelter still has some appeal. If people can’t read our stuff after we’re all dead, we’re really going to be disappointed. Thanks to all for soothing my angst.

ydkm

Be careful mucking around in the package though. As accessible as it is, there is a degree of project synchronisation going on that can get messed up if files are edited independently and directly from the Finder. As a fail-safe, it’s great to have, but until such needs arise it’s best to forget you can do it. :slight_smile:

No problem. We’re talking about apocalyptic needs only.

Thanks,
ydkm

That’s a bit extreme. Without getting overly technical, one of the representations for Unicode is UTF-8, and ASCII is a subset of UTF-8. So as long as Unicode supports UTF-8 (which will be quite a long time), you’ll be able to read ASCII files without modifying them. And, as you suggest, even if reading of UTF-8 is not supported, converting ASCII to UTF-16 or UTF-32 is trivial.

On the other hand, as soon as we run out of fossil fuels and the sky is scourged so that sunlight can’t get through, you won’t be able to turn on a computer anyway, so anything other than paper will be unreadable. And you’ll have to read that by firelight. :slight_smile: