Confused about Compile Draft -> Import

I’m not exactly sure how much I should expect Scrivener to be doing for MultiMarkdown draft compiling and importing. I did the following things:

  • Choose File -> New Project
  • Choose Novel Standard Manuscript Format
  • saved the project as “Sandbox3”
  • clicked on CHAPTER and added a sibling folder after it called CHAPTER ONE
  • added a Text Document in the folder called Chapter Subtitle
  • coped the text starting with “Standard manuscript” and ending with “professional designation” out of the Novel MS Format page and pasted into the new text document.
  • Choose File -> Compile Manuscript
  • Pressed (Include All) button
  • Selected Export Format: MultiMarkdown
  • Saved as Sandbox3.txt

Then I tried to import the MultiMarkdown document via

  • Choose File -> New Project
  • Choose default template
  • saved the project as “Sandbox4”
  • Choose File -> Import -> Import MultiMarkdown File…
  • Select Sandbox3.txt

I then get an error message:

Nothing to Import
The file is not a valid MultiMarkdown file - nothing was imported.

Should this have worked? It seems exporting as MultiMarkdown doens’t guarantee a “MultiMarkdown” document is generated (at least from the point of view of the import function).

Assuming I wanted to do the minimum amount of additional steps to make it so I can import what I exported, what else should I have done?

So I tried again starting with an existing “multimarkdown” scrivener file:

  • Download files.fletcherpenney.net/SampleM … cument.zip
  • extract into SampleMultiMarkdownDocument folder
  • Choose Open and select the file SampleMultiMarkdownDocument/SampleMultiMarkdownDocument.scriv
  • Choose File -> Compile Draft
  • Click on (Include All)
  • Choose Export Format: MultiMarkdown
  • Click on (Export…), Save as Export.txt
  • Choose File -> New Project, select Empty Project template
  • save as Export2
  • Choose File -> Import -> Import MultiMarkdown file …
  • browse around to find Export.txt (it isn’t in the directory I would have expected) - it was in a subdirectory.

This leaves me with a new document which has an Export folder containing what appear to be the sections from the original document. However there is an extra Meta-Data document containing

  Title: Sample MultiMarkdown Document  
  XHTML Header:    
  Author: Fletcher T. Penney  
  CSS: http://fletcherpenney.net/document.css  
  Base Header Level: 2  
  XSLT File: article.xslt  
  XMP: CCAttributionShareAlike  
  Copyright: 2006-2007 Fletcher T. Penney.  
	This work is licensed under a Creative Commons License.  
	http://creativecommons.org/licenses/by-sa/3.0/  
  Keywords: MultiMarkdown, Markdown, XML, XHTML, XSLT, PDF,   
  Format: complete

and then when I check the File -> MultiMarkdown settings it only has the default Title, Author and Base Header level in it.

Shouldn’t those settings get picked up and added to the MultiMarkdown menu?

To my knowledge, Keith has not tried to import the document’s metadata into the Scrivener metadata section. The information is imported (as you note), but not placed into Scrivener’s independent metadata section.

I actually think this is the proper behavior - you can import multiple MMD documents into a single Scrivener project, and it wouldn’t make any sense to keep importing all that metadata “live”. It’s better to import it as currently occurs, and allow the author to decide what metadata to make “active”.

The main benefit of Scrivener’s mmd import feature is that it replicates the hierarchy of document sections from the mmd document (provided they start off in a “logical” way). In my (limited) experiments, this has worked pretty well.

It seems I can’t create a new “MultiMarkdown” document from scratch that survives the export -> import round trip. Here is what I tried:

  • Choose File -> New Project, select Empty Project, save as Test1

  • Choose File -> Compile Draft, Select Formatting pane

  • select Single Spacing, press (Export), save as Untitled.rtf

  • select Untitled document, change title to Part 1

  • Enter some text into Part 1

  • select Part 1, right click, choose Add -> New Text, change title to Part 2

  • Enter some text into Part 2

  • select Part 2, right click, choose Add -> New Text, change title to Part 3

  • Enter some text into Part 3

  • shift-click Part 1, Part 2 and Part 3 in the Binder, choose Document -> Convert -> Formatting to Compile Text Style

  • Choose File -> Compile Draft, choose Export Format: Multi Markdown, save as Untitled.txt

Side question: is there a better way to switch to ordinary courier than what I’m doing?

Anyways, when I try to import Untitled.txt I get the

Nothing to Import
The file is not a valid MultiMarkdown file - nothing was imported.

message again. I’m a little worried about using MultiMarkdown if the only way to succesfully get close to doing a “round trip” export/import is by starting with the sample document.

I know I must be missing something. I should be able to create a brand new MultiMarkdown document that can be exported and imported properly somehow - how else could the sample document have been created?

First - email me the MMD file you are trying to import (or post it here) and I can tell you why it won’t import.

And yes, a proper MMD file will “round trip”, but you will have to manually tweak the metadata portion. But more importantly, why would you want to? When I use Scrivener as my file editor, I try to leave the file there, though I sometimes use TextMate to edit the actual text contents of a given section.

I followed the instructions to the letter in my post above. Here is the Untitled.txt file as an attachment.

As to why I want to do this? Well, I’m really just trying to figure out the workflow for using MultiMarkdown with Scrivener in advance of commiting a lot of effort to it, especially since they are loosely coupled. I really don’t want to run into the kinds of issues I’m bumping into now after I have a lot written.

There is also a good chance that other co-workers will want to edit what I create at some time. That either means they will need Scrivener and will very likely come directly to me with all the kinds of questions I’m asking now or they will have to edit the exported MultiMarkdown documents. After they do that there’s a good chance I won’t be able to edit them in Scrivener myself unless I can tell them exactly what to do and not to do.

Once I have my whole workflow figured out I intend to come back and write it all up as a Scrivener document for other people to follow. I’m happy to share it with the folks here too.
Untitled.txt (718 Bytes)

The problem you are having is that Scrivener is not importing the file because there is no top level header. Keith can address this and fix the problem (it’s not a MMD issue).

To fix it yourself, add the following after the header and before the “A place for readers…”:

# Test Header #

This will allow your document to be imported, and it will appear under a text document called “Test Header”

As for your workflow, it makes sense if you are sharing with other users. Depending on your computer-savvy, then MMD actually is an ideal format for collaborating with other authors because it basically a pure-ASCII format with minimal formatting. It’s ideal for version control systems, which don’t always do so well with word/RTF type documents.

Each other can collaborate on the source text file, and also see a formatted version in the file type they prefer (as XHTML or PDF or RTF or Word, or whatever) by processing the text file with MMD.

In this case, you would want to think of the ASCII multimarkdown file as the “primary” document. Scrivener would just be another way of viewing that document.

Well, for one thing you started out by using a template designed for rich text export, and not MultiMarkdown export. That doesn’t mean you cannot use it for MMD, but it would require a few tweaks to work. The main thing missing from the exported file is a header. Add a header into the resulting text document and you have a valid MMD file. That might be viewed as a slight bug, I’m not sure on that. I can see why that is happening. Scrivener imports MMD files by replicating the header structure into binder structure depth. If there are no headers, it doesn’t know where to place the text. I took the resulting text file following your steps and added one line:[code]Title: NovelFormat
Author: Anonymous
Base Header Level: 1
XHTML Header:
body {font-family: Courier; font-size: 9pt; line-height: 2.000000em; }
h1 { font-size: 1.5em; text-align: center; }
h2 { font-size: 1.1em; text-align: center; }
h3 { font-size: 1.1em; }
pre { margin-left: 5em;}
span.annotation { text-decoration: underline; }

Format: complete

Test Header

Your Name
Your address
Your phone number
Your e-mail address

Your agent
Your agent’s address
200 words.[/code]

Note the “# Test Header #” in there. Now import works.

So, all you need to do to make the novel manuscript functional is turn on title export in Compile options. Also, consider deleting the “Title page” document in the Binder. That is all information that would get created automatically using MMD meta-data, using an appropriate manuscript XSLT, such as sffms.xslt. In Compile, on the right-hand side, at the bottom, you’ll see a matrix of things that are exported, “Titles:” is at the very top of this matrix. Enable that for all types just as an experiment, now turn off any “Keep Format” checkmarks in the selection table. When using MMD, this feature tells Scrivener that you want to manually specify the title for that section, and we don’t want to do that since titles have not been specified. Now export again.

Now it should import just fine. You’ll see the Binder order duplicated precisely. The only difference is that everything will be documents and document stacks (documents with documents under them get a stack of papers icon), instead of folders.

Note, while this seems like a fair bit of work, it is really only because we are taking a template designed for one thing, and converting it to another. You could very easily make your own novel manuscript template based on MMD, and save that as your own template.

I’m with Fletcher on why meta-data does not get automatically set. For one, there is some meta-data that Scrivener puts in automatically, and there is no reason to thus have that in two places, but more importantly, you can host multiple documents in a single Project. Sometimes I’ll import the MMD documentation into a project’s Research folder as a reference. I wouldn’t want the meta-data from those documents to reset the global meta-data preferences for the project!

Pffbt. Great minds and all, ha. I suppose I should read the responses before posting next time.

I’m working with Keith on a way to eliminate this bug - we’ll see what pans out.

Thanks for looking into this everyone.

The one thing that I think I should point out is that my goal at this point is to arrive at a working procedure. I’m only posting my failures mostly so everyone knows what I actually did. If I’m going about things the wrong way then I’d rather know what the right way is instead of having the wrong way “fixed” somehow.

What I’d like to do make sure I have a procedure to is:

  • Start Scrivener
  • … do a few configuration steps …
  • … create a simple document with some structure …
  • … save as a MultiMarkdown file …
  • Quit Scrivener
  • Start Scrivener
  • … import the MultiMarkdown file …

After the import, I would expect to have all the information I originally entered with the same structure. Once I have that procedure worked out, I want to know the limits so I can switch to the following workflow:

  • Start Scrivener
  • … import the MultiMarkdown file …
  • … update the document …
  • … save as a new MultiMarkdown file …
  • Quit Scrivener
  • … edit the MultiMarkdown file with an ordinary editor
  • Start Scrivener
  • … import the MultiMarkdown file I edited with another tool …
  • … update the document …
  • … save as yet another new MultiMarkdown file …
  • Quit Scrivener
  • … and repeat …

I’ll know the procedure works when I can do that repeatedly without losing any information or structure.

It didn’t occur to me that any of the templates were designed with the export settings in mind.

Is there another template I should have started with?

That makes perfect sense, except that the only working MultiMarkdown sample Scrivener document I’ve seen to date doesn’t follow that convention ( files.fletcherpenney.net/SampleM … cument.zip - from the MultiMarkdown author’s site).

Thinking about it more, I think for the person who plans on using MultiMarkdown as their primary format, it would make more sense to have the imported metadata replace the existing metadata.

It would be nice if there was at least some way to specify that (e.g. File → Import → MultiMarkdown (replacing metadata) )

Yeah, to most people Scrivener is a rich text based application. In fact, in its original conception there was no intention to support any form of semantic application to text. Then I came across MultiMarkdown, and realised that if Scrivener could potentially easily call those Perl scripts (and it actually turned out to be more work than just that) with the generated text, it would allow those who care about structured writing to use Scrivener for that purpose. Keith was kind enough to entertain the concept and that is how it got in. However, because it is not the central focus of the application, support for it is less integral. As you’ve already noted, documentation doesn’t cover MMD itself, and so forth. Which is fine, since MMD is documented very well on Fletcher’s site, in my opinion. But this is why the templates are all set up with RTF/DOC export in mind, and so on. I would imagine the vast majority of users never touch MMD, as all they need is a marginally formatted RTF/DOC to send to an editor.

Like I said, they can be made to work, but you’ll have to do a little tweaking here and there to make sure data is going out structurally and not just fontastically.

This should be no problem. I assume you anticipate working on your document in multiple environments? Windows one day, Linux the next, that sort of thing? Otherwise, why the import export cycle?

Anyway, whatever the motivation, as long as you stick to plain text orthodox methods, and use Scrivener purely as document organiser/outliner, this should pose no difficulties. It does mean you’ll lose some of Scrivener’s inline functionality, such as annotations and highlighting. Highlighting (as well as other formats, of course) will disappear. Annotations will export, but since there is no inline commentation syntax in MMD, it will not round-trip. Scrivener footnotes will turn into MMD footnotes, and remain that way thereafter, things like that. And of course if you rely on any of the document notes, references, keywords, synopsis cards, snapshots and other meta-data, you’ll lose that too.

These are all things you needn’t worry about if you work strictly from a philosophy of using Scrivener as a hub, generating various end-products via the MMD exporter. It’s only when you get into round-trip that it will come up. Frankly, I would consider other means for working outside of Scrivener. There is just too much benefit in utilising the application’s full potential to be constantly wiping out the Draft and recreating it using the Draft text alone. I suppose if all you need it for is the Draft content, and all of these other features are useless to you, then carry on. Personally, I would just export the Draft as plain text files (using Export not Compile, so they are all in separate files and folders), and then mark them as edited on the non-Scrivener machine somehow (like putting an ‘-x’ in the filename, and updating the changed files one by one the next time you have access to your Scrivener project.

In short, there should be no Draft data loss or heading structure loss doing round-trip. But only do that if you don’t care about using the rest of what Scrivener can do. Excessive round-trip is not really what the application was designed for. Though as MMD users we do have it a bit easier. At least we get our Binder structure cut up the way we left it. The rich text folk have to manually spit everything back up every time. :wink:

I wasn’t clear enough. This isn’t really a convention per se. The importer puts the meta-data information at the top for your reference, so that it can be used if desired instead of lost. The proper way to insert meta-data is to use the dialogue box. Putting it in a document in the Draft will not actually treat it as meta-data. The importer is just doing you a favour. That is why the sample document does not have it there.

I’ve done more experiments and I’m still having problems with the
export/import cycle. Basically what seems to be happening is that
minor transformations in the process are accumulating.

Here is the simplest scenario I’ve found that shows the issue:

In an ordinary text editor, create the file “Seed.txt” containing
the single line:

  # Seed #

Now import that into Scrivener:

  • Start Scrivener
  • Choose File -> New Project, select Empty Project template
  • Press (Next), enter “Start”, press (Create)
  • Delete the “Untitled” document at the top under “Draft”
  • Select the “Draft” stack
  • Choose File -> Import -> MultiMarkdown File…
  • enter or select “Seed.txt”, press (Import)

Observe now that the binder shows

  Draft
     Seed (folder)
        Seed (document)
  Research
  Trash

Scrivener has imported both a folder and a document.
Now export the seed document as Seed2.txt

  • Select the “Draft” stack
  • Choose File -> Compile Draft
  • observe that only the Seed document and not the Seed folder are selected for Include
  • in the lower right there are 3 boxes for Titles: Files, File Groups, Folders. make sure all of those are checked.
  • set Export Format: MultiMarkdown
  • Press (Export), enter “Seed2.txt”, press (Export)

Seed2.txt now contains

Title: Start  
Author: Anonymous  
Base Header Level: 1  
XHTML Header: <style type="text/css" media="all">
body {font-family:  Courier; font-size: 9pt; line-height:  2.000000em; }
h1 { font-size:  1.5em; text-align:  center; }
h2 { font-size:  1.1em; text-align:  center; }
h3 { font-size:  1.1em; }
pre { margin-left:  5em;}
span.annotation { text-decoration: underline; }
</style>  
Format: complete  

## Seed ##

Notice that there is additional Meta-Data in output and also the
“level” of the Seed title is “## Seed ##” instead of “# Seed #”.
Now import “Seed2.txt”:

  • Select the “Draft” stack
  • Choose File -> Import -> MultiMarkdown File…
  • enter or select “Seed2.txt”, press (Import)

Observe now that the binder shows

  Draft
     Seed (folder)
        Seed (document)
     Seed2 (folder)
        Meta-Data (document)
            Seed (document)
  Research
  Trash

Scrivener has now added a folder and two documents and has put the
Seed document below the Meta-Data document. Now export:

  • Select the “Draft” stack
  • Choose File -> Compile Draft
  • only select Seed2, Meta-Data and Seed documents.
  • in the lower right there are 3 boxes for Titles: Files, File Groups, Folders. make sure all of those are checked.
  • set Export Format: MultiMarkdown
  • Press (Export), enter “Seed3.txt”, press (Export)

The Seed3.text now has:

Title: Start  
Author: Anonymous  
Base Header Level: 1  
XHTML Header: <style type="text/css" media="all">
body {font-family:  Courier; font-size: 9pt; line-height:  2.000000em; }
h1 { font-size:  1.5em; text-align:  center; }
h2 { font-size:  1.1em; text-align:  center; }
h3 { font-size:  1.1em; }
pre { margin-left:  5em;}
span.annotation { text-decoration: underline; }
</style>  
Format: complete  

# Seed2 #

## Meta-Data ##

Title: Start  
Author: Anonymous  
Base Header Level: 1  
XHTML Header: <style type="text/css" media="all">
body {font-family:  Courier; font-size: 9pt; line-height:  2.000000em; }
h1 { font-size:  1.5em; text-align:  center; }
h2 { font-size:  1.1em; text-align:  center; }
h3 { font-size:  1.1em; }
pre { margin-left:  5em;}
span.annotation { text-decoration: underline; }
</style>  
Format: complete  

### Seed ###

Now import “Seed3.txt”:

  • Select the “Draft” stack
  • Choose File -> Import -> MultiMarkdown File…
  • enter or select “Seed2.txt”, press (Import)

Observe now that the binder shows:

  Draft
     Seed (folder)
        Seed (document)
     Seed2 (folder)
        Meta-Data (document)
            Seed (document)
     Seed3 (folder)
        Meta-Data (document)
        Seed2 (folder)
            Meta-Data (document)
                Seed (document)
  Research
  Trash

Now if you notice carefully, I think the problems begin when Seed2 was imported:

  Draft
     Seed2 (folder)
        Meta-Data (document)
            Seed (document)

Given that Scrivener works this way, I suspect the “right” thing for anyone
who wants their structure to be preserved would have been to move the
Seed (document) back up under Draft manually and delete everything else
to get just:

  Draft
     Seed (document)
  Research
  Trash

I believe I explained this earlier.

There is also a good chance that other co-workers will want to edit what I create at some time. That either means they will need Scrivener and will very likely come directly to me with all the kinds of questions I’m asking now or they will have to edit the exported MultiMarkdown documents. After they do that there’s a good chance I won’t be able to edit them in Scrivener myself unless I can tell them exactly what to do and not to do.

I appreciate the good intent, but these sorts of favours only cause trouble if there isn’t an easy way to disable and/or compensate for them. For instance, if there were a trivial way to tell Scrivener

  • Move the settings from the added Meta-Data document into the MultiMarkdown settings dialog
  • Delete the Meta-Data document

either as an Import option or as something that could be done after the document is imported, then the original structure would always be easy to preserve.

Unfortunately as it stands this has to be done manually with each import/export cycle or else your document structure won’t be preserved.

Based on my experiences with the export/import cycle so far I’d have to agree.

I confess I don’t understand what you’re getting at here if you’re going to use Scrivener with MultiMarkdown.

For a MultiMarkdown user, what are all these features?

Sorry, I’m new to Scrivener and didn’t know there was an alternative to Compile Draft for MultiMarkdown. I’ll try Export and see what it does.

I think this is what I’m leaning towards. It looks to me that Scrivener will be most useful up front but that for longer term storage and maintainence MultiMarkdown will be better.

MultiMarkdown is also pretty close to the Wiki format our other engineers already know so they shouldn’t have any problem with it.

Is there any existing documentation anywhere for this specific strategy?

EDIT: Indeed there is, I was just looking in the wrong place: fletcherpenney.net/Using_MultiMa … _Scrivener

I just re-read the documentation at fletcherpenney.net/Using_MultiMa … _Scrivener and discovered the following

In order for MultiMarkdown to understand your hierarchy, you must enable exporting of Titles for both Documents and Groups within Scrivener. Scrivener will then add the title of each Group or Document in the appropriate MultiMarkdown format

Doh!

That seems to be about 50% of my problem. I didn’t do that on my first attempt.

Given that this is required, perhaps it should be the default when selecting the MultiMarkdown export format? Or perhaps there should be a MultiMarkdown starting template with all those options checked?