Very difficult questions Re: Project workflow with Scrivener

I’ve searched this forum, other forums, posted on other forums and Scrivener groups, read several books and watched several Scrivener videos with no success finding an answer.

I'm new to this forum and decided to come to the source for a possible answer. Don't know why I didn't think of this first.

I've finished 5 nonfiction books in Scrivener and want to control the formatting for output to ePub, MOBI and PDF via Calibre. My 5 books are of various pages ranging from the smallest of 80 pages to the largest of 280 pages. I want to have as much control of the formatting as possible.

I don't use Word ( I use Pages, which lacks some of the formatting help that Word provides) and every solution to what I want to do uses Word for cleaning up formatting prior to going to Textmate and/or Calibre. I'm attempting to use the book, "Zen of eBook Formatting" that kind of addresses what I want to do but uses Word.

Here is what I desire to do: Add markdown to my finished Scrivener documents and clean up formatting, doing search and replace and RegExp, that I believe Scrivener supports; Export or copy and paste to Textmate; Use Textmate to add minimum css styling to center titles and other text, change line spacing and other simple css styling; Export or copy and paste from Textmate to Calibre for formatting as ePub, MOBI and PDF. Yes, I know I can just compile in Scrivener to various file outputs. 

Biggest question; is it possible to do this workflow? If yes, what would the process be?

1. Go back and remove the formatting in the documents by coping and pasting the whole document or parts, cut the document/parts to the clipboard and then paste back without formatting into Scrivener.
2. Add markdown to the now non-formatted Scrivener document. Export or paste the markdowned document to Textmate to add css styling. (If I export the file to my desktop which format should be used, multimarkdown, mjultimarkdown html or web page html or something other to open in Textmate, I'm concerned exporting to Textmate will add some unwanted formatting) Or copy and paste to Textmate without exporting the file to desktop and opening in Textmate for css styling.)
3. If I can get to this stage, copy and paste to Calibre should not be a problem for the final output.

I realize that going back and adding markdown to each book will require a lot of work/time, but I'm not able to see a better way to do the workflow otherwise. I should also add I'm relatively new to markdown and css but confident I can do it with no problem. 

After the year or so of writing my books I'm desperate to get the finished products out as soon as possible. With all my future projects I'll be using markdown from the start.

I would appreciate any help, thoughts, ideas and direction that would help me with this workflow, or thoughts etc for a different workflow that allows me to have maximum control of the formatted books without using Word in the process. I'm feeling like this is the best place to find a solution.

Sorry for such a long post, but I figured it would be best to give as much detail as possible. I hope this will also help others who have a similar desire.  

Thanks, 

Z

See Chapter 22 of the manual, on Scrivener’s support for Markdown. As discussed in Section 22.4, the Compile to Markdown options will strip RTF formatting from the manuscript for you. So you shouldn’t need to bother with Step 1 of your proposed workflow, just add Markdown commands to what you have as needed.

I’m not familiar with Textmate, so I don’t know what it will do to a Scrivener/Markdown-created HTML document. Why not just try it and see? After all, you’ll still have the original in Scrivener if needed.

Obviously make sure to back up everything before making major changes to your manuscript.

Katherine

Hi Katherine,

Thank you so much. Reading Section 22.4 and this helps a lot. Requires less work than I originally thought by Compiling in MD. Seems like the most time consuming will be adding markdown, I have a lot of italics and bold throughout the text.

From what I’m reading it looks like I should compile in MMD and not MMD HTML? It also looks like I should be able to export the finished markdown as a MMD file and open with Textmate, since it will be a clean formatted file.

Textmate is a Mac only text editor and works well with Scrivener.

I’m feeling much better, I think I’m getting to where I want to be with this workflow.

Thanks again,

Z

There’s a menu command for converting bold & italics to markdown: Format->Convert->Bold and Italics to Multimarkdown. Since it’s under the “Format” menu, I think it will only work on a selection of text, but you can load your entire manuscript into the editor using Scrivenings mode and select all the text with CMD-a to accomplish that. Give it a try and see the results (back up the project first, using File->Back up->Back up to…)

You can check if it got them all by using the Edit->Find->Find by Formatting menu and searching for bold or italics. Note that because you can violate the technical requirements for markdown with rich text formatting, you’ll want to do searches for the markdown characters to be sure none of the *'s are surrounded by spaces.

In addition to the above advice, another thing to consider is that MultiMarkdown is essential Pandoc compatible, and Pandoc comes with an ePub converter. You’d want to read up a bit on it, as there are a few things you can do to make your life easier with it, such as supplying a stylesheet up front, publication meta-data and a few other tricks. The result is pretty vanilla, but that’s a good thing if you want control over the look and feel. ePub can in turn be converted to Mobi with Amazon’s KindleGen command-line utility (or even drag and drop, with their Kindle Previewer, which is unfortunately very cranky with 10.10 in my experience). I do recommend that over Calibre’s converter, as it only handles either KF8 or legacy Mobi, not the hybrid that KindleGen produces (you can load the latter on anything from a gen 1 Kindle to the latest devices—technically modern Kindles can load legacy Mobi files, but the result will look much nicer with a KF8 half, not to mention it being more future-proof). Combined with something like Sigil or Calibre for tweaking the ePub output, you may be able to cut hours out of your workflow if you were thinking of assembling the e-book component files by hand in TextMate.

So what that workflow you would compile using the plain MultiMarkdown output option in Scrivener, that’s what Pandoc will use to make the conversion to ePub. Otherwise, if you want to control the process yourself, using the MMD->HTML converter may work fine for you. One very important thing to consider is that MMD’s HTML output is HTML5, which is not ePub v2 compatible, and ePub v3, which does use HTML5, is not always supported on devices. Pandoc gives you a choice between v2 and 3 as a command-line flag.

Oh my. Thank you Robert. I’m seeing things that I’ve seen before like in Format->Convert but I never really saw them, if that makes sense.

This I know will greatly reduce the time for markdown. It looks like the majority of the things I need to do for markdown can be done easily with Scrivener. I was prepared to hand markdown each occurrence of bold, italics, and maybe a few others. Wow.

I know now where to come to find real answers to my Scrivener questions. I learned a lot through my research over the past two weeks but I wish I would have come here first. Would have been much less stress.

Hi Amber,

I was replying to Robert when I saw your response. Thank you. I’m looking at Pandoc now, and I can see how it will help and save a lot of time for what I want to do. As you say, I may be able, actually I don’t see why I wouldn’t at this point, go from Scrivener to Pandoc and there to KinGen. Pretty vanilla is good, I like things simple. I’ve heard there may sometimes be problems with Yosemite and the previewer, I’ve only played around with it recently, but didn’t notice anything. But that’s not saying much.

Compiling in plain MultiMarkDown and converting with Pandoc, with tweaking with Calibre (not sure what tweaking I would need to do at that point) and then to KinGen looks like a ticket to go with.

I’ve got my Markdown cheat sheet (still learning Markdown :laughing: ) and I’ll give this a try.

Thank you again, the two of you (and others who may stop by and respond) have really helped me a great deal.

Thank you so much,

Z

Don’t know if you would need it in your workflow, but my Markdown workflow around Scrivener has been enriched with Brett Terpstra’s Marked 2 app.

marked2app.com

Thanks Briar,

I actually got Marked 2 about 2 weeks ago. Haven’t really tried it yet, did play around with it to see how it would work with Scrivener. Worked fine from what I can tell, but I didn’t really try and do much with it.

Tried downloading Pandoc but it’s not opening/working on my Mac. Most likely user error.

Would you mind sharing how you’re using Marked 2 and how it’s helped your workflow?

Thanks,

Z

Pandoc isn’t something you’d “open” necessarily. It is a CLI tool, you interface with it using Terminal. Sorry if that’s what you mean, but if it isn’t, try opening Terminal and typing “pandoc -h” to get started.

Hi Amber,

The file I downloaded from github didn’t work. Found another from the source. Installed went fine. Made it to step 4: Using pandoc as a filter, and having a problem.

Not sure if I loaded the screen shot correctly. WhenI get to the last part of Step 4 and type Ctrl-D, my dock hides and unhides. the markdown in the terminal doesn’t change. No idea what the problem is.

Thanks, Z

The standard keyboard shortcut for showing/hiding the dock is Cmd-Opt-D. Are you using a macro utility perhaps that has remapped it? You can go into System Preferences/Keyboard/Shortcuts/Launchpad & Dock to see if the “official” shortcut is Cmd-Opt-D.

Dave

Thanks Dafu,

Not sure how it happened but it was changed. Changed it back to the default and everything is working. Opening up the users guide to start learning Pandoc.

Z

AmberV, out of curiosity, had you guys thought about adding pandoc to scrivener in the way you connect with MMD?

Whistles innocently. 8)

Strokes his beard contentedly 8)

Thanks everyone.

I’m making progress. Finished cleaning up and reformatting first book. Working on second.

Backed up second book and saved book as markdown to my desktop in order to open in Scrivener. When I go to open it in Scrivener file is grayed out, tried saying with a scriv extension and that didn’t work. So I’m doing something wrong. Searching manual I didn’t find what I’m doing wrong.

I want to open a compiled markdown project to do a cleanup of format. How do I save it so it can be reopened in Scrivener.

Thanks,

Z

Ok, so I’ve figured this out, kind of.

I imported the file into a new project template instead of trying to open it in a new project template. The only problem / question I have is there a way to import the file and keep the oridgal binder formatting; folders, front matter, text beneath, etc. The files imports as a new folder with two files meta-data and a file containing the complete book.

Do I have to copy and paste the text into folders and text files in the binder or is there a way for it to happen when I import.

I use a simple custom template, 3 folder hierarchy.

Z

What is it you are attempting to accomplish with this method? It sounds like you’ve compiled your project to a Markdown file and then are trying to make a new project from that file, but why not just go on using the original project? At any rate, you should be using the File/Import/MultiMarkdown File… menu command, and for best results you should format the text file with headings using hashmarks, because that is what Scrivener can take advantage of, turning the heading structure of a document into an outline. Maybe you forgot to enable full heading output in the original compile, if it is all coming in as one chunk.

Again though, I don’t really understand what you mean to gain by doing that. Markdown (or even MultiMarkdown) is a very simple format used to describe a document, not a Scrivener project. There is nowhere near enough information in it to reconstruct your entire project.

Hi Amber and thanks,

I think I went about this kind of backward. Learning as I go.

My original document was not down with markdown. I saved and backed up the original and used markdown. Here’s where I made my mistake; I used markdown on the document and saved as a markdown compiled file. Thinking correctly that format would be stripped and I would have a clean markdown file. Which I did have. I then imported that file back into Scrivener and continued working on it in markdown. I should have just used the original and completed the markdown, compiled and used that file in textmate to add css, etc.

So what I did was copied and pasted from the imported markdown file into a new scrivener project with my custom template and continued my markdown. I’ll now save that as markdown and open it in textmate. I see now that I added far to many extra steps that I didn’t need to. I have 3 other books to do and won’t make that mistake with them.

I’m not sure about enabling full heading output, I’ll check the manual and see what I can find about that. Yes, I did use hashmarks. I think this has been making something much more complicated than what I needed to do. But I’ve learned from my mistakes.

Thanks,

z

I wanted to mention that I am trying to learn Pandoc, not a lot of tutorials, especially video tuts on Pandoc. If anyone is aware of some good Pandoc tutorials please share with me.

Thanks