Compiling for Kindle and iBooks: Issues and solutions

[Xuanying]

I spent the weekend converting two books, originally in RTF format (NisusWriter), into the MOBI (Kindle) and the ePUB (iBooks) formats. I’ll just share the issues I’ve met and the solutions I’ve found, in case anyone finds them useful. I also have a couple of questions to Keith, or to someone who knows more that I do on these matters.

First, I worked on Book 1, compiling from Scrivener to MOBI. Everything looked fine, but whatever combination I used of original layout, formatting options in Compile, and “overriding” the formatting options, the output MOBI text was right-aligned. Right or wrong as it may be, Amazon does not like this. Trying to isolate the problem, I followed the ConflictCatcher method (remember?) and began to delete portions of text until the output file was left-aligned only. Then I added bits of the deleted portions, until text was right-aligned again. Deleting and adding, adding and deleting, I came to the point when adding a single letter caused the whole book to become right-aligned.

Then it occurred to me that I wasn’t obliged to compile directly to MOBI: I could compile to ePUB, feed the ePUB to KindleGen, and let it convert to MOBI. Result: The whole file was left-aligned only, both on the Kindle Previewer and on a Kindle PaperWhite. I also opened the ePUB file with iBooks on an iPad, and the text was left-aligned. And when I changed the iBooks preferences to Full Justification, text was both left- and right- aligned.

Question to Keith (if you have time to waste): I used exactly the same formatting options for both conversions. Are there differences in Scrivener’s conversion routines to ePUB and to MOBI?

With Book 2, the experience was quite different. Having learnt the trick, I did the same Scrivener > ePUB > KindleGen > MOBI conversions. But while the MOBI version opened perfectly in the Kindle (left-aligned only), the ePUB version wouldn’t open at all in iBooks. The error message was something like “Incorrect Format”. I checked the ePUB file with the validator at epubconversion.com, and the report was that the “content.opf” file was missing. Apparently, KindleGen was able to produce the “content.opf” file from the ePUB version while converting to MOBI, but iBooks would not open the ePUB version if it lacked “content.opf”.

The solution was to use Calibre. I uploaded the MOBI file and asked Calibre to convert it back to ePUB. This ePUB version validated with no errors, and it opened in iBooks on the iPad.

Another question to Keith (is you have any more time to waste): What can the reason be for Scrivener not producing the “content.opf” file? Here I’m mainly asking for advice. The structure of Book 2 in the Binder was a bit complex, with chapter subsections nested within the chapter. While compiling for ePUB, I noticed that a nested file should not begin with its title, or the title would appear twice in the MOBI file after conversion with KindleGen. Can this relatively complex structure be the reason why Scrivener’s ePUB file lacked a Table of Contents, and was impossible to open in iBooks?

PS. Actually I have one more question to Keith: In section 24.11.3 of the Scrivener Manual (version 2.5) you write, “Preserve Formatting only preserves Adjusts how the Format > Formatting > Preserve Formatting feature works”, and you refer to sec. 22.4.4, which is about MultiMarkdown. Does Preserve Formatting also applies when one compiles to other formats, e.g. MOBI or ePUB?

[AmberV]

I used exactly the same formatting options for both conversions. Are there differences in Scrivener’s conversion routines to ePUB and to MOBI?

There are differences in the processes used to generate these outputs. Mobi has some peculiarities that are not present in ePub (and vice versa).

If you could let us know what the single letter was that you traced it down to, or provide a sample project that compiles right-aligned with the culprit letter highlighted in the editor, that would be fantastic.

What can the reason be for Scrivener not producing the “content.opf” file?

As far as I know, there should always be a content.opf file. There shouldn’t be any condition where this doesn’t export, so that must be a bug as well. Again a sample project that reproduces the issue, or instructions on how to trigger the problem, would be appreciated.

I don’t think outline complexity has anything to do with it. Both Scrivener and the ePub/Mobi meta-data systems are designed to accommodate multi-level ToCs and outlines.

While compiling for ePUB, I noticed that a nested file should not begin with its title, or the title would appear twice in the MOBI file after conversion with KindleGen.

Ordinarily that wouldn’t happen, perhaps there was something in the way it was converted that caused this anomaly.

In section 24.11.3 of the Scrivener Manual (version 2.5) you write, “Preserve Formatting only preserves Adjusts how the Format > Formatting > Preserve Formatting feature works”, and you refer to sec. 22.4.4, which is about MultiMarkdown

Thanks, that cross-reference was in error. In fact the features this part are referring to specifically have nothing to do with MultiMarkdown, which is a plain-text format and would not care if the appearance of the text were indented, right aligned, or double-spaced—all of that disappears when you compile to MMD.

This is referring to how you can fine-tune Preserve Formatting to only impact certain aspects of the formatting. By default a blue box around a selection of text in the editor will protect everything about it. If you only want some things protected (like indents for block quoting), then you can adjust how this feature works here.

Does Preserve Formatting also applies when one compiles to other formats, e.g. MOBI or ePUB?

Yes, with the HTML-based formats you can opt to use Preserve Formatting for a different purpose entirely: to inject raw HTML code into the output. This is of course a more advanced usage, and it overrides it typical usage as a formatting protector. If you are interested in that though, you’ll find the option in the HTML Options compile pane.

The correct cross-reference for this section should be §15.4.6 (pg. 214). I’ve fixed this for the next public revision of the user manual; thanks for the catch!

P.S. If you’d rather just send the actual projects in question and would like to do so confidentially, please send them to our standard support address.

[Xuanying]

Hello AmberV,

Thank you for your detailed reply, it was a pleasure to read it.

@AmberV

If you could let us know what the single letter was that you traced it down to, or provide a sample project that compiles right-aligned with the culprit letter highlighted in the editor, that would be fantastic.

It would be impossible to reproduce the whole process… I saved and tested about 30 versions of the document until I got to the point. Moreover, in the meantime my Compile settings have changed. However, I can tell you for sure that the culprit was as an otherwise innocuous b found at the beginning of a word.

@AmberV

As far as I know, there should always be a content.opf file. There shouldn’t be any condition where this doesn’t export, so that must be a bug as well.

Now that I I’ve continued to experiment with converting books, I strongly suspect that the fault was mine. See my answer to your remark below:

@AmberV

While compiling for ePUB, I noticed that a nested file should not begin with its title, or the title would appear twice in the MOBI file after conversion with KindleGen.

Ordinarily that wouldn’t happen, perhaps there was something in the way it was converted that caused this anomaly.

I think the structure of my files in the Binder was not correctly reflected in the Compile “Formatting” window (upper pane). Imagine I have a book divided into 3 Parts and each part contains 2 Chapters. In order to have the titles of “Part 1”, “Part 2” and “Part 3” set in a certain way (each on one page of its own, at the middle/center of the page, in a larger font, etc.), I had made the adjustments directly on the Binder files, and had asked Scrivener to compile those pages “As Is”. This started a chain of other errors (caused by me), too complicated to describe here. This is possibly why, at the end, there was no “content.opf” file.

But one question in this context may be important: If Scrivener builds the “content.opf” file on the basis of the structure and of the titles in the Binder (as it does seem to do), are the title of files to be compiled “As Is” also included in the ToC? My experience, including the more recent experiments, says “No”. If I’m correct, this would be an important point to emphasize in the Scrivener manual. (If it already emphasized, please ignore this. I just missed it.)

There is one more thing that is unclear to me: The 2.5 Release notes say:

@Scrivener Release Notes

Kindle export no longer ignores differences between justified and left text alignment. The “text-align” CSS designations are now removed for the alignment that appears most frequently throughout the text, but are left intact for other alignments. Thus, if most text is justified, then “text-align: justify” will not appear in the CSS but “text-align: left” will be used for any left-aligned text; if most text is left-aligned, then “text-align: left” will not appear in the CSS but “text-align: justify” will be used for any sections of justified text.

I may be dumb, but I understand this as meaning that the “text-align” property does not appear at all in the CSS at the

level, either with value “justify” or with value “left” — but only at the

level where necessary. If this is correct, it should mean that Scrivener lets the Kindle take care of the default alignment, which in the Kindle philosophy is by default (and out of necessity, since the Kindle lacks auto-hyphenation) left-justified only. But then why was my earlier Book 1 fully justified when I compiled for the Kindle?

“Mystery within mystery, and source of all wonders” … 8)

[AmberV]

However, I can tell you for sure that the culprit was as an otherwise innocuous b found at the beginning of a word.

Hmm, it might have been there was another character there as well that you couldn’t see. Sometimes stray Unicode control characters can sneak into files, pasted from somewhere, riding hidden until they make a mess of things.

But one question in this context may be important: If Scrivener builds the “content.opf” file on the basis of the structure and of the titles in the Binder (as it does seem to do), are the title of files to be compiled “As Is” also included in the ToC? My experience, including the more recent experiments, says “No”. If I’m correct, this would be an important point to emphasize in the Scrivener manual.

It could be this is not emphasised well enough. Some of these capabilities were gradually added to the software over time, and the original chapter on creating a table of contents has become a bit dated and clumsy to read as a result.

But yes, the binder name is used. When you turn on As-Is and type in your own title as text in the editor, there is no way for Scrivener to know what you did. There is no formal “title” text in the software—and as you discovered the hard way, the As-Is flag isn’t really meant to be a formatting control. It’s for special pieces like the title page (or indeed a custom ToC, which could be useful to you if you’re using a Binder structure that doesn’t map well to automatic title generation; just make a custom document with hyperlinks to the sections you want in the ToC, and call it “Contents”, Scrivener will use that instead of the generated one) where the formatting is special, and likely nothing like the body text. If all you want to do is stop title output and formatting conversion, you can switch all of that off in the Formatting compile pane.

I understand this as meaning that the “text-align” property does not appear at all in the CSS at the

level, either with value “justify” or with value “left” - but only at the

level where necessary.

Not quite, the HTML exporter isn’t even capable of handling constructs like logically grouping elements into DIVs. Everything is paragraph level. Scrivener just looks for the most common justification style and strips it out, leaving only the oddities like chapter titles and scene breaks.

…But then why was my earlier Book 1 fully justified when I compiled for the Kindle?

This was verified by examining the source? Can you compile with the source export option enabled in the KindleGen compile pane, examine the CSS and see if it is specifying justified for body text? If there is no code for it, it must be a display setting in the reader.