ePub: no more line feeds & strange ToC

Hi,
Something is driving me crazy. I published several times a version of a book to ePub format.
Then in the Compile > Options (the little gear wheel) I checked “Remove trailing whitespaces from document”.
I published.
Since that moment, even if I uncheck this option, all returns in my book between paragraphs are ignored on my Kobo reader.

Instead of
Introduction

Blah blah

Blah Blah2

I now have
Introduction
Blah blah
Blahblah2

If I publish to Kindle Mobi with the same options, everything is fine. Titles have a blank lines between them and the start of the next paragraph.
Funny thing is that if I open the book in Books on my Mac, iPad or iPhone. It’s perfect.

Also the Table of Contents ends by
Guide

  1. Contents

Those two lines appear nowhere. They do not show up in the Mac,iPad,iPhone Books app either.

I tried with another book that publishes fine to my ePub (Kobo). I checked that option, republished, and it’s gone forever, no more linefeeds. Like it ignores all blank lines. There is no cache or stupid things that could cause this.
Any idea why???
Thanks for any light.

How old is the Kobo? I wonder if it’s using ePub 2 with the old ADE stylesheet. In the compile overview screen, click on the general options tab on the right hand side, and test an .epub file with Include standard Adobe Digital Editions page template included.

It’s worth noting that this stylesheet makes the .epub file technically invalid. It probably won’t work with other publishers.

As for the Contents, I don’t really know what to say there except that page breaks generate ToC structure. If they aren’t coming from your compile settings, check the main editor for stray page breaks.

Hi,
I don’t find that Adobe Digital option in my Compile dialog box. I checked everywhere to no avail.
Not even in the preferences…
I have a Kobo Aura 1 (2015) and a Kobo Aura H20 2nd edition (late 2017).
But the problem is that everything was fine before I checked that “Remove trailing whitespace from documents”.
Very strange.
By the way, I have Scrivener 3.1.1 (9852)
Steve

Ah okay, you probably won’t find the ADE stylesheet option I referred to then, as it’s part of the older (and now deprecated) ePub 2 workflow. Both of those Kobos are fairly recently though—do you happen to know if they support ePub 3 in general? I should hope so, but sometimes hardware uses older, more stable tech for a thing that won’t be changing often.

So if you install Adobe Digital Editions on your Mac and open the .epub file with it, do you get the same result? This would be helpful because my old Kobo broke years ago and I never replaced it. So if your book looks good everywhere except Kobo I’m going to have a hard time figuring how what is wrong. No doubt the HTML/CSS of the book itself is fine if everything else displays it normally. It’s probably an odd quirk in the rendering engine on the Kobo—and my hope is that ADE also has it.

But we might as well check that as well. In the compile’s general options area, toward the bottom, you should find a checkbox to export the source files along with the ePub file. Enable that, and go through the HTML files looking for places that display incorrectly. Does anything look odd? Is the syntax broken? Do you get validation errors?

Hi,
I’m not installing ADE but I can tell you that the ePub is fine in Calibrer, in Books on Mac, iPad and iOS…
But it was fine on my Kindle too, I don’t know why it “switched” all of a sudden…
Steve

Since you have the basic trigger identified, I would compile two different copies of the book out:

  • Set Save source files in a folder with exported ePub file in the general options tab of the compile overview.
  • Compile one copy with Remove trailing whitespace from documents enabled, and another with it disabled to a different file name.

You’ll end up with two folders containing .epub files along with all of the loose files that Scrivener creates for them. This will make it a simple matter to compare the HTML and CSS directly, looking for any differences or problems in the output.

I performed a very simple test, but my results were precisely as expected. There were no differences in the .epub save for in the body.xhtml file where the empty trailing newlines had been physically removed.

[size=80][/size]

Both .epub files pass ePub 3.0.1 validation.

OK,
I have no differences. I tried 20 files.
But since you talk about ePub 3… I just recalled that the working versions were published in ePub 2 format.
Now I try to find this option but I see only ePub, no option for ePub 2 or 3… But I’m sure I had that option.
So the only thing that really changed, is that Scrivener has been updated in between. Did you remove that ePub 2/3 option in the latest version? That might be the problem. ePub 3 shows some problems, ePub 2 is correctly displayed on Kobo ?

—added—
Just found a former copy of my VM which has Scrivener 3.0.3 and it shows ePub 2 /3 in the compile dialog box.
Just recompiled my project for ePub2… which actually has the famous Adobe Digital option you were talking about (I didn’t use it).
So the results, I published one in ePub2, and one in ePub3.
Both work fine on both my Kobo devices…
The ePub 3 is better, the TOC is correctly indenting the subsections while the ePub2 aligns the entire TOC to the left.
So my guess is that something changed in the compile method. I’m going to check the differences between the Scrivener 3.1.1 generated ePub and the 3.0.3 generated ePub3 files…
Hope this helps.

—more—
Ok, I compared the cover page between Scrivener 3.0.3 and 3.1.1
In 3.0.3, the paragraph tags (

) are like this:

In 3.1.1 they use a CSS class:

Those are the only apparent differences. Both stylesheet.css files from 303 and 311 are identical though. So to me, it looks like in Scrivener 3.1.1 you have removed the ePub 2/3 option, and you switched to CSS classes implementation for text alignment, etc... in the ePub generation that was not implemented in Scrivener 3.0.3 .

Yes, that’s what I was referring to above, when I mentioned that the ePub 2 method is now deprecated and no longer readily available in 3.1. It is still in the software and can be accessed for projects that had custom compile formats that depend on it. So if you really do want ePub 2 then use 3.0.3 to create a custom compile format that requires only ePub 2 (click the gear button in the header area of the compile format designer to set up its scope). Then when you load that project in 3.1 you’ll have access to the older method along with the Adobe layout option (which is hopefully no longer necessary for anything out there with the functional battery).

Along with the change, modifications were made to the ePub 3 method to better support the kind of WYSIWYG approach you could use with ePub 2. You can read more about it in Appendix E.12, under ePub 2 and Legacy Mobi Deprecated. So yes there will be changes to how the CSS works.

Ok thanks for the clarification,
I will use only 3.1.1+ and ePub 3 as I consider this should be the standard.
But I still don’t get it why all paragraph spacings are gone.
All the


are identical. And the Kobo H2O got a firmware update just about 2 weeks ago, my hint is that they’re supposed to be compatible.
I will write to them with a code excerpt and ask them why the paragraph spacings seem to be ignored (while they are correctly interpreted in Calibre, Books mac, Books iOS…

Thanks
Steve

That is what I would do as well; see if they have any advice to give. Since the .epub file seems to be working everywhere else, and the HTML itself looks good, there must be some other factor involved. I’d be curious to hear what they have to say.

And yes, I think ePub 3 is the right way forward at this point in time.

I’ll post back the answer from Kobo… if any :smiley:

I had a legacy project created prior to the change in 3.1 that used ePUB 2. When I accessed that file today, the ePUB 2 option did NOT appear. So I’m not exactly sure what you meant when you said: “It is still in the software and can be accessed for projects that had custom compile formats that depend on it.” Do you just mean 3.1x will interpret the previous settings, but one can do nothing else? Because, it definitely was not possible to choose ePUB 2 in the compiler or change any settings associated with ePUB 2, like whether tables get converted to graphics or not.

The key concept in my description is custom formats, either in the project or saved globally to “My Formats”. In the latter case, ePub 2 will become available to all projects and the Compile for menu will look just like it did in 3.0.3 even for new blank projects you create.

But if all you did was select a Format, like the stock “Ebook”, and assign layouts, then the project will be transitioned to the new workflow.

OK,
After asking Kobo to position themselves about the ePub 3 compatibility, they did exactly what I expected: tell me they’re compatible and that Scrivener doesn’t follow ePub 3 standards.

They first started to tell me to reboot the device and then make a factory reset. I replied that I’m software and system engineer since 1980… and that they can’t tell me that a reboot is related to the rendering engine, that this is pre-formatted answers that are of no help and they should actually read the question.
Then they told me to submit the case via the Feedback button on their website so I could reach the engineers.
I replied it’s how I contacted them in the first place, and that they just want me too loop from point A to point B and back to point A, etc…
I told them that Scrivener’s ePub 3 files are correctly shown on macOS in Books, on iOS, on Android, on Sony devices, in Calibre, etc… .and the only device that doesn’t correctly render the files are the Kobo. To no avail. They just don’t care.

Now I think that if Scrivener users actually target the Kobo market, they would need to be able to publish to ePub 2 instead. Not all users are former users with a copy of 3.0.3. So you should maybe set a note somewhere with a 3.0.3 download link and instructions about using this version on the side or in a VM.

Take care
Steve

Did they have anything to say on whether or not they still require (or badly need) an Adobe stylesheet in ePub 3 files?

Our ePub files validate using the official W3C ePubCheck utility. So beyond that simple fact, I’m afraid I cannot comment on what they mean by saying such a thing. :slight_smile: Granted there may be some combination of settings and content that produce invalid ePub files, but if that were a problem you’d be seeing warnings in nearly every modern reader. Might as well still be a good idea to run ePubCheck yourself, if you haven’t already, on the .epub file in question. (If you use Homebrew, there is an epubcheck package, and if you don’t have Java installed, the adoptopenjdk cask is what I use.)

To reiterate on a comment I made before, version 3.1 is perfectly capable of creating an ePub 2 format file. We didn’t remove the engine, it is merely deprecated. To get this working to your favour, you would need to download 3.0.3 and do the following:

  1. In 3.0.3, create a temporary project.
  2. Enter the compile view and switch to ePub 2 or Kindle Legacy.
  3. Duplicate and edit the built-in “Ebook” format.
  4. Click the gear button above the pane list and make sure the format is set to both ePub 2 and legacy Mobi, but not ePub 3 or KF8. (Should be how it is set up by default.)
  5. Change where the format is saved to “My Formats”. This will add the dependency on ePub 2 globally, so all of your projects will be able to select from it again.

All right, you can close 3.0.3 and discard the temporary project. Upon returning to your main project in 3.1, you should now find ePub 2 and Kindle Legacy are available in the main Compile for dropdown again.

Hi,
Yes the trick works for people who do have 3.0.3 but for customers starting with 3.1.1 they won’t be able.
But still, I think that a simple note stating that Kobo is not compliant (yet)… would be better. As I definitely think they’re at fault. I’ll do some testing to see if I can nail a reason but I think that Scrivener produced files are fine.

In order to confirm what I suspected.
Just created a 3 pages document: Title, Intro and a page with 3 paragraphs.
No specific formatting besides adding a section break before a header.
Generated the ePub file. Opens and renders perfectly on macOS, iOS and Android. Fine on Kindle after conversion via Calibre. Displays also correctly in Calibre and some online readers (web).
Submitted to IDPF Validation. Result: “Detected version: EPUB 3.0.1 - Results: Congratulations! No problems were found in KoboTest.epub.”

Uploaded it to my Kobo devices: no space between paragraphs, everything is ignored. Now Kobo can’t pretend the problem is not on their side!

Well anyone can download 3.0.3 from the site no matter when they bought the software (or even if they haven’t yet). This is even true for people that buy from Apple, who have no access to older builds via the store itself.

Whether that’s as a whole is a satisfactory approach, I’m not sure about that. Whatever the case, I do agree it is worth a note somewhere.

My guess is that they perhaps have a partial ePub 3 implementation, but not enough to support the version of CSS that is acceptable to that format. We use a newer unit of measurement for most things like spacing, the rem unit, which is a relative measuring based on the overall document scaling as opposed to the font size in the immediate context. It works better in environments like ebook readers that allow the global font size to be adjusted at will. But if a CSS parser doesn’t know what “rem” is, then it might just do nothing—like this.

Just posting that four years later, Kobo still seems behind the curve ball. I’ve got a new Scrivener 3 project, used the epub3 settings, and had a Guide appeared at the end of my TOC with links to the 1. Cover 2. TOC and 3. First Chapter.

Compiling with epub2 settings and no issues.

I have a Kobo Libra H2O running on latest sw update.