Smart quotes fail on Amazon look-inside

Hi - I’ve published to Amazon from S3, as an ePub 3.0, The look-inside and samples show with weird character sequences in place of quotes. How do I correct this please?
Screen Shot 2018-05-20 at 20.49.12.png

That looks as though the Look Inside feature is trying to read the HTML as ASCII rather than UTF-8. Very strange.

Is there any easy way to test the “Look Inside” feature? I’d like to test this out but I’ve never had any luck in finding a way of previewing the “Look Inside” feature for sample books purely built for testing features.

Thanks,
Keith

Hi Keith

Thanks for the reply. I’ve reached out to Amazon as my publisher has (basically) said it’s an Amazon issue. As yet, I haven’t had a reply from Amazon and thought I’d better ask here in case I’d missed something. Basically, I’ve pushed out in both PDF and ePub formats and both work as expected on my Mac. I’m still awaiting the final print-proof and really hoping that this doesn’t appear in the print version too.

Presumably, I’m not missing something obvious in the compile settings then?

P.S. (And to answer your question) I don’t know how to test the look-inside feature. I’m actually one step removed as my publisher submits the files to Amazon, and I’m not dealing with Amazon directly. Frustrating that the publisher has pushed it back to me to deal with, as (apparently) Amazon will only deal with formatting issues with the author. Not sure if that’s just the publishers coating themselves in digital teflon, though! Not sure how I can test without going through the whole re-submission and release process again.

No, there’s no Compile setting for this (other than changing all quotes to straight quotes, which you wouldn’t want to do). One thing I have read is that the Look Inside feature can start off using an older version of Amazon’s mobi format before getting upgraded after a few days, which can cause it to look bad at first. (I wouldn’t have thought it would affect speech marks, though.)

Is there a reason you uploaded as ePub 3 rather than as Mobi/KF8? I did read one thread where someone said that they had Look Inside formatting issues with ePub 3 but not when they converted it using Kindlegen (which is what our KF8/Mobi option does internally).

I’d be grateful if you could reply here to let me know what Amazon says. “Look Inside” seems to be a perpetual headache, so if there’s anything we need to do our end to improve it, I’d love to know, but at the moment - especially as of Scrivener 3 - Scrivener is producing very clean Epub/Mobi files, so I’m baffled as to what is the problem in Look Inside at the moment.

Thanks and all the best,
Keith

Thanks for the response - the publisher asked for ePub, so I went with ePub3. Unfortunately, I gather the book itself also contains these sequences, so the whole thing has failed at this point. I’ve send them a .mobi file in the hope that they can use that instead - again, it all looks fine at this end.

Nothing back from Amazon yet. I’ll keep you posted.

Ugh! Going round in circles here … This from Amazon:


Hello Mike,

Thank you for contacting Kindle Direct Publishing regarding the Look Inside feature for your book.

Since your eBook was published through a different channel outside of Kindle Direct Publishing (KDP), we’re unable to provide information or take action regarding your book. You will need to contact your publisher directly for assistance with the Look Inside feature.

I hope this information helps and I wish you the best with your book. Have a great day.

Regards,

Sara A.
Kindle Direct Publishing

Argh - although, from our dealings with Amazon, that’s not entirely surprising.

It’s interesting, though, that it says that you are not using Kindle Direct Publishing. That being the case, it may well be something to pester your publisher about, as it may be down to how they are submitting it. How are they submitting it to Amazon? Your publisher is presumably au fait with epub files so should be able to look into this for you. An .epub file is just a renamed zip file, after all, so it is easy to extract them and for your publisher to look at the source. One thing you could do is choose to export the source files when Compiling, send those to your publisher and ask the publisher to deal with the source directly (the HTML files etc). Also find out if your publisher is editing the .epub file in any way before submitting it.

All the best,
Keith

Thanks Keith - really appreciate your comments - I have sent them over to the publisher. It’s really frustrating, especially as I’m a deep-tech-geek! With radio interviews coming up, I really want people to be able to download without issue. It’s working well on iBooks, so it’s not the file per se.

Once I have feedback from the publisher, I’ll let you know (especially if I need to change anything to the file format).

On a side note - thanks so much for Scrivener (and v3 too). It really is an amazing tool. I’ve become an Scrivener evangelist!

Yes, please do let me know what they say. I’m baffled as to what is going on because the HTML and CSS should all be good (and you can check that yourself, as it sounds like you already know being a tech geek!). So my guess is that it’s some conversion tools somewhere along the way. It might simply be that the publisher is submitting as ePub, as I did read that Amazon’s online conversion tools for that can mess things up, so it could be worth asking your publisher if you can’t send them a .mobi (KF8/Mobi) file to use instead.

And thanks for the kind words! (And for telling others about Scrivener.)

All the best,
Keith

Have you tried actually downloading your book from Amazon as a customer? If your publisher actually submitted it as an ePub instead of Mobi, or used a faulty conversion method (assuming that the file type is the problem), I’m guessing you should be able to replicate this by downloading the book from Amazon and opening it up on your Kindle or KindleApp for Mac/PC/mobile. I know you’ve tested the file on your own Mac, but I gathered from your posts you were actually testing the file you sent your developer, not a file you downloaded from Amazon after your publisher had gotten a hold of it.

Not overly familiar with how Scrivener handles exporting (never taken the time to read up on it), but I am a Web Developer by day, so if it ends up being something with the HTML/CSS I could try to help out.

Thanks for the responses - it looks like they’ve submitted as an ePub rather than .mobi and something has gone awry in the conversion. It’s a big publishing group, so I’m surprised they’re having issues. I suspected some weirdness between .mobi and ePub and sent them the .mobi version … Your comments seem to confirm that.

Unfortunately, I’m dealing with the UK arm of a US publisher (would name and shame but I need the business and have gone too far to unwind everything), and the UK arm has a front-desk that sends emails to the US, who send back links to Amazon’s ‘guides to producing content’. All well and good if I had direct access to the Amazon platform - I just revert to shouting and swearing at my computer - it doesn’t actually achieve anything other than keeping the dog amused …

So - I think it’s a publishing issue, not a Scrivener issue (I never suspected KB’s amazing work anyway) - but better to check as many channels at the same time as I could.

That sounds very frustrating! Let’s hope they re-upload using the Mobi file and that fixes things. Do let me know if it turns out there is anything on our end that can be done to improve Look Inside support, though. (Look Inside seems very poorly documented on Amazon’s part. Our Mobi files are created to their specifications, though.)

I’ve just had this response back … Any help appreciated …

“When investigated further we could see that you have mentioned single quotes(’) and special characters(`) in the HTML source code. This is causing this issue. Please remove all special characters and use HTML entities to avoid these types of issues.”

Is there an option somewhere to do this?

Further info from IngramSpark (the publisher) …

They’re suggesting this is an issue with the ePub generator - per this statement back to me:

Hi Mike,

I’m afraid to say that your publishers are wrong on this. In HTML4 it used to be that you were required to use HTML entities for special characters, such as ” for a curly right quote and so on. As of HTML5, though, which is based on UTF-8, this is no longer necessary or required - HTML5 includes the special characters directly, with entities only used for certain things.

Scrivener, as per the Epub standard, uses XHTML 5, and as you can see on this page, this requires that entire references are specifically avoided except for 5 special cases:

wiki.whatwg.org/wiki/HTML_vs._XHTML

So no, there is no way to replace special Unicode characters with HTML entities because this would generate bad XHTML5 and would not be the correct format for Epub 3.

However, I notice that you replied to this thread:

The information there did help me discover one thing: there is a bug in Amazon’s Kindle Previewer converted whereby, when it converts an Epub to Kindle format, it does not treat the XHTML5 format as being UTF-8 by default (as it should do). This results in garbled special characters when the converted Kindle file is opened on a Kindle device. From looking at what Sigil does, I found that this problem does not arise if the XHTML 5 explicitly declares it is UTF-8 at the top of the file (something that shouldn’t be necessary given that it’s a default, but seems to make the issue go away for Kindle Previewer-converted files). I have therefore added this line to XTHML5 files inside Epub files generated from Scrivener as of 3.0.3. Hopefully this will make the issue go away in Look Inside as well as in Kindles.

One thing to note, though, is that this problem does not arise at all if you use Scrivener’s own KF8/Mobi export - the bug is in Amazon’s Kindle Previewer and not in Amazon’s KindleGen utility that Scrivener uses. My guess is that IngramSpark is using the same technology as Kindle Previewer for its conversions (this might include uploading an Epub with Amazon’s publishing tools). As I said in that other thread, it is always best to use a Kindle file rather than an Epub file for Amazon because of certain formatting differences.

As for the cover page, you should just have to choose that in the Compile options. Although it states an HTML cover page, so for that you presumably just need to tick “Add HTML cover page” in Scrivener’s cover page options.

All the best,
Keith

Hey Mike,

EDIT - Removed everything in regards to HTML, as Keith covered it in his response I did not see at first.

As far as the cover…do you have a cover.jpg file in your e-book folder inside the Front Matter section like in the attacahed screenshot (this is Scriv 2 btw - I have Scriv 3, but don’t use it yet since it would make my files incompatible with my Windows Scrivener)? Sorry if that seems really basic, I’m just not sure what all you have done/tried or what your set up is.

On a tangent - why not just submit the .mobi and epub to the online stores you wish? I’m planning on using Ingram Spark as well, so I’m assuming that means you’re an indie like me. But I’m only planning on using them for printed editions of my book.

You can submit to Amazon, the Nook Store, iBooks, and Kobo directly without going through a publisher, as long as you supply your own ISBN. Would it be less frustrating to just to work with the platforms directly using Scrivener’s exporting features? That’s actually one of the biggest draws for me when it comes to Scrivener.

Thanks lots.

So my journey here has been less than seamless, but I think we’re getting things resolved now. IngramSpark in the UK bounce everything over to the US, the US refuse to deal with me in the UK as the local office here should deal with it. This takes an age to get anything done (7-hour time delay means I only have one hour per day when both offices can communicate effectively). When I first became aware of the issue (now over a week ago), my book had ‘gone live’ via Ingram and was appearing on various platforms … it’s working on all (iBooks, B&N etc) apart from Amazon.

As soon as I was aware of the issue, I sent over a mobi version and asked that they use that; and kicked off this thread. Initially, I thought it applied only to the ‘look inside’ feature, but I bought a copy from Amazon and realised it actually applied to the whole book.

Meanwhile on the Ingram side … Nothing happened at all. I ended up asking every day - US kept bouncing me back to the UK (but every time I went to the US, it did trigger a response from the UK) - sometimes many times a day. I originally thought it must be an encoding error and even mentioned UTF-8 -> ASCII to a resounding silence.

Finally, yesterday, I had sent a scathing email to the US, UK, Facebook and cut/paste on their live support channel (snotty response saying the UK are dealing with it, with the support engineer closing the connection when I asked how I could escalate it, given nothing had changed for over a week). I was civil, but asked if someone would take ownership of the issue and get it resolved as it was (is) damaging sales.

Reading between the lines, I think Ingram want to fix their automated systems rather than force an update using my mobi file. This probably makes sense given that I could simply upload a new version via their portal and it will break again. Nonetheless, the book hit #2 on the Amazon hot list over the bank holiday weekend - I have no idea how given the formatting issues - and it really bothered me that people were downloading something unusable. I’m praying that I haven’t got a load of refund demands in the accounting queue!

This morning, I sent over a new version of the book having run through the Sigil process (Ingram only accept ePub, so I’m stuck with that format). Also - per Sparrowhawk’s suggestion - I have clicked on the HTML cover image option. Although the Ingram portal has a separate image upload process and it seems to use that image over anything provided in the ePub.

So - rather hoping this will end well: I’ll keep using the ‘Sigil method’ until 3.0.3 resolves it. In the meantime, I’m waiting for an Amazon refresh to go through in order to see my book in all it’s smart-quoted glory … I think I’m too locked-in to Ingram’s platform to back out easily now - and in reality, I’m happy for them to handle all the various channels if we can get them to work properly.

In the meantime, thanks again for your help folks! I’ll post more as and when I receive it.

Glad I could at least help a little :slight_smile: , even if the main issue is still causing a headache. :imp:

I understand them wanting to fix their processes, but YOU shouldn’t be the one paying for their R&D time. I would demand they push the .mobi and fix it on their own time - or make them pay for any and all refunds caused by their refusal to use a solution you provided them - something you already did not have to do.

Just my two cents - or pence I guess, since I’m the only yankee in this thread :laughing: