I know it means it can’t read the punctuation for some reason. How can I prevent this happening when exporting markdown please? Thanks for your help!
Sounds like an encoding problem (e.g. “UTF-8” vs. “something else”).
I believe so. I have set encoding to UTF-8 though. Now I am on iPad I’m not sure I can change encoding until heatwave subsides and I can back on desktop, hopefully Thursday.
According to the Textastic app on my iPad the text is already UTF-8 so I don’t know what the issue is.
I think I’ve had this issue with LaTeX, with curly quotes, em and en-dashes, etc… I solved that by setting up compile-time replacements for the problem characters, since I didn’t want to bother with constantly re-checking the text in Scrivener.
1 Like
Is it possible to share what replacements you used please?
It would help if you copied and pasted the character into a character map type program to search for it, and see what it actually is. All of this about punctuation may be off-base, and the real problem is an iOS bug inserting null characters (which happens, Apple’s editor kit is filthy about inserting junk characters, I often have to clean out source text), or maybe you have a page break separator setting, which inserts the non-printable u+000c, I believe. Some text editors even show you the control code if it can’t print it, instead of an icon.
@StaceyUK: Is it possible to share what replacements you used please?
Scrivener has a dedicated setting for converting most typographic punctuation to ASCII-compatible equivalents in the Transformations compile format pane. This setting is enabled by default in all of the stock Markdown-based compile Formats, simply to cut down on support issues, and in most cases Pandoc/MMD smart-quote algorithms are better than Apple/Win anyway.
So I would definitely recommend trying that option, if you aren’t already, before setting a bunch of replacements.
Replacements are cannot be done on iOS, but “dumb” quotes can be, by adding the following to the Scomp file:
Transformations:
- Dumb Punctuation
@Rdale: I think I’ve had this issue with LaTeX, with curly quotes, em and en-dashes, etc… I solved that by setting up compile-time replacements for the problem characters, since I didn’t want to bother with constantly re-checking the text in Scrivener.
With LaTeX the problem is often that popular and default typsetting engines for it are a bit old-fashioned, and like old HTML browsers, have no built-in awareness of Unicode. If you use special characters (like typographic punctuation), then you need to include special packages that add support for them—or switch to a more modern engine like XeLaTeX or LuaTeX. But that’s the sort of problem you’re going to see in the output, not the text editor (unless the text editor itself can’t read UTF-8).
I don’t have a handy list of such characters, but after you try the “Dumb Punctuation” option, what I’d do is just identify in Scrivener what the problem characters are on the output by searching for nearby phrases. Then copy that character from Scrivener’s editor into the Replacements tab, and put a plain-text equivalent for it in the “replace with” field.
I’m confused by this comment–Isn’t the issue that the output markdown file contains badly encoded characters?
I was referring specifically to LaTeX typesetting issues you mentioned, where if you put a smart quote into a UTF-8 file and try to typeset it with pdflatex, without including the necessary packages for handling those characters, then you would get an error glyph icon in the output. It would still look fine in the text editor that opens the .tex file though.
It is, as I understand it, all a digression from the original question.