Compile style moves question mark to beginning

[drmajorbob]

Here is a bit of text in the Editor; yellow highlighting marks the Babel character style. The first line ends with ?:

This is the style pane in Compile. It adds a star (𐫰) in the Prefix and Suffix:

And this is the result in a PDF. As you can see, the question mark moved to the beginning of the sentence:

That’s weird, right?

[Vincent_Vincent]

The period also.
. . . . . . . .

[November_Sierra]

The period moved, too (second line). Does this happen with all common punctuation marks or with every last character in general?

[drmajorbob]

Those are my only tests so far. I’ll get to trying others when I have more time. Thanks for noticing the period. I’ll create a bare-bones example too.

[drmajorbob]

I’ve tested a few punctuation marks, and I tested with a second manichaean symbol in the prefix/suffix, since 𐫰 is called manichaean punctuation star. Punctuation moves; other characters do not. Here’s a bare-bones test project and pdf output.

manichaean.zip (100.3 KB)

[November_Sierra]

Okay, some observations…

1. One of the Compiler styles was named “another emoji”, so it wouldn’t be affected by the (not) matching editor style. Fixing this results in both causing the same trouble:

2. Both symbols also cause problems in the Compiler settings when I try to edit the prefix / suffix input fields. Like placing the cursor behind the symbol and press ⌫. Won’t work. Marking the symbol and then deleting it works. This is already odd.

3. Choosing another symbol (with no “punctuation” in the name, don’t know if that matters) works just fine:

So something it definetily wrong with the “manichaean punctuation” symbols which derails the Compiler.

Wait, I found something! Manichaean script - Wikipedia
“Like most abjads, Manichaean is written from right to left”

[drmajorbob]

Yeah, I did the test properly and got your results here, but then I renamed the style and lost the thread.

@November_Sierra

with no “punctuation” in the name, don’t know if that matters

I suspect all the manichaean symbols will behave this way, but I haven’t the patience to test a lot of them. If you want to find them, search for manichaean in the Mac’s emoji & symbols viewer. This must be a character set intended for a right-to-left language, but my google searches didn’t confirm anything like that so far.

Oops, I skipped over your comment about the manichaean script. I’m always in a hurry, but never getting anywhere.

[November_Sierra]

Yup, same conclusion. RTL characters seem to confuse the Compiler. Could be a bug (maybe even in macOS) or by design. Short solution: Don’t use those.

[drmajorbob]

It may be markdown behavior.

[November_Sierra]

Hmmm. How does Scrivener create the PDF? Is there even Markdown involved (internally)? I compiled directly to PDF.

While I can’t explain why it happens in this weird way, it kind of makes sense (well…) in hindsight. If a character associated with an RTL language “triggers” a change of the writing direction, the end of the sentence would then have to be on the left side. And so its final sentence punctuation.

I’d bet even deleting the character in Compiles’s style settings works if you try it the other way around (placing the caret before it and then delete… or forward delete? ). But honestly. I’m lazy and very tired now.

[drmajorbob]

As I understand it, a basic part of the Scrivener 3 design involved converting rtf to markdown, and all other exports from the markdown.

[November_Sierra]

Hmmm. Most of it could be outside of Scrivener’s control (e.g. judging from the way the input fields reacted to this character, which would be on the OS level). Not just from a conversion point of view, but also later down the road (think: display hickups in eBook readers). Maybe @AmberV has more insight. I’d say: if it hasn’t to be exactly this symbol, choose a similar one that’s not RTL-related.

[drmajorbob]

I’ll use something else, of course. It was an experiment, with a symbol chosen at random; not a big deal if I can’t actually use it. It was a startling behavior, though!

[AmberV]

I haven’t done too much looking into this, but I think the clue that November_Sierra uncovered about this being an RTL glyph could explain oddities. Putting RTL characters around LTR text, and potentially putting the software into a position of having to decide whether a “prefix” should go on the left or right of the selection, added with potential confusion over the illogical (from an RTL standpoint) punctuation placement, is surely causing issues—but given how much of an edge-case-of-one this surely is, finding another glyph is probably the best route forward.

As I understand it, a basic part of the Scrivener 3 design involved converting rtf to markdown, and all other exports from the markdown.

It wouldn’t have anything to do with the macOS PDF printer, if that’s what you meant—you can get much the same by copying some text into TextEdit and using File/Print and saving as PDF. Markdown on the other hand is used internally to produce components of the HTML for ebooks, since macOS itself has such an ancient HTML converter that the specification it produces is something people were still using Netscape to load, when the spec was still new. It has the nice side-effect of producing way cleaner HTML than the text engine’s RTF → HTML generate ever could, though, so all around it’s a good choice.

Scrivener has extensive Markdown integration, but only uses it directly in its non-MD compile formats for that one task.