Smart quotes problem when changing computer

Same problem here… I hope you can fix this bug on Mac version of the software.
Thanks

Claudione

It’s not a bug. It’s the way that the Apple text engine accesses the upper range of UTF-8 glyphs compared with the way that the Windows text engine accesses them. And it’s not a trivial problem!

While I can understand that it irritates you in terms of smart quotes and maybe one or two other glyphs … for me and a collaborator it is a real problem as it makes using both platforms to work on a project which includes Chinese totally impossible. And the same is potentially true for Japanese, Arabic, Hebrew, and any other language that uses higher ranges of UTF-8.

I know the problem has been ticketed by Lit and Lat for working on, but it’s a really big issue.

Mr X

Actually, the smart quotes changing and the Chinese language issues are slightly different, though both involving the RTF encoding. In the case of the quotes, though, Windows and Mac typically encode them differently, and this was a slightly easier fix once the Windows team were able to do some more investigation into how and why the Mac encodes the specific quotes. I hope this will help too with the more complex problem of the Chinese rendering, but they’re still working on that. The quote problem, however, I believe they’ve fixed for the next update–it’s still in testing, but seems to be working reliably.

Oh that is music to my ears! Thanks for the update, MimeticMouton!

And good luck for the bigger problem.

This is not getting any better. I usually work on this project on PC, and this morning worked on it on my Mac and when coming back to Windows here is what it looks like:

All the ‘«’, ‘»’, ‘’’, ‘+’, ‘é’, ‘à’, etc. have been converted to characters like ‘Ö’, ‘í’, ‘î’, etc.

What is going on here? Is it the same problem or something else completely?

Don’t get me wrong, I love Scrivener, it really is one of my favorite software, but this cross-platform thing is not working and unfortunately I’ll have to stop using it until it is fixed. I can’t risk having my precious content being transformed like that. :frowning:

Thanks again for your help, L&L—I’ll stay tuned for a solution.

I kind of stopped using Scrivener on Mac because of this problem, so I have not downloaded any last updates. Should I? Has this been fixed?

Been nine months since my original post, and I would love to be able to use Scrivener on Mac again.

Thanks in advance. :slight_smile:

EDIT: Finally managed to update Scrivener on Mac after a couple unsuccessful tries: problem described in this post is still happening.

This is to do with the way Windows/Qt codes access to the upper parts of UTF-8 — which includes such things as guillemets — as against the way the Apple text engine does it. Both methods are legal. I know the guys are aware of the problem. Believe me, you have problems with guillemets and other punctuation; I use Chinese and the whole text is turned to gibberish as a result.

I have installed Scrivener for Windows under a version of WINE for when I collaborate with a Windows using colleague. It’s frustrating, but I don’t know how the team are going to get round it and it’s going to be a problem with the iOS version as well.

I know that doesn’t give you much comfort, but I hope you can see that this is a fundamental problem at the level of the systems, which is therefore not easy to find a solution for.

Mr X

This problem also shows up when moving cross-platform in LaTex. The underlying issue, as some of the posters have suggested, is the difference in file encoding. Have you made sure that your file is UTF-8 encoded on both the Mac and your Windows system?

The default encoding under the Mac is MacRoman. In Windows, it’s ASCII (I haven’t looked up this up in Windows and am going by memory–Arabic text file names on my Windows system display “???.???” while the name is properly displayed on the Mac). I am guessing that MacRoman is the default in Scrivener. I just looked at the Scrivener “Preferences” panel and don’t see an option for UTF-8 encoding.

So moving the file back and forth between the two systems can lead to mischief.

You can convert a MacRoman-encoded file to UTF-8 using the following command line program under MacOs:

iconv -f MACROMAN -t UTF-8 your-roman-encoded-file.txt > utf-8-encoded-file.txt

I do not know if this works with “.scriv” files and perhaps someone can answer this question, but it will work with text files.

Also, since you mentioned that you are working on a thesis, it may be that if you open your compiled draft to a typesetting program such as LaTex you may find that all the characters which do not display properly on the screen (under Windows or Mac) nevertheless typeset properly under LaTex.

Well, to be clear it’s not quite the same thing as plain-text encoding differences. It is true that RTF is, technically speaking, just a plain-text file in the same way that an HTML file is, and like an HTML file, the format has been around for a lot longer than UTF prevalence, meaning it has had to store extended characters using a portable method, again, much like HTML does. You don’t actually have a “©” character in RTF file, bare like you would in a plain-text file. Instead it’s some numeric code written in plain ASCII–8 level characters. On the Mac, it would look like 'a9.

Now that particular character may be addressed the same on Windows, I don’t know, but the problem is that there are differences in how the full UTF table is expressed using these RTF character codes. The two text engines use different addressing systems, which means that when you bring that RTF file over to a Mac or vice versa, the addresses point to the wrong characters, and you get gibberish. In a document that is all high UTF characters, like Chinese, you get 100% gibberish.

This is old information. Macs have not used the MacRoman encoding by default since OS 9. Mac OS 10.0 was, I think the first major operating system to support UTF–8 through and through. We’ve been using UTF–8 LF files for a long time now. That is the only format that Scrivener supports (and again, it doesn’t really matter for RTF since it doesn’t use any non-ASCII characters inside the coding itself). The only time that really matters is when you’re importing plain-text files.

I’m not as familiar with what Windows uses these days. I believe they switched to UTF–8 as well, but if not they are still using ISO–8859–1.

Everything text inside the Scrivener project format should be UTF–8 already. But a command like that wouldn’t work anyway. A project is actually a folder with many files, potentially thousands of files, inside of it.

So the root problem is that the Qt and Cocoa text engines do not write out characters using the same numbers, and changing how the text engine converts a keystroke for a character into a numerical address is something that is very difficult to control from a high level of programming. Hence, this is a tough one to fix.

I don’t want to hijack the thread, but I’ve noticed sometimes when bringing a compiled file from Scrivener into Latex some of the characters have gone missing, like ñ. Apostrophes are also a frequent problem. Texshop is a LaTex front end on the Mac. When I open the compiled file coming over from Scrivener, it often reports the file as having MacRoman encoding. There is a macro under Texshop which can be used to convert the encoding and when this macro is run the extended characters return. It’s good to know that Scrivener’s default is UTF-8 (which is the default set for Texshop as well). This may be solely a Texshop issue, but I thought it might be worthwhile for the original poster to try given that he is writing a thesis and that means, more likely than not, that he will have to get introduced to LaTex at some point.