Greek and .txt

I’ve been trying to find a way to create more ‘durable’ text files (as in plain text rather than rtf). I thought Scrivener’s ‘Export as plain text’ would be the way, but the trick is that I need to include polytonic Greek in these documents.

When I export as plain text from Scrivener, I get a .txt file that—when opened in Bean or TextWrangler—proves to preserve the Greek. (Whether or not it’s really a .txt file I wonder.)

But, when then imported into Scrivener, the Greek is all shot to hell.

I’m a novice at such things and am just puzzled. I guess my question is:

Is it possible to create a plain text file (.txt) with polytonic Greek? Scrivener’s export seems to suggest that it is. But its import suggests otherwise.

I’m sure it’s simple to you wizards, so maybe you could ‘splain it to me?

What is probably happening is that your exported file needed UTF-16, and Scrivener is only capable of understanding UTF-8 and ASCII (which comes along for free with UTF-8) during import. UTF-16 is a perfectly valid format, and as you have noticed, it does work in applications that support it. It is probably reasonable to presume that as time goes by, its adoption will increase (it’s already the default encoding in the Windows world), especially since UTF-16 adoption is necessary for some languages to become commonly transmitted. It’s just that, if this is indeed what is going on, for a while your exports will not be Scrivener compatible.

If you needed to, you could probably get the data back in via copy and paste from Bean.

And yes, these are just ordinary text files. Modern text files can have a massive character set to them, allowing for alphabets from many complicated languages. Before Unicode, you had to do this with hundreds of different encoding sets.

Thanks, Amber.

I’m primarily concerned about the integrity of the .txt file. If Scrivener has a problem for the moment, that’s ok. As you say I can copy/paste if need be.
My idea, though, is to use the .txt files as an archive for safekeeping. I’ve been reading much of what you and others have had to say about MMD, but it seems just too much for me. I need footnotes, italics and Greek. And I don’t care so much about portability as I do about not losing the text. (I’ve lost both Greek and footnotes in the past, with the conversion to OSX.)

So, if I understand, using Scrivener to export to .txt is a fine way to go for my purposes.

Thanks.

I’m confused, how are you getting italics and footnotes in your text files? I know of ways of indicating where italics and footnotes should go in things like MMD, but if your stuff in Scrivener has embedded styles like italics, isn’t that going to get lost in the text export?

Text is great for archival, that’s what I use too (MMD is just stuff inside of a text file to make it be something more than a text file if need be). Technically RTF is just text as well (try opening one in TextWranger), but you are right in that a lack of feature parity can mean lost information—especially when information is conveyed strictly through font appearances alone.

Whatever the case, yes. UTF-16 is safe to archive to.

No doubt I’m confused (too).
I’m thinking: In Scrivener use Text>Convert>Bold and Italics to MMD syntax. Then export as plain text.
This at least tells me what should be where.
Of course quick formatting into a nice document is not as straightforward I’m sure as it would be if I could figure out MMD. But at least all the info is there. I suspect that a search/replace could rectify italics. Footnotes I’d probably have to reenter manually.
RIght now this seems easier than trying to understand MMD. Maybe I’m wrong about that…

Oh okay, sure that would work to preserve italics and bold. MMD footnotes look nothing more complicated[^fn1], than that which is probably pretty close to what you’ve already got. You can use Scrivener’s Footnote editor to make these automatically. I never even bother with typing the number and reference in myself. I just let Scrivener handle all of that and export as “Multimarkdown” (not any of the conversion modes).

In other words, you don’t have to use the MMD conversion stuff to get some benefit out of the format, since its “syntax” basically just looks like what most do with text otherwise.

Of course, use whatever you like; it just sounds like you are already basically using MMD, to me. At least the authoring end of it, which all that matters for archival.

[^fn]: Footnote data.

Ok.
I find that exporting as MMD from Scrivener doesn’t preserve Greek.
So plain text seems better.
Maybe some day I’ll figure out what LaTex is and all the rest and how to turn those plain text files into other things.
Thanks for your help.

Ah, Scrivener must be forcing UTF-8 for that export method. Yup, in that case plain-text is what you want. :slight_smile:

hmm… my conclusion is that the import for UTF-8 encoded text files in Scrivener is broken (the export is perfectly ok). Polytonic greek requires UTF-8 not UTF-16, so this can’t really be a problem. I’ve added a couple of german Umlauts (i.e. non-ASCII-characters) to a passage from Plato’s Crito and the Umlauts are the only characters that get properly imported, while the greek text turns into rubbish.

Bild 3.png
Bild 2.png

That doesn’t make any sense. The Umlauts use the same two byte encoding as the greek characters. :confused:

Thanks for this input.

In any case, it seems that something is amiss with Scrivener.

Perhaps this thread is in the wrong section of the forum?

Amber, if you’re still there, do you think this should be brought to Keith’s attention?
(Surely he can’t read all forum postings!)

Thanks for investigating this further. I didn’t have any source material to work with so couldn’t determine if it was indeed a UTF-16 problem. Since it is a problem with UTF-8, I’ve moved this thread to the bug report forum. It is very likely an Apple problem, but no harm in illuminating the issue. Have you tried loading the files in TextEdit, incidentally? That would help determine whether or not Apple’s text engine is to blame.

In my case at least, files with polytonic Greek exported by Scrivener as plain text show up fine in TextEdit, Textwrangler and in Quicklook.

Same situation here, so the text engine is probably not to blame. Here’s a test file in utf-8 encoding.

Hi, seeing as this thread suddenly turned up in the Bug Hunt forum with 10 or so posts, and given that it’s the weekend, could someone please summarise the problem in a few lines to save me from reading through the whole thread on my day off? (Not that I have days off.)
Thanks!
All the best,
Keith

Hi Keith,

thanks for investigating this. To reproduce the bug, download this test file and drag it into Scrivener. It’s a plain text file (Unicode UTF-8 no BOM) that contains some polytonic greek characters, some CJK characters and a bunch of german umlauts. Only the umlauts will appear correctly in Scrivener. The file was originally exported from Scrivener in UTF-8 encoding, so it should be possible to reimport the file.

This is what I see in Scrivener after importing and changing the font to Monaco:

Is this not what I’m supposed to see? Note that when I first imported it I couldn’t see all the characters because my default font (Optima), which gets applied to plain text imports, didn’t support all the characters. But after changing the font to one that supports more characters, it looks fine. Or am I missing something?

All the best,
Keith

You’re right, the file gets properly imported. The problem is, that my default font is already set to Monaco, which explains why I fell for this in the first place. There’s something wrong with the text engine here. I have to convert to Default Text Style to reset the style or reapply Monaco in the font dialog to make the greek characters visible, which is a bit of a nuisance. :frowning:

EDIT: I’m not sure if this is a good idea, but perhaps you can (re-)apply the “Convert to Default Text Style” function automatically to all imported plain text files (that would include MMD)? The text engine normally has a fall back mechanism to display characters that are not included in the default font (the greek in our screenshots is actually Helvetica, not Monaco). For some reason this mechanism doesn’t get triggered in Scrivener, yet.

This is good to know. Of course none of us thought of the font. My default is Verdana which can’t handle the Greek, but switching fonts shows that everything is preserved.

Thanks.