ODF support in 10.5?

A website is reporting that Open Document (ODF) support is in the latest build of TextEdit for Leopard.

http://impulsivehighlighters.blogspot.com/2006/08/leopard-preview-textedit.html

And if it’s in TextEdit, it’s probably in the system itself, which may open up the potential for applications such as Scrivener to support ODF and named styles without a lot of custom programming. Import and export of more than simple text could get much easier.

There’s also a MacWorld article about what ODF means at:

http://www.macworld.com/news/2007/08/13/openxml/index.php

–Mike Perry, Untangling Tolkien

I doubt it will be that simple. I’m just taking a wild guess here, but RTF is probably still the basic rich text engine on the system (changing that would break a lot of applications, Scrivener included). Therefore, the basic rich text engine is still completely clueless when it comes to semantics. I’d only expect this to be something slightly better than their current Word importer/exporter, and then only because the specifications are open instead of locked up in a vault. The ODF exporter still isn’t going to “know” that a 24 pt, bolded line is supposed to be a chapter, newspaper headline, section, or book title.

Would be nice though.

Let me speak as an editor and publisher rather than a writer. RTF, particularly the truncated version implemented by OS X 10.4 and earlier is a bad idea, a very bad idea. It reflects a state of ignorance about how documents should be handled that was becoming discredited about a decade and a half ago. That it still exists today says something about the state of the software industry—too many kludges and patches when a major change like ODF is needed.

When a writer tells me, with eyes glowing, that they can supply a book in Word format, I groan. That typically means hours trying to root out stray fonts, particularly that virus of all fonts, Times Roman. I had one book only a few hours from going to press when a sharp-eyed proofreader spotted some stray Times Roman.

The problem is compounded by the fact that since version 5.1 (for the Mac), the user interface for applying named styles in Word is so dreadful it’s difficult to imagine how Microsoft could have done it more badly if they’d deliberately tried. (For a good if somewhat dated style UI, see FrameMaker.)

There is an answer. If writers would only used named paragraph and character styles and use them consistently, InDesign (with Quark, the typical software publishers use) is perfectly capable of ignoring all the unwanted specifics (font and font size) and importing only the text and named styling, applying the publisher’s style meanings to the result. In that respect, it is far ahead of the current OS X, which only understands primitive, named-style-free rtf. And that’s the direction writers tools such as Scrivener need to go. Publishers have their own house rules about fonts. They don’t want to have to cull out a writer’s odd choices about fonts before they apply their own. (And I realize that scriptwriting is different. There the product is a few dozen xeroxed scripts.)

To be honest, the real problem is the ignorance of a market driven by a Microsoft that gives people bad choices and, because they can see no other, creates consumers who simply don’t know how gosh-awful products such as Word are. Because named styles are so difficult to use in Word, few people use them and, as a result, few people learn how valuable they can be and demand that Word handle them better. I changed a 500-page single-column book into a far more reasonably priced 280-page, double-column book in about an hour because the book used named styles. Defining meaning rather than appearance is that powerful a tool. And it is particularly useful in an era like ours when documents need to move between applications or get repurposed for different media. And when fully implemented it means that you’ll be able to take up again a book that you abandoned twenty years before.

I’m not sure how well the 10.5 core will be at handling ODF. The problem with the Microsoft/Windows market is that so many of its users are as clueless as Microsoft. The problem with the Mac market is that it’s too often driven by fashion rather than good sense. (Notice how dated and ugly the colorful first-generation iMacs now look.) But Apple does know that pros also buy its products and that their more discerning tastes matter. That’s why fonts in OS X display more accurately in OS X than in Windows. OS X chooses to give up some legibility for accuracy. Windows chooses legibility (blurring the distinction between fonts) over accuracy. If they wanted, Apple could give us a very good implementation of ODF in the core OS. Whether they will or not is another question. As I mentioned, applications like InDesign have evolved to work around the limitations of Word and rtf. And with CS3, scripting in InDesign is so powerful that it can clean up almost any level of corruption. I’ve already developed a simple way to mark up Scrivener files that InDesign scripts can use to fully layout a document in a few seconds.

One final note. Keep in mind that the rtf solution, defining the appearance of a document by coding the output is very difficult. A lot of programmers I’ve talked with are afraid of ODF type changes because they think they must be be at least as hard to implement as rtf. Not so.

Setting aside all the complex rules of XML which have little bearing on ordinary books, defining a document semantically is no harder than basic HTML. It is even coded similarly. My scheme for turning Scrivener into a named style writer is no more complicated that putting «» around words that need to be italicized and «bt» at the start of paragraphs to be formatted as Body Text. That simple sort of information provides InDesign with all the information that it needs to format extremely complex documents, documents than can be changed in a flash. And that’s what basic marked text is. Defining the meaning of something is far easier than defining what it looks like in excruciating detail.

In the end, I suspect ODF or something like it will replace rtf as the interchange format of choice. But getting there may take time and at present is complicated by the fact that, having screwed up the document market with its proprietary format, Microsoft now wants its own in-house version of XML to define the future, creating the potential for two competing standards that merely confuse most users. There’s a good discussion of that at:

http://www.markshuttleworth.com/archives/132

–Mike Perry, Inkling Books, Seattle

I absolutely agree with everything you have to say. I have often wondered why Apple went with RTF in the first place, as it is a pitiful format for anything but one shot documents that do not need to go anywhere, and just need to look pretty a certain way. My (somewhat) educated guess would be that when OS X was first concieved and in initial development in late 1999, RTF was the most open and standard document format available (that was additionally easy to control from the user’s end). ODF wasn’t even conceived yet (and wouldn’t become an ISO standard until fairly recently). By the time it had gained ground, RTF as the basic rich text engine had become firmly entrenched in the OS. I’d love to hear that they completely abstracted it, and that Apple could merely replace RTF with ODF, letting 90% of the existing applications still function—perhaps that is the case—but I have a feeling it isn’t entirely that way, and even if it was, Apple’s stripped down implementation has meant that many developers that care to supply their users with more features than TextEdit, have had to get in to the guts of RTF to do so. Scrivener has numerous RTF hacks which allow it to be one of the better RTF engines on the platform (in my opinion). It reads more and exports more than even some word processors. NWP is probably the most noteable exception.

A switch from RTF to ODF would probably take a gradual change with a long development cycle, but it could be done. Who knows, maybe it has but nobody that really does know can talk about it.

For now, I continue to recommend the use of MultiMarkdown for anyone that needs true semantic output. It is true, it has its limitations and does not support excessively complex documents, but the syntax is beautifully simple and easy to learn. The results are a crisp, clean, and fully valid XHTML document which can be readily converted to many other formats. While none exists yet, an MMD to ODF converter would be a nice addition. In fact, there might even be a LaTeX to ODF conversion path that would be a possible route to take. It probably would not even require LaTeX to be installed on the system.

I hate to put a positive spin on this situation, but I find it amazing how comparatively successful Apple has been doing such a Microsoft-like thing–that is, relying on the fruits of its ancient R&D.

I think that Apple’s choice to use RTF is the consequence of NeXT’s earlier development efforts. For example, compare the OpenStep spec from 1994, page 89, with the current documentation on NSText (circa 2006).

levenez.com/NeXTSTEP/OpenStepSpec.pdf
developer.apple.com/documentatio … hText.html

My impression is that they basically just pulled much of this stuff into OS X… Apple has added several features to the text system in each version of OS X, but I too hope they get around to a more fundamental re-design soon. While they could just support more RTF control words (the spec supports styles, for example), maybe WebCore/WebKit will lead the way to an XML based solution? But more broadly, I think that the effectiveness of the interface (not the document format) will determine whether developers and users embrace the change or simply go on using the text system in the same old way to make life difficult for editors.

Bryce