No support for Pages?(!!)

I’ve been methodically working my way through the Scrivener tutorials in anticipation of using it as the platform for writing my dissertation, and–fully unexpectedly–I was shocked to learn there is apparently no support for iWork’s Pages. I have several thousand files that I’ve collected for my research, including several hundred Pages files. I encountered this surprising fact when I first began importing my files into Scrivener and then received the import error.

Scrivener seemed like such a uniquely Macintosh piece of software, so well-designed and Apple-like, that I at first couldn’t believe there wasn’t iWork support. But from what I can glean from the Scrivener support site, this indeed seems to be the case. What a shame!

What’s doubly surprising is that all the other research writing packages I’ve been evaluating–DevonThink, Journler, et al–all seem to take Pages support for granted, as any Mac user should, so I am baffled to explain this.

In any case, as much as I think that Scrivener is in many other ways the better platform it’s back to DevonThink, since it would be impossible to imagine working with a program without Pages support!

This will be slightly improved in the next version, with blanket (non-editable) support for all file types. This would be roughly similar to what these other softwares provide, in terms of Pages support. You can place them into your Project/Database/Journal/whatever and “activate” them somehow to open them in Pages.

The main reason why this is so is, frankly, Apple’s fault. They have done extremely little in terms of making Pages “friendly” with the rest of the Mac world. I think DEVONthink can display Pages files via QuickLook hacking (even that isn’t officially supported by Apple). In fact, it doesn’t even really have a good RTF importer. Out of all the Word Processors for the Mac, it is probably the least integrated with the rest of the world (Word aside), and for this reason many have just stuck with Word or gone with other vendors like Nisus or Mellel. Another problem with Pages, from what I’ve heard, is that it isn’t the best platform for final processing of very large files, but I’ll freely admit that is anecdotal. I’ve just seen a number of threads where people have tried to fit Pages into an academic or book-level environment and ended up regretfully going back to Word or some other software. A lot of writers in academia like Mellel for its ability to handle huge documents with ease, strong bibliography support, and multiple note streams.

Which isn’t to say I think you should “switch”; just providing a little background information for as to why support for Pages is thin. If you do wish to stick with Scrivener (most do not find DEVONthink an environment conducive to the actual writing process itself, and I don’t believe it even has any kind of built-in support for citations), you might consider exporting your Pages documents as RTF or Word doc and importing them that way.

A lot of this is of course subjective, “your mileage may vary” indeed. There are also “the brave” who prefer LyX, which produces high quality documents and has a very large community of support for academic use. Unfortunately it is not at all “Mac like”, and requires working with a typographic sub-system that is a shade more advanced than products like Word and Pages.

Each has its strengths. If you’re already using Devonthink to compile research, I wouldn’t even consider switching to Scrivener for that – Devonthink has some specialised approaches to research that Scrivener doesn’t really attempt to get into. However, Scrivener utterly outshines Devonthink as a writing tool.

I’m a recovering academic. Meaning the books I now write have no footnotes, just a list of sources at the end. I use Devonthink Pro, Scrivener, and Pages, in that order. Absolutely no trouble in the export and import process, down the line. Pages handled a 300-page manuscript, with no problems. Editors used Word and Track Changes, no problems. So, take it from this user, Pages is OK. BUT remember, no footnotes here and no advisor-from-hell, either. (I did buy Nisus Writer Pro and will use it on the next book, so stay tuned.)

Druid,

Can you describe your workflow, either here or in a new thread? Although much (OK, most) of my writing is academic (until I admit I have a problem, I have no problem!) I am curious to see how you use DTP, Scrivener and Pages together. And how you manage to get Pages and Word to talk nicely together.

If you have already explained, any chance of posting a link?

Thanks.

This would be a fine solution, there’s no need to have to reinvent the wheel to work with Pages docs natively within Scrivener. The important thing is to be able to organize these Pages files within the Scrivener research and writing environment, and to keep them an integrated part of one’s ongoing research (just as is the case with DevonThink and other apps, as you mention). If you could create some way to integrate Pages docs as simple objects within Scrivener–even uneditable–and to allow the user to associate Inspector information to it (Synopses, Notes, etc), that would be great. If one needs to work with the document, just double click on it (or some button) and it would open in Pages. Being able to do this would be a huge benefit for Pages users.

From what I understand, such functionality would not be available until Scrivener 2.0, correct? And it will be some time before this version is released–next year some time?

[i]

[/i]

I’ve come to really like Pages a lot; after many years of Word I find working in Pages a refreshing change in its elegant simplicity and reliable consistency. I find it an incomparably more conducive environment for writing that includes structured formatting and other media (images, video, audio, etc). I still find it remarkable how easy it is to create sophisticated, desktop-publishing-like documents with little effort.

I must admit that I haven’t encountered problems with sharing docs since I switched to Pages a couple of years ago. Most of the time I save to PDF when sharing docs, but have encountered only minor problems when having to save as a Word file or RTF.

[i]

[/i]

I’m admittedly a trial user, but so far for me DEVONthink appears to be a document organizer/database and not much more. By using a slick interface with powerful cross-linking capabilities, it makes your documents more accessible than Leopard’s Finder doc system. I can appreciate DT’s powerful organizing capabilities, but to be frank I consider its $150 price tag far out of proportion to the gains over Leopard’s native file system. I now have several thousand research files (some 40 GB worth), carefully organized into folders and many levels of subfolders, and this has worked adequately thus far, though I am looking to take it to the “next level” with a DT-approach to file organization.

[i]

[/i]

Ideally I would like to see the functionality of DTP and Scrivener integrated into one app or working environment. Right now it would appear that you can use DTP for file organization, and Scrivener for writing, and the two exist in sort of parallel, balkanized universes. But this is an artificial distinction, since there’s a lot of things you do with those research files as you carry out your own writing. For example, I really, really like the Inspector functionality in Scrivener, and I would like to use it for annotating and better organizing (with metadata, etc) my research files, and not just for my writing. I would also like to associate or link those files to my own writing in Scrivener.

Having separate universes for file organization (DEVONthink) and research writing (Scrivener) is only a moderate improvement over the Leopard Finder/Pages (or Word), but to better integrate file organization and research writing would be taking it to the next level, and a major one at that. My sense is that the DEVONthink and Scrivener folks are skirting around with a paradigm-shifting breakthrough in how we structure and support the personal research process. The real challenge is to better integrate research and writing.

To address your first complaint: Import and export is a two-way thing. Pages support is the bane of my life, as I get frequent e-mails from users - some inexplicably angry - asking why Scrivener doesn’t support Pages files, as though I am being somehow ignorant or deliberately obtuse, or going out of my way to make things difficult, or somehow living a lie by producing “Mac software” that doesn’t support Apple’s own software(!). But this all presupposes that the sole responsibility for providing import and export capabilities for a given format lies with the software that will do the importing and exporting. If you think about this for more than a minute, though, you soon realise that this is not the case. File formats are not simple things - try looking at the contents of a .pages file yourself (change the extension to .zip, unzip the contents, and open the dozens of constituent files in a text editor). Now ask yourself: would this be trivial for someone to interpret, even someone with programming experience? Hardly - all of the internal codes for any given file format are decided by the creators of that file format. If you create a file format and someone wants to read that file format, you need to tell them what the various parts of the format mean.

In other words, it’s a two-way thing. For Scrivener to be able to read a Pages file, Apple would need to have published the file format - but they have done no such thing. Apple released some of the specifications for the Pages 2.0 format, with this caveat:

And:

(And indeed, the current .pages format is different - the .pages format was package-based back when that was written; now it’s zipped.)

In other words, Apple make it quite clear that they do not intend to make the Pages file format public; they have no intention of making it possible for developers of other software to support their file format. (Contrast this with Final Draft. I asked them if I could support their file format and they provided me with specifications of the Final Draft format, so that I was able to write my own importers and exporters.)

Rather oddly, Apple makes it incredibly easy for developers to support rival software - they provide developers with importers and exporters to .doc, .docx, .rtf, .html, .odt and so on - but they provide no such importer or exporter for the .pages format.

In other words, I would love to support the Pages format. I really want to support it. But Apple do not make it possible.

Moreover, your comparison with DevonThink and Journler is misleading and not comparable. Neither program imports and translates the file. Importing a Pages file into Journler merely shows the Pages icon; to view the file itself, you need to launch Pages from Journler (you will be able to do this with files that cannot be opened directly inside Scrivener in Scrivener 2.0; in 2.0 you will be able to import any file at all, and those that cannot be opened inside Scrivener’s editor will be opened externally). DevonThink does much the same, except instead of showing you the Pages icon, it shows the view you would see in QuickLook. The DT guys have hacked QuickLook to do this, though - Apple do not provide developers with any way of using the QuickLook technology in their own apps, so the only way to do it is by reverse-engineering the QuickLook APIs. This can be tricky and is prone to break in certain situations - I tried it in Scrivener but because of the different window set up in Scrivener - which, with the split view would mean that you could potentially have a QuickLook view open in two panes - it won’t work. QuickLook isn’t set up to work in two panes. Besides, while an un-editable QuickLook view of your Pages documents is fine for DevonThink - which is based around storing research - this would hardly be acceptable in Scrivener. In Scrivener, users would expect to be able to import Pages files and have them become editable text inside Scrivener, as with other word processing documents (.rtf, .doc) etc. But as I have explained already, because Apple will not make the .pages format public, this is not possible.

I thinks this belittles the achievements of both DevonThink and Scrivener, actually. Scrivener has been described as a “paradigm shift” by others. Scrivener is still the only application out there (though I daresay not for long), to the best of my knowledge, that allows you to view research documents - PDF files, images, or even other text documents - right alongside your text in the same window, in different panes. It also introduced integrating outlining with the writing process itself, via document synopses. If you find this only a “moderate” improvement over Finder and Pages, then I would suggest you stick with that. For me, though, I couldn’t disagree more. Having multiple documents in the Finder represented only by their names (no synopses) and having to launch them in the various programs just to see what they contain, then shifting windows around to have them side by side - that was extremely annoying for me. With Scrivener, all that frustration is gone.

Finally, no software can do everything. That way madness lies. I set out to make Scrivener the tool I needed - something that would help me write the first draft of a long document by providing access to different documents alongside one another, allowing me to see research documents at the same time, and to integrate outlining with the writing process itself. It did not set out to replace dedicated research management software or page layout programs. (Scrivener is project-orientated - you create a Scrivener project for each writing project - whereas DevonThink, as befits research software, is database oriented; they have different structures because they are for different things; the similarities are only superficial.)

The “Scrivener folks”, incidentally, comprise myself and one other person - and I do all of the design, development and coding. We are not exactly a big company.

Best regards,
Keith
(Scrivener developer)

How about you leap to that next level:

http://www.ironicsoftware.com/leap/index.html

The idea behind Leap is that your whole harddisk already is your storage place and you don’t need no extra data base or storage app folder where you put your files.

Leap offers ways to classify and tag those files that go beyond what the Finder does without forcing you to change any of your folder structure.

Personally I found it a little pricey although it is way cheaper than DevonThink. And don’t have so much stuff to classify and I needed more of a notetaking app for my own texts. I found the best solution for me was Together which is similar to Journler and interacts very well with Scrivener.

But with that amount of reference material you mentioned I would definitively give Leap a shot.

Others started a thread on workflow today, so I posted there: Work flow questions - #3 by howarth

As for getting Word and Pages files to interact, my experience is limited to these instances:

Sending Pages files to students who mostly use Word: I export Pages to pdf.
(Preserves the exact look of font and images; web links remain live)

Reading files from students who only use Word: Pages will open doc and docx files.
(I write my comments in a Scrivener file; copy/paste them into e-mail)

Sending Pages files to editors who only use Word: I export Pages to doc.
(Track Changes works on both sides; had no formatting issues; don’t use footnotes)

Sending Pages files to Scrivener: I export Pages to rtf.
(No particular issues, but then I use mostly plain, bold, or italic text, and no footnotes)

Keith: first, don’t take constructive criticism regarding Scrivener personally, I’m aware of what goes into a sophisticated program like Scrivener. If you are familiar with methods like Continuous Quality Improvement (CQI), then you should know that even the maker of the iconic iPod is never satisfied and continually strives to make it even better. I picked up on the Pages sensitivity already from Amber’s post, and your repeating her points make it clear that you’ve spent a fair amount of time dealing with it. My concern is that in so doing, you’ve missed the most important points I made in my postings, points that you didn’t mention at all in your response. Rather than my belittling DT and Scrivener, for example, you elsewhere in your posting seem to agree with my main point, that integration of functionality–in this case file organization (research) and idea creation (writing)–is a desirable goal in a computer-based research tool.

Another important point I made is that Pages support does not mean being able to edit a Pages document in its native format within Scrivener. Support for me means that Scrivener be able to at least have a pointer to the Pages document within its file organization system (integration!), and clicking on it would simply open it in Pages. I disagree with you that most of Scrivener’s users would want or need to be able to edit the Pages document within Scrivener, just having it in the Research part of Scrivener would be a big plus. Converting the Pages document into the Scrivener native format for editing (via RTF perhaps) would be a nice addition, but not necessary.

I also disagree that I was belittling DT and Scrivener when comparing with Leopard Finder/Pages: a versed Leopard user can go far in creating a sophisticated file organization system, and having multiple documents open simultaneously in Pages or other programs is certainly possible. Both DT and Scrivener add more to the equation, but I chose my words carefully, and a “moderate” improvement when comparing Leopard Finder with the current versions of DT or Scrivener is, in my view, a fair assessment and most certainly not a criticism of DT or Scrivener. A propos, I purchased Scrivener almost two years ago, wishing to support software development that I find promising, so with your program it is more a question of using it and not of purchasing it (as is the case with DT).

Sticking with the iPod analogy, I would say that Scrivener is much like the early MP3 players a decade ago that betrayed all the earmarks of a paradigm shift in music, with the iPod beckoning on the horizon. From my use of Scrivener, DT, Journler, and the others I believe we can recognize the contours of that iPod for research and writing, but–and I say this constructively in the spirit of CQI–we’re still waiting for someone to bring it all together like the iPod did. Scrivener is really really close, and having some form of native support (in the Research organizer) for Apple files would be a big step.

Hi,

I’m not taking anything personally, I’m just pointing out what I consider to be slightly unreasonable expectations considering Scrivener’s scope and brief, that’s all.

Well, obviously this is true for me (or any other developer), too, or else I wouldn’t be working on a version 2.0, would I? :slight_smile:

I think it’s you who seems to have missed my point, given that I did answer this in my reply:

So I already made it clear that in Scrivener 2.0, you will be able to import any file type, and have it open in its default application, just as in Journler. To reiterate: in 2.0, file types that are supported for viewing in Scrivener will be viewed in Scrivener; file types that are not will appear as icons that can be opened in the relevant app. If this isn’t what you are asking for, perhaps you could explain a little more clearly.

And you must accept that I sincerely disagree that this is a “fair assessment”. If Scrivener was only a “moderate” improvement over Finder + a word processor (I personally wouldn’t choose Pages because of its poor compatibility with anything other than Word), then I most definitely wouldn’t have spent years of my life working on it.

I generally recommend users not to buy Scrivener based on their own interpretations of what they hope it to become. From our About page:

Purchasing Scrivener (or any software, for that matter) based on a subjective judgement of “promise” is bound to lead to disappointment, as although I am committed to making Scrivener the best it can be - and 2.0 is a massive improvement and refinement - the “best it can be” depends on my own subjective viewpoint. There are many different types of users who want Scrivener to turn in this or that direction (full-blown scriptwriting app, timeline or mindmapping, layout application, blogging additions - to name just the first few that come to mind), but as I said before, no one program can fit everyone’s needs.

I’ve already said that more files are supported with 2.0. But I think you may have a different kind of app in mind - one that works the way you want. Scrivener (especially 2.0) already integrates all of the workflows I always wanted, and that, for me, is what it was about. Many users find that it does the same for them, and I find that very gratifying. But ultimately I have to develop Scrivener the way I believe in, not by trying to follow some Holy Grail of someone else’s vision, as I am sure you can appreciate.

All the best,
Keith

Yes, it’s just a database. But unlike the Finder, Devonthink automatically maintains a database of how your documents relate. Any time you’re in a Devonthink document, you can open up a “see also” pane to see what documents contain similar word patterns overall. The Finder can’t do that. You can also have Devonthink give you suggestions on where to file a document based on your custom filing system, since it will compare the document’s contents with the contents of what you’ve already filed in different folders. The Finder can’t do that, either.

Suavito: vielen herzlichen Dank für den Tipp über Leap! I’ve downloaded and am now testing it. I like the interface, and it has some nice features. As you noted, it’s nice that it doesn’t build its database by copying in all of the full-text files. Because my DevonThink database had grown to 40GB, I had to move it to an external drive. I’ll spend the next few days exploring its features. DevonThink is growing on me, and I still need to get a better handle on its cross-linking capability. I’d have to say that Scrivener’s ability to attach notes, a Synopsis, and other data to a file is a “killer” feature that the others don’t have, and will make it difficult for the others to compete.

I checked, and once again I’d have to disagree with Keith in his belief that the DevonThink folks “reverse-engineered” Apple’s QuickLook to get Pages functionality in DT, it’s a lot more than that. Though it doesn’t appear one can edit Pages docs in DT, one can certainly open a Pages doc and read it, which is the most important thing for me. I’ve run across a number of other programs that have Pages support, so I’m not sure what Keith is referring to.

I think I might have been the one that said that, but Keith might have stated it as well. Either way, DEVONtechnologies themselves claim this, and that is where I got the information from:

Note it relies upon QuickLook for display of proprietary file formats, and falls back to Spotlight when there is no QL support. And they are definitely “hacking” or “reverse-engineering” it because Apple has never published QL access to developers. This sort of support means they’ll have to constantly be changing their code to adjust to whatever Apple changes internally on a whim (and Apple does that a lot, even for things they shouldn’t with public documented APIs).

Because a feature doesn’t look like QuickLook (a zoomy grey bezel with a small preview of the document in it), it doesn’t mean that it’s not actually QuickLook. QuickLook lets you open a file and look at all its pages without needing to open the creator application. Isn’t this what DevonThink is doing – providing read-only previews of Pages files?

Well, you can disagree, but you’d be downright wrong. :slight_smile: This isn’t just supposition on my part, it’s an unequivocal fact - I have read up on reverse-engineering QuickLook and tried it for myself; I immediately recognised the QuickLook view of Pages files in DevonThink; and if you still want to disagree, then I’d recommend asking the DT guys, as I’m in touch with them and have spoken with them about this, so one of their main developers has confirmed that this is exactly what they are doing.

Just open a Pages file in DT; then in Finder hit the space bar with a Pages file selected - note how the view of the Pages file inside the QuickLook HUD is identical to the one in Pages. Also note that when saving a Pages file you have an option to save the preview (for QuickLook) inside the file or not - if you do save it in the file, it gets saved in the .pages file as a PDF; if you don’t, the Pages QuickLook plugin generates an HTML representation for QuickLook. Note how Pages files with internal previews get represented in DT as PDF files, and Pages files that don’t get represented as HTML. Hmm. That’s not a coincidence.

Now, as I said, this involves some hacking, and the very talented DT guys have got away with it, but it’s not a public API and thus not supported by Apple and, again as I’ve already stated, it does not work when you have two panes as in Scrivener (because QuickLook isn’t designed for that and is hard-coded for its intended use). So, I would recommend that you check your facts on what QuickLook does and on how DevonThink does things (the same in this case), then refer back to my previous reply for reasons why this won’t work for Scrivener. Hopefully then you will “know what I was referring to”.

Please name them. So far you only named DT and Journler so far and I have already explained how both those do things. Believe it or not, I do research these things, and have so far found no program that can properly import Pages files, so if you have found one, I’d love to see it.

I also note that this time it’s you who has ignored most of my reply, which addressed your main concern, explaining that 2.0 will be able to import any file type. You gave the impression that this was what you were after, so without further clarification I will assume that my previous reply answered your query and that this one clarifies your error about how DT works.

Amber, Keith, having worked fairly extensively with XML DTD’s, I’m aware developing for them can be tedious and time-consuming, and I’m certainly not faulting you for not taking the time to delve into this realm to provide more extensive support for Pages than is necessary. I’m sure you’re aware that Apple has a fairly extensive area on its Developer site for Pages and its XML schema, and I know from experience that Apple is very supportive of third-party software interfacing with its iWork apps.

Whatever you want to call the DT programming for Pages support, the folks at DT have done a very nice job of building in the necessary support for Pages in DT, all that I believe most DT users want or need: database integration and a simple integrated viewer. A number of products that I use offer similar levels of support, including the mindmapping packages I have used (MindManager, Novamind, Brain, etc etc). It sounds to me like you’re getting frustrated by attempting things that are not necessary.

I think a big part of the problem–and the need for this exchange–is that you are tripping up and misinterpreting what I’ve been saying by your ingrained preconceived notions. When you, for example, say:

“I generally recommend users not to buy Scrivener based on their own interpretations of what they hope it to become.”

This is not all what I said:

“I purchased Scrivener almost two years ago, wishing to support software development that I find promising…”

With commerce in free societies, anything you purchase is an affirmation of your support for a product. Even though it’s more expensive, I make the conscious decision to buy organic produce to support a particular economic model. More than other markets, software has developed a model that promotes the ongoing support of software development. I have a number of software packages that I’ve owned for years now, and have regularly purchased the updates to support ongoing development of the ideas represented by a software package. Just as with organic produce, I want to voice my support, financially, for particular software “communities”.

Finally, your believing that you are developing a product only for your needs is really only a part of the bigger picture. If this were truly the case, and you were doing this only for yourself, you would not have the Literatureandlatte Website to actively foster a Scrivener community, and to actively solicit feedback. Your software is, as is most software, a communal project in which you, as “lead” software developer, are making “executive” decisions as to what to include and what not to include in updates, as well as to act as guardian of this process. To take credit for all the feedback and effort we, as customers, put in to offer suggestions and use your software, is, frankly, pretty self-centered. Rather than emphasizing the “you” in this process, you would help foster the Scrivener community–and your business–much more by emphasizing the “we”.

Apple openly states that it designs its products as devices they would really want to have and use. This is a lot different from claiming that the MacBook is for their own employees, and if others “happen” to like it, then that’s fine. From the beginning Apple has always been a company that sees itself as part of a much larger community, and this is why Steve Jobs always uses the “we” when talking about product development, and why their customer service and support is exemplary. Apple is like a huge sponge, soaking up input from all sectors in society, and translating needs into electronic devices. And just as you are quick to point out, there’s no question that Apple can disappoint because it cannot be all things to all people. Just as with you, however, it has to stay focused and it has to protect its proprietary business assets (knowledge-based and otherwise). Few consumers would question this need, and those that do should be ignored.

Some folks just “don’t get it”.

Consider the prevailing sentiment on these forums and then reread your post. Slowly. You shouldn’t be surprised if you find yourself on the receiving end of a few sharp comments.