First of all, let me join the many minions in saying how much I adore Scrivener. In fact, it has revitalized my writhing life!
However, I have to say that I am more than a little apprehensive about upgrading to Mac OS X Lion. Scrivener’s auto-save after {user selectable} seconds is perfect in that it causes no lag in writing. Also, I can automatically create back-ups onto my thumb drive requiring no thought or effort on my part. Perfect!
Now what worries me is that, if Lion is creating all these mirror images, there will be times, like in Microsoft Word, when what I see on screen is behind what I am typing-- this completely pulls me out of my writing trance and can destroy my train of thought. It is downright annoying.
Also, I hope that Keith doesn’t decide that Lion’s autosave is good enough. Please, Keith, don’t take away Automatic Backups. I like to have my thumb drive with me so that if someone steals my computer or my hard drive crashes, all my Scriv docs are safe and sound. Oh, and Time Machine doesn’t cut it (I don’t want to carry around a hard drive).
Call me paranoid, but I keep all my Scriv docs 256 bit encrypted. I use an IronKey as my flash drive (waterproof and hardware encrypted) and I use a dmg on my Mac. Hey, I have spent many hours writing my ideas; why would I take a chance on somebody else stealing them?
So, Keith, I await your assurance that my planet is not about to implode sometime in July.
And, again, thank you so much for inventing the greatest writing software of all time!
David
Actually, for the time being at least, Scrivener is continuing to use its own autosave on Lion (developers opt-in to the new auto-save features), and backups work as usual too. So there’s no cause for concern - I won’t be switching over to using Lion’s autosave without a lot more testing and investigation - I would have to be sure that it offers benefits over Scrivener’s current system. Lion’s autosave works very nicely for flat documents, but Scrivener’s file format is more awkward and the current code gives me a lot of control over this. So, although Scrivener is taking advantage of several nice new Lion technologies such as full screen, the auto-save remain the same for now at least.
Keith, what would you think of a full screen mode in Lion that put everything in full screen (like iPhoto) and then have a “writing mode” button to go into full screen as we know it now?
Personally I’d like to put my voice behind the idea of implementing Lion Autosave and Versions.
I had some of the same concerns as the original poster, but after a day of using Lion Scrivener has already started to feel slightly held back by the lack of those features. (Don’t get me wrong, it is still possibly my favorite piece of software ever.)
The great thing about the Lion implementation is that it gives you the best of both worlds of manual and automatic progress saving and in a very unobtrusive way. If you aren’t familiar, basically Save becomes Save a Version and forces not only an auto save, but marks the state so that you can revert back to that point.
For instance:
You open a document, and do an hour of heavy editing. Great, you think you’re finished so you do a manual Save Version. But wait, maybe you should do a little more. You do a bunch of experimental edits. Then suddenly after 30 minutes you are called away or the power goes out.
As it is currently, autosave will give you the most recent version. With Lion’s Versions you can come back and again see the most recent version. But there are two really great additions. First, you can very easily revert back to that manual save you did, getting back to the last “stable” state. Second and even better, after you’ve reverted back to the manual save you might realize that you actually really did like the first ten minutes worth of edits you did. Fortunately they aren’t lost when you reverted, and you can go back in to pull up the auto saves from after you did the manual save. Then you can either copy text out or switch to one of the more recent versions.
Obviously if you are religious about using snapshots, you can get the same protection now. But that’s like the pre-autosave days. You have to remember to do it and do it all the time. Lion’s versions are just lighter weight on the user.
I wouldn’t want to remove the snapshot feature. That would still be great in parallel so you can save a version of a single document forever. But having the Lion feature would allow you to use them more sparingly rather than clutter the list with a new snapshot every single time you start making a change you’re not sure about.
Sorry for the long-windedness. Also, I do realize that there are undoubtedly big technical hurdles for you to leap because of the file format. However programs like Keynote that produce complicated files do manage, so I’d hope it isn’t strictly impossible.
Actually Versions and Lion’s autosave isn’t compatible with Scrivener’s project-based format. We really need to put something up about this on our FAQ, but basically, in order to support the ability to import any research files and thus large projects, Scrivener only ever opens and saves the files you open and change during any particular session. That is, if you open project A and only edit documents 2 and 4 out of 200 documents in that project, then only documents 2 and 4, and Scrivener’s binder structure, will be saved - all other documents inside the .scriv project will be untouched. Scrivener’s project format is basically a hybrid of a regular document-style program and a database-style program. However, Lion’s autosave and Versions only works with file formats that can be saved in their entirety every time you save. In other words, for us to support Versions and Lion’s autosave, we would have to ditch the ability to import any file you wanted as research - importing PDF files, images, movie files and wotnot would have to go, otherwise every autosave and access to Versions would take forever for long files.
That is why Scrivener won’t be supporting Versions or autosave - because one of Scrivener’s main features would need pulling to do so.
Certainly the ability to import PDFs and other files is a key feature. I also have appreciated the package based format for allowing you (in a worst case scenario) to crack open the file and read individual rtf files.
That said, if there were some way to have it all, I would love it. I’ll go back to my Keynote example and point out that you certainly can have very large files that are essentially a mix of PDFs, Images, and Movies. I haven’t had a lot of time to experiment with it, but it doesn’t seem too dragged down when using autosave and Versions.
Now, they have moved away from the package based format. That has some advantages, certainly, especially when dealing with things like Dropbox that still don’t quite know what to make of packages. Though of course you sacrifice the ability to get at the files inside easily. But there are plenty of export options, so I don’t suppose I’d be terribly worried about that in the end.
…And yes, I realize I’m in pie in the sky 3.0 or beyond territory here, but it is something to think about.
Again, great job. You’ve got a truly great application here.
My first thought was that it might be possible to have Versions enabled for each individual file in the package, but that would probably be rather non-trivial.
And then I realized two things:
Scrivener already auto-saves.
There’s already a keyboard shortcut for creating a snapshot. You can even go into System Preferences and change the shortcut key to make it more like Versions in other applications.
So you can just treat it the same way as Versions in other applications. The only difference is you won’t get the fancy GUI with it. I don’t think you’d want it to do Versions across the entire package anyway, as it would be harder to browse and see changes. But if you really want a snapshot of how your project looked at some point, Scrivener has an auto-backup feature already.
(I actually am curious if what I said first is possible in a package format. I’m not sure it’s a necessary feature, but I’m interested on a technical level if anyone happens to know.)
Just touching on the backup feature, remember that you can use the Back Up Now command at any point when working to save a “Version” of the entire project if you really want that, and you can set your preferences so that the Back Up command runs when you use “Save”–making it even more like creating a saved Lion-style version.
I find Apple’s notions on the UI for this to be very strange. Why is it that poking through versions for a single file demands that I set aside my entire computer until I’m done doing so? This sort of tyranny makes sense in the context of Time Machine. When restoring from backups it’s a good idea to focus on that task—but for a single file? I don’t know, I think Scrivener’s two-pane table setup is superior to Lion’s version browser. I’m not really talking so much about the specific details as the overall feel of the interface. There doesn’t seem to me any advantage to the Versions UI. It would be kind of hard to compare two different versions, too—and if takes up the whole computer (or at the least the whole project window) you couldn’t drag snapshots to an editor to examine them in a full split, either. You wouldn’t be able to leave a snapshot open while poking around elsewhere in the project (something I do a lot).
Kind of strikes me as being like when Apple applied Cover Flow to browser history, while forgetting the cover in the name. That feature works fine for flipping through CD covers (and even to an extent icons, though that always seemed a touch gratuitous to me), but squinting at little web page previews; is that really useful? Same thing here—the Time Machine interface was a good idea, it works well for a simple backup restoration interface that isn’t intimidating, but the whole thing doesn’t pair well with Versions. It’s like they got stuck on having this good idea at one point in time and now they are trying to use it too much.
MM points out a good tip for emulating Lion style versioning. Up the total saved backups to the maximum, turn zip on, and you’ve got pretty much everything Lion offers. The history doesn’t go back as far (though if you use Time Machine that isn’t the case as TM backs up those backups meaning you can have years’ worth of them) and “locking” is achieved by putting the .scriv file in a zip container so you can’t accidentally open it and edit it.
Yes, it would be difficult, especially since a “file” in Scrivener is more than one file in most cases. If you’ve ever used synopses, then you’ve got two files not one, for each item that has a card. Document notes add another file. Any linked comments, footnotes, or Scrivener links adds another file. So each item in the Binder can represent up to four actual files. What does Versions do here? Does it act like Snapshots and only work on the main text itself? That would probably be the best solution—but even then it is still not technically possible.
The Auto-Save/Versions feature in Lion is a thing you can toggle on for a window (that’s a bit of a gloss, but you can think of it in those terms). So you can only get the user-end interface as that arrow up in the title bar. You can’t program it to appear just anywhere, and there are no hooks into the deeper system itself so that you could code your own interface and put it wherever you like (such as in the header bars) at least not and still maintain a presence on the Mac App Store. Hence, it would only be applicable to the project window itself, and that isn’t “a file” that can have versions.
That’s the main problem. It’s a technology designed specifically for single-file document based applications like TextEdit and even more complicated examples like KeyNote. Until/If Apple changes that, you probably won’t ever see it in things like Scrivener, DEVONthink, or any programs that distribute the data load over thousands of files.
For those talking about Snapshots and backups on manual save, there is one key difference in functionality (not just UI).
With Scrivener’s autosaves, the autosave overwrites your document and you have no record of the state of the document between the last backup and that autosave. Snapshots and backups only happen when manually triggered.
In Lion’s autosave/versions setup you get a full version archived every time the autosave happens which you can force to happen (and mark as special) by doing a manual save. It is as if Scrivener did the full backup every time the autosave happened, not just when you do it manually. Except that (if I understand correctly) it does a bitwise delta to save space so it only has to remember the differences along the way.
With the Scrivener setup you have to remember to save a backup by doing a manual save (with the option enabled) or by using a snapshot every time you think you might want to be able to go back to that state. I’d argue that this is the 2011 version of not having autosaves. Sure, you might have said back then, there’s no reason you’ll lose your data. You can just save whenever you want. Now I’d say versioning is moving that way. Sure you can save a copy of the state of the file at any time you want, but isn’t it just that much nicer when you don’t have to think about it and it does a regular track of your progress in the background?
I’m not talking about the technical hurdles here. I understand that they are substantial and that advances from Apple may be necessary or at least helpful to get there. I’m just talking about why such a thing would be a desirable advance from the current state.
(That said, for the long term, I would continue to watch what Apple itself does with its more complicated file formats. I know that in Keynote you conceptually have a single file where as in Scrivener you conceptually have a collection of files, but both are essentially databases of collections of files.)
Also, as far as design quibbles about the versions interface, I think it is overly cute, but it is decently functional. You get a scaled down live preview of the current state and the selected old state. If you click on the old state it zooms out and you can move around in it, go to different parts of the document, go do a different slide, search, copy, etc. I would not, however, replace snapshots. I see them as having overlapping, but distinct purposes. I’d still use snapshots, sparingly rather than compulsively, whenever I wanted to do something like cut something major out. I’d probably stop having it do the full file backups on open and manual save, though.
Edit: To answer you a little more directly,
In my opinion from looking at what Apple does, Versions would be for full database versioning (like backups) not individual item versioning (like snapshots). You would get versions saved of the entire database. When you go to browse them, just like in Keynote where when browsing versions you can select different slides from the bar on the left and see changes in one slide or another, you would be able to select files in the binder to see changes in one file or another. Play around with how they handle it there.
The “file” here that is getting versioned is the entire database which includes multiple discrete elements. In the old days Keynote used to work sort of like this. You could crack open the package for an old Keynote file and see all the images and PDFs you had imported into it along with cached icons and an xml file that put them all together. They have since moved to keeping it all in a database that is, according to the file system, a single logical file. But logically it is still a collection of imported files distributed over many conceptually separate slides. Whether you store a collection of files in a folder or mince them all up into a sqlite database or some such may matter a lot to the developers, but from a feature and complexity perspective, it isn’t that different.
I don’t see why (again long term) there is anything fundamentally impossible with having a complex database with many components versioned. On the implementation side, it might require a change in the way the file format is stored on the drive, which is why I say this is a “3.0 or beyond” kind of thing where you might decide eventually to make a change to your file format. But I don’t think that it is impossible per se, and I do think that it is desirable.
Also, imagine what would happen to your hard drive if a file that could be a gigabyte in size were saved as an extra version every single time you made a change…
That’s one of the nifty things about how it is implemented. It doesn’t always save a new copy every time. It breaks files into chunks and saves only updated pieces. Here’s an explanation from John Siracusa’s Lion review: arstechnica.com/apple/reviews/20 … -internals
I’m not saying it’s trivial. I’m just suggesting an open mind when thinking into the far future for your next major revisions.
I only kept posting because there seemed to be a general feeling of “it’s hard and why would you ever even want it.” I’m sure it is hard! But I also have great respect for you guys, so I think there’s no challenge you aren’t up to!
It seems to me that some of these tradeoffs have been considered elsewhere, too. Scrivener files can get huge, but so can Pages documents consisting of hundreds of pages with giant high resolution graphics or Keynote files full of movies.
I’m not saying do it tomorrow, but I think it is worth keeping an eye on how it works out in those applications, because if somehow all the problems could be solved, that would be great.
And finally, in case it got lost in all this, I do think your application is great. I wouldn’t be posting like this if I didn’t have huge fondness for it and respect for you.
The thing is, Versions are saved in the same spot as the original. So if your hdd crashes, you lose everything. The great thing about Scrivener is that you get the best of both worlds: Snapshot acts like Versions and Automatic BackUps save to a secondary location (which could be a thumb drive) allowing you to always have a backup safe in your pocket.
To me, Scrivener works perfectly because it gives you full control (as a professional app) while backing up to a secondary location without having to think about it.
This is an important point. Lion’s feature uses some deep level HFS+ filesystem tools to pull off what we are talking about, and this has some concerning issues in terms of portability. Scrivener isn’t just a Mac program any more. Its file format is fully Windows and Linux compatible. The projects you work on today will be something you can open years later on another platform entirely, with all of those snapshots intact and ready to use. Heck, the whole project just uses open standards that even a non-Scrivener application could conceivably salvage from in thirty years. They can survive being managed by a concurrent versioning system, or an aggressive synchronisation engine like Dropbox. Users who are not aware of the technical intricacies and requirements of the feature will not lose their version data by naively transferring files between computers on a non-HFS+ formatted Flash stick drive or a home/office file sharing server. Et cetera.
Second point; conceptual not technical: the notion of flooding your version stack with automatic snapshots has been discussed a number of times over the years. There are definitely two types of thinking on this issue. Those that phrase things in terms of “you have to remember to…” and those that don’t have a problem with that and wouldn’t want a couple of hundred minor varieties to wade through when needing to find a restore copy (or facing the prospect of losing important old copies due to a roll-off limit designed to reduce the flood and save space). Snapshots are fundamentally different than Lion versions because they are points in time where you specifically told the system that the file deserves to be immortalised in its current state. Blending that system with one that produces chaff states on the off chance that you might forget to set a milestone would reduce the effectiveness of those intentional states. Now in the past various strategies have been discussed for how this could be mitigated. For example, intentional states could be bolded in the list, and not suffer roll-off once the system saves 25 auto-save copies, etc. These are not horrible ideas in my opinion, but they are undeniably different ideas for what this feature represents, and good though some of the notions might be, there has been wonder over just how useful a bunch of 3-letter delta entries would truly be.
I’m certainly curious to see how useful Lion’s feature will be in heavily used files. Will restoring a mistake from four months ago be easy when there are several thousand frames to wade through? There are mitigation strategies—basically what Snapshots are but without the unifying interface: duplication. Apple seems to recognise this could be a long-term problem and has provided a mechanism for slicing off an intentional “super-version” right in the UI with the duplicate feature. It’s interesting that they didn’t try to blend this into the All Dominating Version Interface itself—merely made it a function of separate files on the disk. It seems they’ve come to the same conclusion Keith did with all of these suggestions from the folks who see things in terms of having-to-remember. Even with delta saves being used to optimise space, you still have a human problem of quantity and a lack of any kind of content-biased mapping in the current implementation. The only mapping method is chronological, and that’s not good enough in large quantity situations, for most people. You remember what you changed (or what you want restored) better than when it existed. When is a pretty powerful map, nobody needs to convince me of that, but when is much more potently memorable when you yourself say, “Now”. That’s what a snapshot provides. It’s chronological as well (and can be annotated for even more detail), but since you “have to remember”, each snapshot becomes much more useful than a hundred micro deltas to that particular group of people that doesn’t place so much importance on the having-to-remember problem.
Ultimately the conceptual problem is I think less persuasive than the technical problem. The main strike against the idea, conceptually, is bloat—having two philosophies of versions in the same program (and to be clear, bloat is often not the presence of too many features, but a poor presentation of those features). The fact that these versions are saved in a hidden and somewhat volatile way makes me consider them to be less of a rock-solid safety net, and more of a “whew, glad that was there” type thing. Time Machine, in a sense. Nobody would call TM a rock solid backup, but it’s awfully nice to have around in those situations where mistakes happen and you realise you need to restore from a “grey-state” that you yourself never specifically created. Lion versions are a lot like that, and even have a similar ultra-simple, not terribly useful beyond chronology recovery system. TM isn’t a good backup system because it relies upon having a disk plugged into your computer non-stop, and that means the disk suffers all of the same risks as the computer in terms of catastrophe, burglary, and human error. But it is that same flaw that also makes it superior to an offsite periodic backup—it’s coverage is massively finer grained.
Lion’s Versioning has the same exact use case. Weak at the safety level, but catching a flood of data that is difficult to otherwise catch, and in a way that anyone can do. Saving deltas into the filesystem position of the file itself means it’s hard to screw things up so long as you use your computer very simply and nothing goes wrong with it. Like Time Machine, so long as you don’t face a true restoration event, it brings an otherwise complicated (or hard to remember to do) process to absolutely everyone, most of the time. It’s a good feature—but it shouldn’t replace something that can work all of the time, on any operating system, for decades to come.
I think the importance of “being able to go back to previous states” of a novel might very well be over-rated. There are examples in the history of literature when complete works got lost and have been re-written by their authors (“Seven Pillars of Wisdom” by T.E.Lawrence, for example), and most probably they got rather better in the process.
It is, however, true that the feeling of “nothing gets lost” has a liberating effect on creativity. You delete long passages more easily if you know you could get them back if you wish (which you won’t in the majority of cases). When working in Papyrus, I don’t delete such passages, but put them on the clipboard (a special area for text snippets, not the copy-paste-mechanism of the OS); when working in Scrivener, I make snapshots or – even more often – I copy the passages into a dedicated folder named “fragments” (very easy via context menu!). Recently I stumbled across a video in YouTube where somebody using an ordinary text processor (WORD, I think) did exactly the same for exactly the same reason by cutting and pasting such passages below a dotted line at the bottom of his document.
