I only used snapshots occasionally, and not as a backup practice. I tend not to edit without knowing more or less what I’m going to do, where I’m going, and so I don’t often have that “oh, shoot, I wish I could get back the previous 15 minutes of edits” kind of moment (until I lose a few days of them!).
And note that the older / earlier version of the project I opened was not given to me initially by Time Machine after the crash / loss: Scriv. produced this earlier version when I opened the project (after answering those two queries about the “project on another machine” and the “earlier version of Scriv”). It had the same name, of course, but it was simply a version that was made before I’d done about a week’s worth of edits, new labels, a couple of new saved searches, etc. etc.
It was only after I discovered that Scriv had not in fact returned to the current version of my project that I thought, “oh well, not to worry, I’ll just go find the latest version in Time Machine; at most it’ll be an hour out of date.” And at that point, I went back over copy after copy of the project, and none was the most recent; the closest I got was a few days prior–a little better (i.e. newer) than the one Scriv. had given me, but not up to date . . .
j
Thought on how you can “get the best of both worlds” from TM.
Exclude .scriv files from TM backup.
Set TM to backup every hour.
Once every hour, create a zip backup (backup to).
At the end of the session remove all backups but the last one.
This will give you hour by hour snap shots in TM and a full day on disk without you needing to care about what drives are plugged in. It will also ensure the integrity of your project in scriv land.
At to my statement about TM giving you the earlier version, I am not implying that TM was asked to put it there as much as TM was involved with the change.
Is it at all possible that you have a snapshot that matches the project post incident?
wonderfully clever, Jaysen–I’d forgotten that you can exclude files from TM–the whole “it’s so simple there’s nothing to do but plug it in” approach of TM had addled my brain, I guess. nice touch to exclude scriv. files.
I had done the zip file backup with the previous thing I was working on using Scriv.–a review essay on the war journalism of A. J. Liebling foxyurl.com/7oy
–but had, well, for some inexplicable reason, neglected to do that with the book introduction I was working on. I have, subsequent to The Great Disaster, managed to rewrite most of what was lost.
And, after all, when considering data loss, one can always think of Hemingway’s lost MS on the train. Poor Hadley had to take most of the blame for that one . . .
thanks for all your thoughts about this!
j
As mentioned elsewhere I read in another forum about a case of data loss by a mysterious reset of a file to its state of one or two weeks earlier. (No, Keith, I have not forgotten that I wanted to send you the URL and/or translate it. I was just very busy. And still am.)
And this did NOT involve Time Machine and, more important, Scrivener. It was a text written in Word which had been saved manually “after every paragraph” for security reasons (I pity them Word people!). I do mention this because it has in common the very high frequency of file saving with the back-in-time Scrivener projects.
Speculations aside why this bug occurs—I couldn’t come up with more than what has already been written here anyway—this is what that Word user was recommended:
And it worked, she was able to find her file in the newest version!
Of course with a .doc it is way simpler than with a .scriv package file. But why not try it?
I never had to use the recovering software, but I do know that two things are important: 1. Make sure that you do a backup before so it can not cause further damage. 2. Do it quick because the longer you wait the bigger is the chance that the sectors your file is in will be overwritten.
Hi,
I have skimmed thorough this topic because it seems appropriate to my issue.
My hard drive crashed and was replaced my Apple under warranty. I used Time Machine backup for the new drive.
Then I discovered that my dissertation file was missing (not in Documents, not listed in Finder searched by project title, nor by searches for .scriv files)
Searching TM , I found the file in a February 28, 2009 backup.
That means 6 months of work missing
I admit sometimes I ‘sleep’ the computer instead of turning it off. But I certainly turned the computer all the way off lots of times in the last 6 months. And each time I opened Scrivener and went to ‘recent projects’ to work on the dissertation, the file was in the list, and it opened. It had to be saving somewhere.
Is there any way for data recovery? Any more news on what is going on here?
thanks
Asharah, that sounds like a slightly unrelated issue as the primary reason for your data loss was a drive crash; the others here have simply had their Scrivener project suddenly reverted while the rest of the system seems fine and dandy. The issue you are describing is what looks to be a “simple” TM malfunction; potentially related to some of the causes of the above bug, but we don’t really know yet.
As for recovery, the only back-up you’ve made at all in the past six months was the TM disk? I hate to be the one to convey bad news, but it looks like TM might have really screwed you over on this one. I cannot stress this enough: With important work always make as many backups to as many places as you can. Defined “technically” TM is only marginally a backup solution as the disk is located in the same place as the host machine, most often on the same power circuit, and as you’ve witnessed can be unreliable.
Data recovery might be an option, if it is worth it to you, and you still have the original disk, you could take it down to a data forensics technician and have them go over the broken disk. It isn’t something a casual user can do themselves with a broken hard drive as it requires expensive equipment and advanced low-level techniques. It probably won’t be cheap either. You are right, it had to be saving somewhere, and if you still have the broken drive it probably can be found.
Not sure if this helps. Firstly, I have never had a single problem with Scrivener throughout the years I have used it. In fact I would go as far as to say it’s probably the most reliable application I use on a regular basis.
A few weeks back I updated to OS 10.5.6 (this is a well know buggy update), after Installation I restarted and instantly ran into problems. The computer kept restarting and other problems arose. I decided the only safe root was to erase my hard drive and start from scratch. Before doing so I decided to do a Time Machine backup as added security should something go wrong - I haven’t used TM in years after finding out it wasn’t backing up files properly, as outlined in a previous post of mine. Having tried to keep a backup for my extensive iTunes library I then found to my horror it wasn’t backing up the most important file, the darned library. Useless I thought. Anyway, that aside, this would be my first TM backup in a long long time.
The process:
Absolutely no applications were open when I did the TM backup. Afterward I did one last zip export of my Scrivener document then copied the original document to an external drive. I then proceeded to erase my entire disc and reinstalled OS X.
After Installation I went to open my original Scrivener document only to find the save date was wrong. On opening it, as the date implied, I’d lost four days work. I opened the zipped file and it was all present and correct.
I may be barking up the wrong tree here but I find it somewhat strange that having never had an incident with Scrivener, using it nearly every day for a couple of years, I use TM once and a file seems to backdate itself by four days. Funnily enough I intend to disable TM again, I’ve survived happily enough without it till now.
So the moral of the Time Machine story seems to be that we should get in the habit of saving our work, quitting Scrivener outright, and telling time machine to back up when we’re finished with work for the day.
I’m sure that there are many kind of saves that could suffer from TM interrupting them, iTunes certainly being one. This certainly wasn’t a Scrivener issue since Scrivener was not open at the time TM did its work; neither was Scrivener opened again until ‘after’ I noticed the document’s save date was not as expected.
Similar problem, not TM related. Over the weekend I made a backup using ChronoSync. Now, when I try to open the project file, Scrivener gives the “Not closed properly, re-synchronize search strings” dialog. If I click OK to continue, the dialog window disappears, but nothing else happens. Scrivener stays responsive, but no project is open.
Four questions:
1 (Urgent): I can browse the package contents, so I have hope not all of my work is lost. What is the best way to recover all the individual files, retaining the structure of the project?
[Edit: just found out that creating a new project, and then using File > Import > Scrivener Project recovers just about everything. Label and Status info is lost and reset to their default values, document notes are gone, but all text is available within the proper structure. Nice, BUT: might I conclude that there is nothing wrong with the project file itself? And Scriveners handling of the “not closed properly” situation is, well, not very robust?]
2: Is there any command line method to set the flag (or what have you) to make the project file ‘properly closed’? It might be that Scrivener is just a bit too finicky…
3: The message throughout this thread (and elsewhere on the forums) is: backup! backup! backup! But paradoxically, making backups seems to be the cause of this problem. How should I backup my files without running into these problems?
4: Why is Scrivener so sensitive to this “Not closed properly” status? I can’t recall using a Mac OS X app behaving the same way.
[Edit: Scrivener remains a fantastic program, but imo this ‘instability’ issue should have high priority.]
Thanks for the kind words about Scriv. I hasten to add that this isn’t a “stability” issue, though, so let me explain:
First, the “project not closed properly” message. You are right that you don’t see this in other programs - that’s because it is something I built into Scrivener. It may seem annoying, but the alternatives are worse, but you need to know a little about Scrivener’s file format to understand why. Scrivener uses a package format rather than a “flat” format. A package format is really just a regular folder, except it has a file extension and on OS X it will also have an icon and act like a single file. Were you to move it to a Windows machine or even onto another Mac on which Scrivener was not installed, however, a .scriv file would just appear as a folder. Inside that folder resides all of the components that make up your project - text files, images, the binder structure file and so forth. By contrast, “flat” files are single files that contain all of this information packed into one file.
Because Scrivener uses such a file format, it needs an efficient way of searching through the files inside a .scriv project. For this reason, inside every .scriv project there is a binderStrings.xml file. This is a simple XML file that contains plain text versions of all of the text in the project. Being plain text, it generally doesn’t take up much memory, and so can be loaded whenever a project is opened. When you run a search in Scrivener, it can then look through this file, and that will tell it about files associated with various texts and so forth. It can also be used by Spotlight to locate Scrivener projects containing certain text snippets. Although these file is unlikely to eat up much memory, it can nevertheless grow to a few megabytes in large projects, and thus take a few seconds to save. This can slow down Scrivener’s auto-save process, which is potentially annoying. For this reason, the binder strings file is only saved when you explicitly hit cmd-S or go to the file menu and select “Save”, or upon project close. It doesn’t get saved during auto-saves. If it isn’t kept up to date, it’s not a big deal as we can rebuild it, although that may take a few seconds. Therefore, Scrivener needs some way of keeping track of whether this file has or has not been saved. For this reason it keeps an internal flag, which tells it whether a project is open or closed and saved. If you open a project whereby this flag tells Scrivener the project wasn’t closed properly, this is an indicator to Scrivener that it needs to update that internal text strings file - thus it shows you the message and rebuilds the search strings file. However, it could equally mean that you have the same project open in two places and that it hasn’t been closed in one of those places, so it will warn about that, too.
And of course, if you copy a project while it is open, then the copy will have the “not closed” flag set as the search info won’t be up to date in the copy (unless you explicitly hit Save), so when you open the copy, Scrivener will need to rebuild that file.
This is the reason the warning appears - however, in your case, this is not the cause of the problem. You say that after the search strings are rebuilt, nothing happens. I bet if you open up ~/Applications/Utilities/Console.app you will see an error reported there when you try to open this file - probably something about being unable to unarchive a dictionary. Do you see something like that?
This would indicate a different problem entirely - that in the process of synchronising the project, the process has somehow corrupted the file that holds the information about the binder structure.
No, this doesn’t mean that there is nothing wrong with the project necessarily. Was the structure intact? If the structure was intact then it would indicate a different problem other than file corruption. But if you just received a flat list of files, then this would indicate that the import process couldn’t get the information from the binder structure file and thus instead rebuilt what it could from elsewhere in the project directory.
You can delve into the project package and fine the “UI.xml” file and change the “ClosedProperly” flag to “YES”. All that will do is prevent the search info from being rebuilt, but you could try it to see if the project opens then. If it does, then it does indicate a problem at the search rebuilding stage - if that’s the case, I would be grateful if you could send me your project to support AT literatureandlatte DOT com so I can take a look.
The cause seems to have been synchronisation - that can be different from backing up. By far the best way of backing up is to use Scrivener’s File > Backup To feature and choose to zip up the backup. This ensures that the search info is updated etc before creating the backup. Of course, to be doubly sure, unzip the backup and check it opens. That way you know you have a working back up.
The problem with synchronising a .scriv file is that it is prone to the same problems that synchronising a folder is prone to. I would need to know a little more about your exact synchronisation set up, though. I have never used ChronoSync so would need to look at that, but if you could tell me exactly what you did, that would help. Were you synchronising two ways (that is, trying to update an existing version of a project with an updated version) or one way (just copying a project), for instance?
Incidentally, given the problems mentioned above, you are probably wondering why Scrivener uses a package-based format, then, rather than a flat one. The reason for this is that Scrivener files need to pack a lot of information and data, as you can import movie files, PDF files and so forth. “Flat” files tend to get read into memory in their totality. A Pages or Word file is unlikely to get so big that reading it all into memory at any one time is going to cause a problem; and besides, you want to see the entirety of these files in the interface at once. But reading a Scrivener project containing 100MB movies, hundreds of PDF files and so forth all into memory at once, and keeping it there while you work, could slow everything to a crawl. And really there is no need to have everything in memory all the time anyway, as you generally only ever look at a couple or three files at a time. The package-based format allows programs such as Scrivener to load into memory only what it needs. It is ideal for the job, but unfortunately the pay-off is that it does not play well with synchronisation.
Looking into this more deeply is on my list for 2.0, though.