CHANGES LOST w Dropbox after offline work

Dearest Scrivener-people:
I hope you have some hint for me to resuscitate 10 hours of work from digital Nirwana.

System:
MacBookPro 13" late 2011, OS X 10.8.2.
Scriv 2.3.1.
using Dropbox (project is never open on more than 1 machine at a time)

What happened
in short:
MacBookPro ran out of batt while working offline. An hour later, on connection to power, MBP wouldn’t wake up but restarted to last open state, i.e. autostarted Scrivener, Scrivener recalled & opened last open project. Dropbox started syncing.
Next time I looked at the project, all changes made while working offline had disappeared, the actual version of the .scriv document stated a modification date from the previous workday.

What happened?
Any chance to restore?

In detail:
Yesterday, while working on the road (i.e. no internet /dropbox connection), my MBP ran out of battery & blacked out.
I wasn’t really worried, because I had saved (even manually, old reflex) many times while working, and felt safe knowing auto-backups was on.
When at home I reconnected, the MBP wouldn’t wake up, but directly restarted. (Some issue of my MBP that had happened before on several occasions, when batt went totally empty.)

After restart, Scrivener automatically opened the project I’d been working on.
At that point I didn’t make any new changes, and I did not look close enough at the project to see wether it contained my new changes, but went to sleep. (so did the computer)

Today in the morning, when I started to work I realized that the whole project was back at the state of the day before - ALL CHANGES I HAD MADE ON THE ROAD HAD GONE LOST.
Newest Modification Date was at the of the day before I made those changes. (day before yesterday)
There was one “conflicting synchronization” file (.scrivx) but that couldn’t be opened. (documents missing) and later just disappeared from Dropbox.

So I tried to go back to a backup copy: Only to realize that the latest backup was TEN DAYS old - Wasn’t aware that I had Autobackup enabled on Project Close - and hadn’t closed it in many days.
(proposal: Would it be difficult to introduce an option for a regular timed backup (I remember many film-editing softwares doing that very reliably)

So far so bad.

Am i right to suspect that the special set of circumstances created some confusion that might have led to my dropbox sync overwriting the Scriv file with the newest one available (which was from the day before). Which is quite scary to imagine in itself: SAVE used to be pretty much SAFE!

Anything to be done?
If not: What would I need to be aware of in order to avoid breaking my neck a second time???

thanks for help,
Ocab

PS:
Not that I believe Timemachine would have really helped much with the core-problem, since I hadn’t done any backups on the road, the latest backup time after offline work was only several hours after the restart. (which was where I guess that things went wrong)

But trying to restore from that Timemachine backup , I made a new, equally uncomfortable observation:
A message came up saying:

The action couldn’t be completed, because you don’t have the necessary access rights for “(projectname).scriv”. <<

On a second attempt the “no access rights” message referred to a specific doc, a pdf that was included in the scrivener document. But equally, none of the .scriv project was restored.

I tried the same with another .scriv doc, this one DID restore without any probs.

Might this “access right” prob hint to something relevant about the general problem?

So when the computer starts up this way, does it have a different kind of progress bar along the bottom of the screen initially? If so, it might be waking up from hibernation. If you leave a computer asleep with a low battery, it may run out of battery. At that time it attempts to write the RAM to hard disk storage and do a full shut down. So the next time you turn on the computer it appears to boot, but what it is actually doing is reading the state of the computer from disk. It’s essentially identical to opening the lid on a sleeping computer—only the saved state of the machine is on the hard drive instead of RAM and so it takes much longer and goes through a sort of mini-boot to come back online. If that is what you are describing, then nothing actually automatically restarted, it was running all along, just as when a computer is put to sleep.

I don’t think that should cause issues with Dropbox though. I’ve had my old MBP do that a few times and I use Dropbox on it; never had any problems with data getting overwritten, but maybe I just didn’t notice. Perhaps there is something about the hibernation start-up that confuses the system clock. I don’t think so, but I’m just thinking out loud here.

There could be a read-only PDF files in the project. That wouldn’t typically manifest as a problem unless you tried to edit it externally (or overwrite it). I doubt it is a clue. If there was a deeper permissions problem, Scrivener has safeguards against that and would have shut down as soon as it couldn’t write, and dump all RAM to recovery files in your Documents folder.

It sounds to me as though you hit a perfect union of bad variables this time around. I unfortunately have no advice other than some tips for future prevention:

  1. I don’t trust any synchronisation. It’s a convenience, but it’s risky. Always back up your computer immediately before leaving on a trip and immediately upon return, before plugging it into the network, but this is especially true if you have singular copies of files in folders that are moderated by a machine.
  2. Enable the Scrivener back up option to create a backup on force save. It sounds like you already do that as a habit, so now your mostly-useless habit can be made useful by binding backups to it. This is in fact how I work. I close too many useless projects to want them all backed up (testing projects and such), so I leave that off and just remember to back them up “manually” by hitting Cmd-S now and then. Timed backups were tried for a while, but it was too intrusive. You have to be locked out of the project while it backs up, and that can be a few seconds with larger ones.

Amber!
Thanks a lot for your detailled answer, and your hints.

I am familiar with the hibernation wake-up process, but the sad truth is that in a few occasions my MBP has had problems with that, so it actually restarted instead of the slow wakeup you described. I suppose I should have that behaviour looked on - especially after this experience. From what I saw in those few cases (including yesterday’s), it seems to me that at occasions the safety shut down on low batt takes place so late that it doesn’t manage to terminate writing the RAM to disk.
Which is what I suppose started the chain of incidents causing the data loss in the first place.

I actually found a “conflicting copy” file on the Dropbox website for a moment that contained SOME of my changes from the day- just sadly some of Dropbox’s automatic processes eliminated it before I could take a deeper look.

I already heeded your advise nr 2), that is, backup on manual save. Also, for the time being, I took the main project out of the Dropbox folder, and just direct the backups on there.
Still it would definitely be more convenient for me to keep my whole work folder available from dropbox. So I’m thinking of ways to make that more secure.
My thought is that if I had kept the main project on dropbox as I did, but setting the Backup to “on manual save” (zipped), going to my MBP harddisk,(and those being backupped by my Timecapsule whenever I’m home) this wouldn’t have been any problem.
The only additional safety measure I’m thinking of is to return to making manual backups of any important piece of work to a flash memory - especially while working on the road.

I feel that way I could avoid any repetition of this kind of desaster… Agreed?

Anyway, it’s always great to know that Scrivener is not only a great piece of software (honestly my favorite) , but that also there are such competent people like yourself to swiftly come up with answers to all the problems.

Thanks for your work!
Oliver

By the way: There is one very strange thing that took place in the whole affair, not sure whether it might be of any interest.

I found at some point that my project’s “mother”-folder in the finder suddenly appeared with a .scriv extension.
Both the scrivener project as well as the “mother”-folder shared the same name, but obviously without the extension - so normally this wouldn’t be any problem. I don’t have the slightest idea how this could happen.

Confusing? Yes. Let me put it like this:

If my project were called “Dada”, the folder structure would look like this:

  1. USER folder -> 2) "WRITING PROJECTS" folder -> 3) “Dada” folder -> 4) “Dada.scriv” along with all kinds of other stuff.
    Now what happened is that 3) “Dada” (folder) changed to 3)“Dada.scriv”, which still contained all the same stuff as before, including 4) Dada.scriv, just hidden inside a packet.

Point is that this way the finder showed it with the icon (and name) of my actual scrivener project, but in the wrong folder level (3 instead of 4), which confused me a lot at first. Obviously Scrivener displayed an error trying to open it.
The whole issue got solved easily by just deleting the .scriv extension.

First I thought this might have happened while trying to fix it, but in hindsight I really didn’t do anything to mess with the extension of that folder. So might it be just this behavior that could have caused the whole desaster?

Just a thought…

best regards,
O.

You’re welcome, I only wish I had better news. Well the way I work is slightly less convenient, but has no risk at all of project woes. Basically I do things the opposite of how most do: I have all of my projects on my local computers, but all of my copies of Scrivener are configured to write backups to Dropbox. So way as soon as I hit Cmd-S, I can see the Dropbox status indicator light up and I know a few seconds later my backup will be fully uploaded and eventually downloaded (and thus copied) to each machine.

Thus, I have a stack of the latest backups on Dropbox, available to all of my computers. So what I do when I switch computers is grab the top backup off of the stack and replace the current working copy of that project on the computer, with the backup. Now I’ve updated the local computer from the latest version. When I’m done I just backup as per normal and the other computers get all of my changes.

So the idea is to store your changes as a stack of files on Dropbox, rather than having the project on Dropbox, and saving the stacks of backups on each computer. It’s just inverted. Backups global: work local. Then you never run into any conflict issues even if lightning strikes.

But to be fair to the available techniques, the developer of Scrivener uses the method you use, as do many others. I chose a more conservative route because I’m always forgetting what I’m doing and make mistakes that end up in conflict files in the project.

If you feel something broke with the hibernation process, then that could indeed be a clue, but even if the contents of RAM were destroyed, you should have still at least had everything up to the last auto-save. That wouldn’t explain a ten day reversion. That seemed to happen automatically when it booted up.

That works as well. It’s basically the normal way of working. You should keep at least one copy on and another off Dropbox just to be safe. So if you store the live project there, it is best to point the backups some place else. And yes that would have protected you in this case.

I think so. Working with synchronisation is always going to be more risky, but we’ve gone over steps to mitigate that significantly.

I have no idea what might have caused the “.scriv” to end up on the parent folder. As you noticed though, it is the appearance of that extension that Finder users to determine if a folder is a “package” or not. If a folder ends in .scriv, it gets an icon and acts like a file, no matter what is inside of it. The same goes for actual Scrivener projects. Remove the “.scriv” from their name and they will act like normal folders until you replace it.

Have you checked through versions yet, in the Dropbox website? There is a dim hope that it updated to what your computer had, then downgraded to the older version on account of some time stamp confusion, thus saving the originals as older versions.