Verifying integrity of cloud backups

Good morning, all –

I’m trying to get a cloud-based backup system in place and a quick bit of research said that Scrivener doesn’t play nicely with all cloud services. Dropbox was a safe(ish?) one, it seemed, but my brother-in-law mentioned another option he uses called Sync.
Briefly, it’s an old (in digital terms) and reliable company and affordable and oh, as a nice bonus, the initial, free account you get is twice the amount of storage you get from Dropbox. Sounds good.

However, my question is a mishmash of a couple of areas: While it seems pretty straight-forward to assign a destination for a backup in Scrivener, how do I ensure that the backups being made there aren’t corrupt? Are there any telltale signs from just looking at a file list?
I realize I could probably just open Scrivener and open up the backup file from the cloud service to ensure it’s one piece and readable by the program, but will that then consider the backup the live version? Which it would then backup again? Maybe? I’ve never opened a backup file so don’t know what effect that has on the live version of the file or what happens with the backup version as a result, and given the trouble people seem to have with cloud backups already, I don’t want to risk muddying the waters for myself.

Of course I may well be way overthinking all of this. It’s the not knowing the details and effects of actions that’s tripping me up.

Essentially I want to ensure that a third-party backup system retains the integrity of the backup(s) in the most efficient way, without risking overwriting the live version of projects or confusing the program about which I want to be the live version, but I’m not sure how to do that. Hope that makes enough sense to be answered. :slight_smile:

Thanks for your time.

You tell Scrivener to save the backups as .zip files and there shouldn’t be any problems. And preferably with the date in the filename to make it easier to decide which backup to use.

Dropbox is the recommended cloud service. There are likely reasons for that, so why fight it?

I had the same concern about wanting to check the integrity of the backups.

In Tools>Options>General, uncheck the box that tells Scrivener at startup to open the project that was open when you quit Scrivener.

Create a new folder for the test project. From Dropbox, Drag/Copy (NOT Drag/Move) the latest ZIP file of your Scrivener project to the test folder. Unzip to within that folder. In that folder, find the YourProjectName.scrivx file and double-click to open it in Scrivener. If it works, all is good. Close the project, close Scrivener. Delete the test folder and all contents, just to keep it from accidentally being used.

Find your ‘live’ YourProjectName.scrivx file and double-click to open it. If you want, re-check the box to tell Scrivener to open the project at startup that was open when quitting Scrivener. You are now back to where you started.

To add to what Lunk said:

  1. In my opinion you should set your backup preference in options to zip the backup and include date and time in the name … that way you can easily identify the latest backup.
  2. If you zip the backup, it is a single file and less at risk of corruption, and it doesn’t matter which cloud service you use to store the backups. I use iCloud because I have plenty of spare capacity there.
  3. If you have reason to want or need to keep your live project in the cloud, Dropbox is the most tried and tested option, but for particular reasons I have some projects shared with a collaborator using Sync—she is in China where Dropbox is blocked. But do not use Google Drive (or whatever its called) or One Drive, I believe as they are known to have corrupted Scriv projects. Also, if you use either Dropbox or Sync for your live projects, do not use the same cloud service for your Backups.
  4. If you have iCloud, turn off its smart “space-saving” feature and automatic linking of your desktop and documents folder as that can corrupt scriv projects badly. I personally wouldn’t entrust my projects to iCloud anyway, as it’s not so transparent about when files are fully uploaded or downloaded.
  5. That last point is crucial: always make absolutely sure that Dropbox, or Sync, has fully finished it’s process—you can see on the menu bar—before you shut down a machine or on starting up before you try to open the project. Patience and care is essential.

I’ve been using Scrivener with Dropbox for nearly 10 years, with the late-lamented Cubby for about 5 before its demise, and with Sync for the last five years or so. In all that time, I’ve only had a project corrupted with a conflicted file about twice, and that was when I first started collaborating with my friend in China … about 7 years ago.

Oh, and I back up my whole hard drive to an external hard disk on a weekly basis under normal circumstances.

Good luck and good Scrivening. :slight_smile:

Mark

If we’re talking primarily about the storage of Zip files (which is what Scrivener project backups are by default), then it would take a pretty awful service to get that wrong. :slight_smile: You really don’t have to worry as much if that is all you’re doing—it’s the folks hosting complicated multi-file formats (like our project format) in a live sync environment that need to do a little research and testing in order to ensure the system is safe. Sync looks like a pretty good system. The full end to end encryption is important, I would say, whenever considering storing your important data on someone else’s equipment. Not many services will give you that courtesy (though fortunately there are many more these days than there used to be).

If you do intend to use it for live projects, just keep yourself extra backed up—set Scrivener to back up on open and close, and to save as many old copies as possible. For a while, keep tabs on what is being uploaded and downloaded. Periodically check your project file by file and make sure everything is as you left it. After a few weeks of monitoring, if things look good, then you can scale back on the paranoia.

But to reiterate: for storing .zip files? Don’t worry too much about that, especially with an established company like Sync. The one caution I would have is that if you aren’t really going to be using this service for syncing, you might also consider one-way true backup services as well. Cloud isn’t really a good backup because this stuff by definition has direct authority to modify your hard drive remotely, without any per-instance authorisation. That’s how it works after all: you edit a file on your phone, their server is updated, and back at home your computer receives the update and modifies its local drive automatically to match what is on the phone.

Convenient though that may be, that is just about the opposite of a backup. That’s what we keep backups to protect ourselves from. :slight_smile: If you’re using sync as a one-way, one machine upload and that’s it—it can be all right. But do consider systems where you have to go and tell the system to download your files explicitly, and grant further confirmation to overwrite or modify existing files when doing so. Actual two-way cloud sync is overkill for what you need, which means more complexity than you want, more things that can go wrong.

So for storing a backup of your backups: You close your project, Scrivener copies the whole project into a .zip file and saves it into a folder. With the folder designated as a sync location (using whatever mechanism the service of your choice uses), then a few moments later the new file is detected, uploaded, and stored on some computer somewhere. There it sits, much like a file would on any hard drive anywhere else (though arguably a hard drive in a huge server farm being run by professionals is a bit safer than the one sitting on your desk). The single largest threat to the file is that counter in your Scrivener preferences, the one that schedules it to be deleted once a sufficient quantity of other newer backups exist. If that counter is set to 25, then it’ll sit around until you close the project 24 more times in the future. And even then it might still live on, in your service’s deleted-file restoration system if it has one, for a time.

Nothing specific—but one file in the middle of a list that is abnormally small or large might be a hint. In general it’s not really something you have to worry about though. Outside of long-term entropy of physical storage medium, files don’t “corrupt” by sitting around. They are most at risk when being opened, modified and saved—and that’s not something that happens to .zip files very easily. That’s one of the big reasons for why we use zip compression by default: it means people have to take a step to get their project back out of the .zip and the very act of doing that creates a whole new copy every time you do it, with the zipped version untouched. It means people sometimes have to write in ask what a .zip file is—but we’d rather have that than have backups that everyone can freely open, modify and close without barriers—because that is what puts stuff at risk. Even if that risk is low, .zip is even lower.

I’d add one more potential checklist item when testing backups: when you load the backup, go into the File ▸ Backup ▸ submenu and switch of backups for the backup. Of course if you intend to stick with this copy you’ll want to leave that on :exclamation:, but it will be less confusing if you aren’t creating backups of backups of backups as you go through a stack of old projects.

I also heartily second tomwood’s advice to keep it clean. Keep your backups zipped and your working copy the only one hanging around that you can open directly in Scrivener. It’s certainly how I work—things just get far more confusing the moment you have more than one version of one project capable of being opened. In the short term you may need it, having two copies open side by side for comparison, when restoring from a backup for example, is how you do it. But it should be a temporary state, and I like to rename things very clearly when doing that. “My Novel - DO NOT USE.scriv” is pretty clear when next to “My Novel”. :slight_smile: If you’re anything like me you’ll be 50 minutes into editing your backup before realising where you are.

I just want to second Ioa’s advice about the difference between “backup” and “synchronization.” A true backup service makes it nearly impossible to overwrite your data by mistake. A synchronization service deliberately does exactly that.

Katherine

Thanks for all the great feedback, everyone. Much appreciated!