Is the backup system unreliable?

Take a look at this video from a novel-writing YouTuber.

She claims that due to a software update, she lost all the data in her current project, and the backups were only readable as headlines. She also claimed that tech support was ineffectual and unconcerned. She was able to rescue her own work by finding a manual backup and a device running a pre-upgrade version. She then sums up by saying she quit Scrivener.

How common is this experience? I use 3rd party software to backup my projects, but have been thinking of using Scrivener’s built-in backups more often. Is the backup system unreliable as she claims?

It would be extremely unusual for a software update – to either Mac OS or Scrivener – to cause this sort of data loss. Something else is going on here.

If the “backups were only readable as headlines,” that almost certainly indicates that she was relying on Dropbox or a similar service, and that only the .scrivx file was transferred to the local system. I don’t know what specific backup mechanism she was using, but that is not how the backups Scrivener creates behave.

By default, Scrivener’s automatic backup mechanism creates ZIP backups, which compress the entire project into a single file and are therefore immune to this particular error.

I’ve said it before, but it bears repeating.

  • Do not rely on Dropbox or any similar service as your only backup of critical data.
  • If you do use Dropbox or a similar service, ensure that folders containing Scrivener projects are configured to be “available offline.”
3 Likes

Is there a problem with syncing Scrivener’s auto-backup folder with a cloud sync service, so the ZIP backups would be duplicated in the cloud?

Can you please clarify your question. Not sure I understand your hypothesis/scenerio.

My take:

  • I’d never rely on Dropbox sync for backups. Any flaw local or server gets replicated to other. Any data loss by this is down to user error, but with proper backups can be mitigated.
  • Re Scrivener Project Syncing to iOS and other macOS devices: Partially downloaded Scrivener projects (if Dropbox sync set for “online”) will only lead to tears and data loss. That’s on the user if that happens. Dropbox synced “offline” set on for Scrivener works well with the purpose of syncing with the Scrivener iOS app. I’ve been doing that for years.
  • Re Scrivener Project Backups: Certainly, Scrivener backups saved as one-file ZIP files I think are less prone to file corruption than backing up projects (hundreds if not thousands of files). But I put them in a secure backup location which itself is backed up. For me that’s multiple locations, but none on any cloud sync service.
2 Likes

That was my thought as well. My question was whether there are known problems with syncing the folder of ZIPped auto-backups to a cloud service.

I’ve tried extensively with multiple cloud services (all set to offline) while writing my books on Scrivener without issues. My concern was to not recommend any service that could result in data loss.

ZERO issues apart from the need to watch iCloud carefully. It can be lazy syncing, but as long as you are only using it as a backup, the local copy should be fine.

ALLWAYS use the .zip option.

2 Likes

Yes, there are possible problems with data loss due to possibility of file corruption replicating in a sync. Don’t put backups there and assume all will be ok forever.

This is not a Scrivener problem. It may well be a problem self-inflicted by authors using a cloud based backup.

Edit: See my previous posts with what I do and recommend as folder configuration for Scrivener projects and backups, including system backups.

2 Likes

On the assumption we’re talking about Dropbox here…

Short answer:

No.

Slightly longer answer:

Nooooooooooo.

I personally rely on a foolproof method as my first line of defence against data loss, which is to write absolutely nothing of a sufficient quality to warrant saving. I appreciate that this might not be an option for some of you, in which case you should avail yourselves of additional backup methods.

The potential issues from Dropbox sync are twofold:

  1. You absolutely have to wait for syncing to finish before opening elsewhere. Not a problem if you’re using Dropbox as a backup or online storage for a single device, but potentially an irritant if you are using Dropbox to keep files up to date across multiple devices you wish to use Scrivener (or other software you are using in a similar way across devices) on.
  2. As typically set up and used, Dropbox is replicating your folders from your computer. That means an error, corruption, deletion, etc on your Mac (also etc) will be replicated in your Dropbox copy too. However, you can also use “Selective Sync” on Dropbox to have folders that don’t appear on your local computer at all. This is not the same as having them as “online only” copies which are visible on your Mac, but need to be downloaded each time you use them — these are in folders that aren’t visible / interact-able at all on your Mac and are only visible through Dropbox’s web portal. These types of folders make excellent Backup options as even most services pitched as true backups (eg Backblaze, etc) are still subject to the dreaded desktop error replication flaw.

Dropbox preserves file structures (not all cloud services do), so you can actually run your main copy of Scrivener project off a Dropbox-synced copy — this is how the iOS versions are designed to work, in fact. ZIPed backups are just as safe on Dropbox. This (along with occasionally manually uploading copies of important backups to the aforementioned only-in-the-Dropbox-web-portal folders) is how I work as standard.

1 Like

As an aside - I simply don’t believe what she’s saying in her video. That doesn’t sound at all like the Scrivener Support humans. I’d take that with a large pinch of “artistic license” flavoured salt in her storytelling.

2 Likes

Can I just say, it’s really ominous to see the little “AmberV replying…” logo just under a thread for 15 minutes.
IMG_0117

There are no technical challenges to syncing a folder of files, whether it be some zip files or whether it be the full contents of a .scriv project. There are a lot of misconceptions about the latter, that it is a “package” (a fiction off of a Mac—which means any sync server), and so on that make it “hard” to sync properly. No… it’s just a bunch of .rtf, .txt and .xml files in some folders. If your sync service cannot handle that it is garbage (although it is true, copying one compressed file over the Internet will always be dramatically more efficient than the original).

It’s how you use the files that matters more (and how you download them, as noted).

Can I just say, it’s really ominous to see the little “AmberV replying…” logo just under a thread for 15 minutes.

I went to make coffee! I’m back.


Oh, and briefly on whether Scrivener’s backups themselves are unreliable—I’ve never seen any evidence of that. What we tend to see are garbage-in/garbage-out problems, which are unavoidable. If the project being backed up is compromised, so will the backups, and if you only keep five of them, the good ones can roll off pretty quickly.

3 Likes

What is your coffee of choice? I seem to remember that, despite the choice of company name, KB has quite modest coffee tastes.

This raises two questions:

(1) Are you saying Scrivener often corrupts its own files prior to being backed up as a ZIP file?

(2) If a user syncs their Scrivener autobackups folder to a cloud drive in archive mode (add-only), how can the sync create data loss?

…especially if you keep opening and closing your project to try and figure out what you’ve done wrong!

2 Likes

What are the top three most common ways users compromise their Scrivener projects?

I don’t even know how to answer that! Are you talking about regions the original bean was grown in? The roasting methods? Blending? I suppose if I had to narrow it down to an apex choice, it would be this selection of beans from Ethiopia, circa 2015, roasted by this tiny little artisanal specialist in the Portland area with one bedroom-sized cafe, that blew my mind.

But like all the coffee in that place, it was gone in a week and became but a memory for everyone involved.

2 Likes
  1. Bad sync habits (as noted above).
  2. Bad sync provider defaults. This is a more recent issue with the fad of deleting local data after it is uploaded.
  3. Bad sync provider. (Full stop.) Mostly this is confined to Google Drive usage.

Otherwise it’s pretty difficult to really mess things up, at least in terms of common mistakes. A fourth problem that is somewhat of a tangent to syncing is editing the contents of the project directly with word processors. It isn’t often a fatal mistake, but it can lead to issues, much like opening the iTunes database in a database editor and messing with it directly might lead to issues, but people think differently of .rtf files.

1 Like

That certainly wasn’t my interpretation of rms’s comment, and if that was the case I think these forums would have a dramatically different culture and tone!

In my experience of tech systems generally, the leading causes of data loss on any system are (in this order):

  • user action (e.g., they’ve deleted or changed something they now want back)
  • lost device (e.g., stolen hard-drive or laptop)
  • damaged device (e.g., internal components such as hard drives or batteries fail)

EDIT:
I should probably also add — “Employee leaves the firm and it turns out they never saved anything to the server, ever.”

3 Likes

Absolutely not saying that. I’m saying that if something goes wrong with files in the sync folder (corruption, disk failure, user-deleting accidently), the flaw syncs and “poof”, backup gone. Why take that risk. Put the backups somewhere else.

That being said, if people want to use Dropbox sync to hold backups of anything, go ahead but don’t say I didn’t mention this risk.

Same as above. Sync folders, IMHO, are not sufficient robust to hold backups. Maybe Dropbox’s or macOS sync software can cause data loss, but that nothing to do with Scrivener. And if using Dropbox “online” feature, risk of Scrivener data file corruption is not low. That risk covered so many times here by so many people.

I use the following:

: Projects: ~/Dropbox/Projects/Scrivener (Dropbox settings for “offline”)
: Backups: ~/Backups/Scrivener

Both of the above are backed up by system backups (TimeMachine for entire system to USB and Synology NAS, Backblaze to get offsite backups to mitigate against theft/fire/etc, and Chronosync copying very important files (which includes Scrivener Projects) to the NAS in a backup folder.

In terms of mitigation of risk, I’ve had more laptop crashes and data loss due to hardware or theft than ever having my backups corrupted or unable to restore after the incident.

3 Likes

I was blown away by the quality and choice when I visited a few years ago!