OK - Help me out with this scenario… I appreciate this might be a silly situation, but based on the quoted comment above, it needs to be asked.
Phase 1
All three files are in sync, i.e., iMac, iPad and Dropbox server.
Phase 2
You are out somewhere with iPad and update the file substantially, but do not sync (no connection)
Phase 3
You are back home, and turn on the iMac. Both iMac and Dropbox server are in sync (keep in mind, that you did not sync you iPad with Dropbox server). You proceed to make changes on the iMac and sync with Dropbox server.
===========
With the above 3 phases, you now have a situation, where you are back on the iPad and want to update the Dropbox server with the earlier updated iPad files that was not sync’d.
Question: How does the iPad / Dropbox sync address the version on the Dropbox that was changed prior via the iMac, but not on the iPad before it got changed, too. Please tell me that there’s some ‘magic’, that merges the two distinct versions ! Least a warning flash that you are about to lose something.
Working with Dropbox is like working with a remote hard drive. If you have an iMac and an iPad you then have two more hard drives.
Maybe it’s easier if you don’t use the word “sync”? Think of it as Save… instead. And there is no magic.
From what I understand of how Dropbox works, your scenario would then read like this:
Phase 1 - all three hard drives have the same copy of the file.
Phase 2 - the file on your iPad is changed.
Phase 3 - you change the file on your iMac and the Dropbox app then copies that file to the Dropbox hard drive (in the background, automatically, without you doing anything). The iMac and the Dropbox hard drives both have the same version of the file.
Back on the iPad you tell it to Save (“sync”) the iPad version of the file to the Dropbox hard drive, which it does, which means that it overwrites the old version so that the iPad and Dropbox now have identical files on their hard drives. When you go back to the iMac, the Dropbox app will automatically detect that the Dropbox file has changed, and will download it to your iMac, meaning all of your previous changes on the iMac are gone.
Neither machine can possibly know what you think, only what you do. Dropbox is just like an intermediate hard drive, whre files are stored so you can easily copy them from one computer to another. So there is no magic merging.
Just don’t try to edit a project on either the Mac or iOS device without synchronising it first, or you will get a conflict where the Mac doesn’t know the changes on the iOS device, and the iOS device doesn’t know about the changes on the Mac. But the new versions implement a way that you can resolve the resulting conflict … neither machine can do that for you.
I’ve split this into its own thread so that it doesn’t get in the way of the other user’s technical problems.
It’s this sort of problem that gave me many months of headaches - and it’s exactly the sort of reason I chose the Dropbox API and manual sync in the end.
In this situation, one of two things would happen when you next sync (either on iOS or on the Mac/Windows):
You’ll receive a message saying that were conflicts were detected and successfully resolved.
Or:
You’ll receive a message saying that conflicts were detected and some conflicted copies of documents have been placed in a “Conflicts” folder at the bottom of the binder. You would then need to look in this folder and determine whether you want to keep the copies of documents there against the ones that are in the binder, or delete them. (This folder will only contain copies of documents that were edited on both devices in an un-synced state, so there should never be many to go through when this does happen, unless you edit on two devices without syncing for days on end.)
A nice thing about Dropbox is that the desktop client automatically creates conflicted copies of files if it detects that the same file has been edited in two places. Scrivener can then look for such files and add them to the project, finding what document they belong to, and tell the user about the conflicts.
But more importantly - and part of the reason 2.8 is necessary for working with iOS projects - I have updated the way the file format works for iOS. When you edit a project on iOS, it does not directly change the files used by the desktop version. Instead, it “checks out” copies of the files into an extra folder inside the .scriv package. When you open the project on the Mac or Windows, these checked-out files get merged back into the main project. This means that a scenario such as you describe does not result in data loss. Scrivener has access to both copies of the files and can compare them. There are various algorithms involved in this, but basically Scrivener does its best to resolve all conflicts, and won’t create conflicted copies of files where one file is unambiguously newer than the other. But it errs on the side of caution: if there is a chance that you have made different edits to a single file, it will create a conflicted file so that you can recover anything you want.
Of course, none of this is ever a problem if you are careful about syncing, which just involves:
Always waiting for the Dropbox client on the Mac or Windows to finish updating files before continuing to work on a project.
Always hitting “sync” in the iOS version when you know there are pending changes from the desktop version (the iOS version will tell you when it detects changes and prompt you to sync) and when you have made changes and are ready to return to the desktop (projects needing sync will show an indicator so that you know you need to tap sync when you are ready). There is an option in iOS to have it sync automatically whenever you close a project, and if you try to open a project that the iOS version thinks has changes pending from Dropbox, it will warn you and prompt you to sync.
Ultimately, you won’t lose any work, but you might have to deal with some conflicts here and there if you don’t get into good sync habits. My philosophy is that I would rather have users complain that our sync isn’t automatic and “magic” and that we force them to get into good sync habits rather than complain that our sync has lost their data.
Keith, I’ll tell you right now…there are gonna a lot of whiners if users have to manually do anything. We’re in an age of automatism and people don’t like to manually do anything, no matter how much you officiate that it’s ‘good for you’ or that you should develop some kind of discipline. I’m not saying I won’t; I’m just giving a heads up that there are gonna be a lot of whiners to this.
In an ideal world, you should be able to work on Scrivener iOS, not have to save in any fancy way, and when you close the app on your device it should automatically save whatever you are working on so that when you open the same file on your desktop you can work on it. The same should go the other way. That is, you work, you close (and not manually save) and open the updated file on iOS. This I hope is the eventual goal…otherwise, the plebians will revolt I say, the plebians will revolt…
People will always complain about something. And sadly, we don’t live in an ideal world. If you would like to consider all the factors in our sync processes and have a better solution, I’m all ears… But trust me, I have worked months on the sync code and considered it from eery angle. Possible if we were Google with whole teams and dedicated servers and APIs, we could make everything seem like “magic”. Sigh.
Keith knows this a lot better than I do, but from my experience of iOS devices I say that’s impossible. You never close iOS apps, you just start using another app, so you leave the Scrivener app for a few seconds or for weeks. How is the app to know? To make it work the way you want it to the Scrivener app would have to save with short intervals, roughly like Mac Scrivener does, but with the difference that the iOS app would have to save to Dropbox all the time, which in turn require a constant WiFi connection.
“But it works for Pages, Drafts, Evernote, etc??”
Yes, sometimes, and only for fairly simple things. iCloud works fairly well for some Apple apps, but only if you work with one file at a time, and only if they are reasonably small. It’s the same with Drafts. Often it works, but sometimes even those small snippets of text screw up. Imagine the reaction you would get if the software automatically screw up and destroyed your 150 000 word novel? It’s actually much better that people do it themselves.
If all computers – laptops, desktops, iOS, Androids – always had WiFi connection, they could all be interconnected all the time and all machines would know what all the other machines were doing. Then it would easily be solved, except for the major security problems.
lunk - yes, basically what we need for flawless sync in file formats such as Scrivener is an internet based on quantum entanglement or spooky action at a distance so that every upload and download is instantaneous, and all computers and devices all connected all the time. And I’m not being facetious - that would be the only way to avoid all problems.
You evaluate the problem well. iCloud does a very good job at syncing “simple” file formats such as TextEdit, Pages, Scapple and such. (By “simple”, I really just mean flat - a single file.) But there just is no “magic” solution for syncing a file format such as Scrivener’s, where you have a package of interdependent files. Of course I’d love it if I could just make it so that all the files synced magically and invisibly and just appeared on your other devices - “just like that” as Paul Daniels used to say. Of course that would be the “eventual goal” - once the technology is there so that it is possible.
The problem lies in the chasm between the expectations raised in users by Apple’s “It just works” mantra and the technical realities of syncing a complex file format such as Scrivener’s without data loss. It’s like the old “fast, good or cheap - pick two” conundrum. I can give users “magic” background sync via Dropbox as long as they don’t mind data loss and projects being in unusable states when a connection drops.
Don’t worry Keith. It sounds like you did a great job thinking about pro and con and you decided that the option that could result in data loss always had to give in to the other option.
Thanks for that.
And having the option of having mobile Scrivener sync automatically on close of the project seems like a nice automated option.
Like in Spider-Man: with great power comes great responsibility. You have the freedom of working on Scrivener projects everywhere now. Having the discipline to think about the sync’ing is a a normal expectation.
And it seems like the app gives you LOTS of safety nets for when you forget once.
Please note, I am not on the side of the Plebians; even though I am no programmer, I understand your frustrations, and realize the issue is not simple. But no matter how much I might understand, there will be many who will not. And when it comes to users potentially losing their data over the consequences of your decision, Keith, Rome might fall.
Again, I am no programmer, nor do I pose as one. I am simply the soothsayer, in this scenario, ever so admonishing…beware the ides of march of iOS/MacOS synching…
I’m curious – if you’re not arguing a position you yourself don’t believe in, why ARE you making such a big fuss about this? If you’re trying to be funny, it’s not coming off that way on this side of the screen.
Scrivener for iOS is an amazing piece of engineering. The workflow is about as simple (and reliable) as it can get. Keith’s done a stellar job of making a very complex problem very usable and safe.
I’m not trying to be funny. It’s a legitimate concern; one that I personally will not be bothered by, but I imagine many will. I see no need to insult.
I will be the judge of this when the software comes out.
I never thought about it like this, but you are absolutely correct. People want spell checkers, software checking grammar, software saving the work when the author forgets to, and wasn’t there a link recently about an app that could actually write the body text as well? So you could start the computer in the morning, send the finished novel to a self publishing company by noon, and have the book printed by night fall.
But come to think of it, wouldn’t it be even easier to skip the computer and the writing-publishing-printing phase, and simply go down to the nearest book store and buy a ready made book? Or if you don’t want to manually do even that, you could have it sent home, or download it so you can listen to it instead of manually flipping the pages.
But thinking a bit more, I think you are dead wrong. People like to do a lot of things manually. And if people can remember to charge their iPad or phone, and to bring it along when they leave the house, surely they can remember to sync their manuscript to Dropbox before they shut down the computer?
Having to remember to do things isn’t necessarily a bad thing.
Discipline, yes is a virtue. But the Devil must have his due, I’m afraid. We don’t all drive stick for a reason: our laziness supersedes any desire to save gas (which, from what I’m told, when driving stick, tends to happen).
Writing should be about writing. Not saving. That’s something a computer should figure out for itself. Or at least that is what the masses will think or want. Or think that is what they want.
People like thinking. Just not thinking too much. And that’s what I’m afraid what Keith’s choice will lead to.
Oh no, not in my book. Cats have no discipline and they seem to be more happy than most humans.
I agree, writing should be about writing.
“Hey, you, where do you think you’re going with that iPad? What…? Oh no you don’t. Sit down and keep on writing!” grumble “I don’t understand all this about moving around, going elsewhere to write and all that. Writing should be about writing, and to do that you sit down and sit still.”
What? Are we suddenly discussing politics?
It reminds me of an old saying in Sweden when you describe people: “Oh, he thinks a lot, but not very fast.”
Since the copy of your project on Dropbox is a normal .scriv file, presumably it has the usual facility for backups and snapshots? I routinely back up my WiP after a major session, just to be on the safe side - and it would seem to me that this is sensible with any multi-device setup. As the recent meme says, “there is no cloud - it’s just a hard drive on someone else’s computer”.
The most work I’ve ever lost was not in Scrivener but in Notebooks before they introduced “proper” syncing that handles conflicts - and that was maybe 600 words. Fortunately I was able to reconstruct the scene from memory
Yep, you can take snapshots and make backups in the Mac version, and you can export or send zipped backups of the project in iOS. It’s the same project on both devices, but you won’t see snapshots in the iOS version as that feature is not supported.
