Working off of network drives (MobileMe, thumb drives...)

Scrivener Scrivener for macOS
1

This concerns using any sort of technology which promises to keep local copies of your work synchronised between multiple computers over the Internet, such as MobileMe (iDisk), DropBox, WebDAV based technologies, and so forth.

Due to the way Scrivener works with a project file, and the manner in which most of these technologies keep themselves up to date, it is not recommended that you use these methods to work on active projects. Unlike many file formats, Scrivener projects are actually composed of many small files (sometimes this can range in the hundreds), which depending on how you work, are updated nearly continuously as you arrange, write, edit, and sort the material in your book. This level of complexity in combination with the number of network connexions required to keep everything up to date can dramatically increase the probability of suffering lost work, and corrupted projects.

It is possible to reduce the probability of this happening by adjusting how often Scrivener automatically saves your project, and as such if you choose to use Scrivener to edit projects that are located on these network services, you can probably reduce your risk by changing that setting to something closer to a minute or greater in length.

Despite, I still recommend that you keep your working projects on local hard drives (or external drives that are plugged in via USB or FireWire), and use these network services to store your backups. Scrivener makes it very easy to make backups, and you should be doing this as you work anyway. I highly recommend using the zip archive option so as to make sure that the many small files in the project do not get lost in the flurry of network activity that would otherwise result in saving a Scrivener project to a network drive.

To summarise: Use of Scrivener with network synchronised folders and mountable drives is not recommended and you do so at your own risk. The recommended use is to use Scrivenerā€™s back-up feature (found in the File menu) to produce zipped archives which can be safely saved to these drives and available to all of your computers.

Work local; back-up often; and develop habits for backing up your last copy of the day so that youā€™ll always have the latest version available to all of your networked computers.

2

I keep my files on the MacBook, but I do send the backups to MobileMe. Iā€™ve never had to go back to one, so I canā€™t say how well it works, but that (in addition to Time Machine) makes me feel better. Is it still worth doing?

BTW, Amber, thanks for the info. I appreciate your willingness to investigate technical things while the rest of us are just pounding our keyboards. And you manage to write about them in a way that makes it clear to all.

3

Thanks for this, Amber. I stored a couple of small projects on Dropbox without any trouble, but I wonā€™t do it again.

4

Youā€™re welcome!

And yes, it is absolutely worth doing. Time Machine can help you out with some problems, but what would happen if your house burned down, or you were in a flood and you lost all of your computer equipment to water damage? The stuff you had put on your MobileMe account will be around for you to retrieve it. I try to think of Time Machine as a system level ā€œundoā€. Yes, it is technically a back-up, but a back-up is there to protect your work no matter what happens, and if you have all of your back-ups hooked up to one power outlet, or even in the same building, it wonā€™t matter how many you have in the unfortunate scenarios. Since Time Machine is, by definition, something constantly plugged into your computer, it is only as safe as your computer is. But for all the little disasters that happen within the digital environment of your computer, it is an excellent technology!

5

Adobe products are the same way to some degree. The files are not packages but the way certain Adobe Applications save can cause problems if you are not working off of the local drive. Adobe even had a tech support knowledge sheet on the subject and showed that working on files from a network drive could cause loss of work or corruption.

Plus working off of a network drive can cause excess network traffic which can cause a higher chance of packet collisions on a busy network.

As a rule of thumb I always copy the file from a server/network drive. Work on it locally. Then save a copy back to the network drive.

6

But I love dropbox! Am I doing this wrong? Here is my procedure:

I turn on my eMac, open Scrivener, and start working on my latest chapter. When the alarm sounds, I hit ā€œsaveā€ on the Scrivener menu and close the program. Then I open my dropbox locally and get back into the chapter I was just working on. I see the blue arrows indicating a file update is in progress. When they turn to a green checkbox, I close Scrivener again.

I know I am saving first to my hard drive - thatā€™s just common sense, after all - and I have not yet tried to work on my project(s) using dropbox on a remote machine. Iā€™ve thought of it, but havenā€™t done it yet, and, after this warning, I guess I wonā€™t. But will I get into trouble using the procedure Iā€™ve described above? :confused:

7

Iā€™ve been using Dropbox to sync for a few weeks now with no problems. I work from the dropbox file on my local hard drive and donā€™t pay any attention to what Dropbox in the sky does with it. All I know is that when I switch computers, itā€™s always there. I also use Time Machine and Mozy. I believe that constitutes belt, suspenders AND parachute.

I like Amberā€™s description of TM as a drive-level undo. It fits.

I donā€™t doubt what sheā€™s saying. It took almost all day to do the first sync of my Scrivener file (Itā€™s large.) with Dropdox. I donā€™t know any of the tech things it was talking about. It said it was indexing ā€¦ downloading ā€¦ uploading or some such. Iā€™m guessing this delay was due in some way to the vagaries of Scrivenerā€™s file system. Since then, itā€™s worked without any problems. In fact, I donā€™t pay it any mind at all. I do my do; it does its do.

Dropbox is so convenient, Iā€™m going to keep on with it. If anything happens, I assume that TM will have a recent backup of anything Iā€™ve done. If the house burns down, I have Mozy ā€¦ although I probably wouldnā€™t have a computer to download it to. What I like about all these things is that I donā€™t have to give them one thought. They run on automatic.

Thanks for the heads up Amber. I hope Iā€™m not setting myself up for a fall by ignoring it.

Rebecca

8

Hi Rebecca,
I too have been working like you. In my experience it works well provided: 1) you make sure Dropbox is fully synchronised with the green tick-mark and wait a minute or so before doing anything like turning off your computer; 2) you donā€™t work with two computers synchronising with Dropbox, and do some editing on one of them when you are unable to be online.
You might like to read the discussion on this thread:

:slight_smile:
Mark

9

Markā€“

Backing old data over new data is an old headache. Back in the bad old days when I was using a pc with Windows 3.1 and backing things onto floppies, I did this several times. Made me want to toss the computer out the door and into the yard.

Iā€™ve been careful to just open up my laptop and let it ā€œtalkā€ to my iMac for awhile whenever I get home from working where there isnā€™t an internet. I think Iā€™m blessed that I only have two computers. More than that is too much for my delicate organizational skills. Iā€™m getting ready to start my busy time on my job. That means long hours and lots of stress; the sort of situation that manufactures dumb mistakes. My solution last year was to turn off the iMac on days when I was working such long hours and just use the laptop. Then, on my day off, I shut down the laptop and went to the iMac. I always used Chronosync to get them right with each other when I made the change-over. It worked without a single glitch.

I have an internet at work. So the two computers should sync with Dropbox. I think Iā€™ll still set the iMac to shut itself down on long days and then leave them for a while to work it out between them when I really get home (as opposed to just showing up to shower and sleep) for a day off. Itā€™s best if I do what I can to defend myself against myself when Iā€™m distracted and tired.

Oh me. The tangled webs we weave when we try to simplify.

Thanks for the input.

Rebecca

10

For the last few days Iā€™ve been trying MobileMe, using the iDisk sync as a way of keeping my two laptops in sync. Found that itā€™s even more dangerous than Dropbox. I just canā€™t get it to upload an updated file on one computer and then download the update onto the other. So Iā€™m abandoning it. Iā€™m shutting down iDisk sync and will only use my iDisk as an archiving system, logging into it manually and uploading stuff manually when I need.

Iā€™ll go back to using Dropbox for the synchronising job, but will take care to only put a back up of the Scriv file on Dropbox and will keep another back-up on a portable external. Itā€™s as near as I am going to get to Jaysen level belt-and-braces ā€¦ scriv file on MBP and MBA hard disks, synchronised through back-ups on Dropbox, with a back-up of the latest version from whichever laptop on the portable and the MBP backed up fully every night to an external FireWire drive. And when I finish with a project, Iā€™ll zip it up and archive it on iDisk.

Mark

11

Hi Markā€“

iDisk is hinky. Period. I put a couple of databases (big databases) on it back when I first switched to macs, and theyā€™ve been there ever since, archiving away. It will sync simple little files, and it does a good job with iCal and Address Book and Mail, which makes it worth the $ for me.

They only way I try to use it with Scrivener is with a zipped file. It burps on anything else. Even then, it takes a long time to back it up. It can be a real pain in that regard. However one time a few months ago, I tripped and fell into search strings hell with my Scrivener file. It was on my laptop, and I hadnā€™t plugged in my portable drive to let TM work its way. I would have lost quite a bit of work except hinky 'ole iDisk came to the rescue.

For my money, iDisk is mostly unrealized potential. Apple is so good with most of its products. I wonder why they canā€™t get it right with this one.

Rebecca

12

Has anyone ever tried using ā€˜symbolic linksā€™ to sync Scrivener via dropbox? Iā€™ve just discovered this technique for syncing address book, ical, and safari and it seems to work just fine. Iā€™m tempted to give it a tryā€¦

Info on symbolic links can be found here:
wiki.getdropbox.com/TipsAndTrick ā€¦ herFolders

and here:

dl.getdropbox.com/u/87620/Dropbo ā€¦ mlinks.pdf

Basically, you create a symbolic link (similar to an alias) of your data (eg ~/library/calendars) and put it in your dropbox folder. Then you do the same thing on your second computer (but follow the more detailed instructions on those links). Then somehow / automagically they seem to sync. You can add and delete ical entries on one computer and when you quit ical, they get synced. Then they appear / disappear in the ical app on the other computer. The trick is to be patient and let dropbox do its stuff before trying to open to synced data.

It could work for Scriv as well. Iā€™ll let you know if I have any success. Although the technical amongst you may be able to shed more light on thisā€¦

13

The same sync issues would still apply. All a symlink is, is a pointer to a real location. Just like an alias on the desktop. Basically drop box ā€œfollowsā€ symlinks as if they are real files which allows this to work.

I would not recommend this technic for the same reasons that Amber points out in her original post. Basically this changes nothing, it is just a different way to do the same thing.

14

Thanks Jaysen, good to get the confirmation. I did a little test and there were problems immediately, so youā€™re right, this wonā€™t work. Pity.

15

Iā€™m still worried about what problems, because Iā€™m using dropbox as the main backup for my novel(s) in progress. Iā€™m only using one computer for actual writing - all I have done on other computers, or on the website, is print out files for others to read. And I do save to my hard drive first. Am I going to have problems? Because so far, itā€™s been seamless, accurate and easy.

16

SImply use the backup feature (To create a zip) or Zip the file before copying over and no worries. BUT make sure all the files are shut down before zipping or copying (unless you use the backup feature to make a copy and only copy the backup and not the original.

17

For those of us who work a lot on the road (or work frequently between two different computers), it would be great if perhaps in the 2.0 release the saving structure was modified to work with these services. Iā€™ve corrupted a few files before I found this thread.

Several other well-regarded Mac apps have adapted their structure for such syncing (1Password, etc), and Iā€™d love to see this in place for my favorite writing app as well. :slight_smile:

18

Iā€™d be interested in how this could be achieved. The problem - at least as I understand it - doesnā€™t really lie with Scrivener so much as with how many of these services work. Scrivener projects are ā€œpackagesā€, and any Cocoa application could recognise them as such - as distinct from a folder - using NSFileManagerā€™s (the Cocoa file manager) -isFilePackageAtPath: method. So in theory, any of these programs could look at a Scrivener file, determine that it is a package, and thus update it in its entirety (except for those services that just provide a box to drop in such as, er, DropBox, of course). Iā€™m not sure why some of the Mac solutions donā€™t do this, to be honest.

But if these services arenā€™t offering a solution, how would Scrivener do it? How would the saving structure be modified? Surely the only solution would be for a .scriv project to be a single file (a zip package itself, perhaps). But in that situation, every time you saved, the whole project would have to be saved, not just the current document. For those with large projects with lots of research material in them, this would not be a fun experience, having to save 100MB or so every time. With the current set up, if you import a 100MB movie or sound file, once itā€™s imported, that file never needs saving again. With a single file, it would have to be rewritten every save. The current format is actually more secure on a local drive than a single file - a single file corrupted would lose everything; on a local drive, if the .scriv binder structure gets corrupted, you still have access to the underlying writing; you can extract it easily enough. It is only in these network and shared drive instances that it becomes less safe. But the price for making it safer is getting rid of the multiple file format and having longer save times, updating every file unnecessarily every time there is a saveā€¦

All the best,
Keith

19

KB,

Forgive my ignorance, but can you lock the file from the OS level. I use flock on solaris which can be flagged to prevent even read only access. I this would theoretically prevent any backups until the lock is removed.

On a different front is the idea of leveraging the fs journal. I would expect that to be implemented on the backup software side, but it ā€œcouldā€ be done by scriv. In this model you snapshot the package at the fs level and all calling progies see the static snaphost. Scriv makes changes to the real data and when done (or at regular intervals) it releases the new data. We use this method to backup the SAN (24TB).ā€ 

All that said, it might be more trouble than it is worth.

ā€ [size=75]Ok to really do this right you would need to use a dmg with the package on it. I think this would require quite a bit of effort. It might be solvable as a service but that would be outside scriv.

This had got to be one of the worst solution I have come up with recently. [/size]

20

OH! OH! OH!

New ideaā€¦ what about an ā€œauto archive changed files toā€ preference? Using rsync you could update a copy pretty efficiently. All you would need to do is freeze auto-save for the duration of the sync. The interval could be large, say 30 minutes.

Ok. Iā€™m done.