Syncing !

Scrivener Wish List
1

One functionality would really make my day : syncing .scriv packages through Internet to another computer running Scrivener !

The usage scenario I have in mind goes like this : while working within a Scrivener project, I hit a “Sync this project” button. Scrivener then goes out on the web, to a user-defined WebDAV directory, an Amazon S3 bucket, a MobileMe account, whatever the user chosed, and pushes the .scriv archive (updating it if it exists an earlier version). Then, back home when I need to work on that project, I fire up Scrivener, hit a global “Sync” button and am offered to update any or all .scriv documents present on the share. I select the one I want to work on, a local copy is either created (if none exists) or upadated (if an earlier copy was present), and I start working. Once done, I “Sync” it again, the server version is updated, and tomorrow at work I’ll be able to get back to work where I left it.

I reckon syncing is not a trivial task ! It opens the door to a whole gamut of new problems (network problems, security problems, integrity problems…) but if done right that seems to be a real bonus to Scrivener.

One of my other favorite OSX app (Omnifocus) took time to offer a Sync option, but it really made a difference when we were able to use it from different computers !

And happy new year to you all !

2

Have you tried Dropbox, J75? It works beautifully with Scrivener. Many people here use it just to sync backups, but I’ve used it with live Scrivener package files (though, admittedly, never my most important project) and never had a problem.

getdropbox.com

3

Ah yes, I forgot about Dropbox (I even learnt about it in those forum earlier !). Yes, it’s probably a good workaround, and more generic, too. But yet, I feel a built-in sync in Scrivener would have its purpose.

4

Hi, I use Dropbox. I started by just saving my current Scriv projects to the Dropbox folder, which means that every time Scriv saves, the file is also automatically uploaded to my web-space. Both my computers sync with it, so whatever I do on one gets updated on the other when I start it up. In several months of working like that, I have only once had a problem when I had been working in a coffee shop, had just exited and their network dropped while Dropbox was in the middle of syncing the project. I returned home and started up the computer at home and it promptly had a syncing problem and I lost a bit of work – not very much, thank heavens.

So I started saving the files to the appropriate folder in Documents on the computer I’m working on, and then using Scriv “Back-up to” to put a copy into Dropbox; but with another app which doesn’t have that, I’m ending up confused as to where the latest version is! So I’m thinking of going back to my original method and just keeping a closer eye on the syncing process for some of my projects. For a major one I only get to work on once a week, it’ll be the “Back-up to” method.

Dropbox is great, and does exactly what you want.

Mark

Edit: By the way, I also have a Mobile-me account, but even with the iDisk duplicated on each computer’s desktop, Dropbox is way faster and more convenient. Less prone to network problems too …

5

Scribbler™ is an iPhone novel-writing app I’m working on. Ok, an early look at Scrivener sync with iPhone…very early…some new screens as well.

visualnewt.com/Products/scribbler/

Comments and suggestions welcome.

6

Hello all,

for a few days now I play around with the all new Voodoo Pad 4.0. I wish a syncing feature like that one owns. Totally rocking and for over two weeks and heavy working on different machines no problem. It integrates seemlessly in the workflow.

Take a look and you will be surprised how simpel sync could be.

Please, Keith, build it in… :wink:

7

I’m not sure Gus would agree with that!

8

Why not? Because I called it simpel or because the people “only” take a look and do not buy?

Ok, it’s hard work to make it work in such a smooth way, so come on you all, buy yourself a copy of VoodooPad.

Better now? :smiley:

9

The part in red is where you went a little off the mark. The use may be simple. Fine. Building it … Not simple. That was what AmberV was commenting on.

Voodoo Pad has frequently been acknowledged by folks here as a good app. It serves a different purpose than scrivener. There are features in scrivener that voodoo pad just can’t touch. If that means that we don’t get sync to a phone native in scirv, who cares. We will let Wizlord help us with that.

Add to this that Keith has already indicated is position on implementing sync native in scrivener and your statement might be considered a little out of order.

BTW for someone who is accusing folks of “look and do not buy” why do you want to undercut Wizlord and voodoo pad by asking KB to implement this feature in scriv? I will probably buy what ever Wizlord builds. If KB does it I wont. Not really the way I think you meant your comment, but certainly the way it sounds.

10

Ah, ok, I see. I’ve been a little misunderstood, I think it’s my “very german english”. Sorry for the confusion.

What I ment is, syncing documents in VoodoPad to MobileMe is that easy, I wish I could have something similar in Scrivener. Something, that doesn’t make me think about it. That isn’t simpel, I know. That was never the point for me. As you mentioned, Gus and his software are great!

If Keith has been stated about syncing earlier, I don’t read it. And if Keith won’t build it in, it’s ok for me, because this is the wishlist and not the programm code in here… 8)

But nevertheless, Keith, feel a bit pushed to syncing…

And again, sorry for the confusion on my postings.

11

No problem.

Keith started looking into an iPhone app to do this, but decided that it wasn’t worth the effort at this time. Enter Wizlord and his idea. Keith may not need to build his app as Wizlord is focusing on phone ↔ mac and phone side writing. Since scriv can easily import and export file for Wizlord’s app it seems to me that we are all set.

It is a shame that Apple decided to keep the iPhone storage systems closed to owners. That sure would make some things a whole lot nicer.

12

Well synching seems like a simple thing to do, but it’s very difficult to do right (which is one reason not all apps support sync). Not only do you have to sync in both directions, but you have to build all of the communications code to handle the transfers. Comm code in its own right, is not simple. Add the 3rd variable that the iPhone is a new platform and there is a learning curve involved in that as well.

Conceptually, synching is easy. In practice there are lots of variables that will bite you if not done properly. What happens when you delete something from a document, for example? Did you delete it or add it on the other side? You not only have to keep track of what changed, but when it changed. What happens if you change the same line in both apps? Yes you can ask the user which one to keep, but is that the correct solution? Then there is the UI to worry about. And if you’re doing it within Scrivener itself there is all the legacy code that may have been built not to support synching at all (I don’t know if that is the case or not since I don’t work with or for L&L…I just use Scrivener.)

You then have to worry about how the Scrivener project is put together. The project is made up of multiple files, so you’re not synching a single file, you need to sync multiple files in multiple places. How does that map to what someone does on the iPhone?

Another variable is that Scrivener supports something that amounts to RTF within the project. The iPhone has no built-in RTF support, so you have to roll-your-own or find a library to do it for you.

Lot of variables and situations to juggle before anything that resembles a good sync works.

And finally, if it were that simple you would have built it by now :wink: Its only simple because people don’t understand all that’s involved.

I’m trying to teach myself a lot of things at the same time (iPhone dev being the main thing.) I’m working on figuring out how to get sync to work without a lot of pain. I’ll have something, but probably not for the first release. You can see in my screenshots I’ve gotten Bonjour on the iphone to discover all the machines on my network. That’s the easy part. The sync is the hard part since there is nothing on the desktop/laptop that lets you sync Scrivener – so I have to design/build that part as well.

So unless you can prove something is simple by actually doing it yourself, you’re just voicing an opinion.

Cheers, and thanks to all who have shown an interest in what I’m doing with Scribbler™ My problem is I have more ideas than time/knowledge at this point to build. But I’m making progress and will definitely put something out. Tomorrow I’ll actually register my app with Apple so that I can test and debug it on an actual iPhone.

13

And just for the record, it took Gus three tries to get sync working, and Voodoo Pad is now up to version four. It’s excellent software (I own a pro licence myself) with a very talented programmer behind it, but simply looking at the dialogue box and saying it must be simple because anyone with a MobileMe account can set it up is a far cry from actually developing the framework behind that dialogue box.

Scrivener projects are an interesting problem in that it is not only a bunch of files, but a bunch of files that are integrated together by a central index, and each of those files can be “dimensionally” represented by snapshots. Simply copying the “latest” version of every internal component would be project corruption heaven. There would also be a problem of “uniqueness” in the current project version, I believe. I don’t think there is any sort of internal unique project ID setting apart any other project from another. Further, the component files are added sequentially and start over in each project—so the component IDs are not unique as well.

In other words, if you have your Scrivener Server set up and you are working on a project named “Novel” on one computer, but an entirely different project named “Novel” on another computer, how would the server know that these two “Novel.scriv” projects are not the same novel at all? How would it know that Novel.scriv/21.rtfd should not under any circumstances overwrite Novel.scriv/21.rtfd in the second novel? VoodooPad has the benefit of each document having its own unique ID number, and then each page within each document has its own ID as well. So if anything changes they can be pushed to the server without fear of accidentally overwriting data from another document entirely.

And yes, you also have the RTF problem itself which Wizlord went over. Scrivener uses a hacked version of Apple’s RTF engine which allows it to use annotations, footnotes, and so forth. So even if a standard RTF comparison library existed for Cocoa, it probably wouldn’t work out of the box. One of the reasons why we don’t have the ability to compare differences between snapshots is due precisely to this problem. If it is difficult to even show potential differences to the user in a colour-coded manner, you can imagine how much more difficult it is to ensure 100% accuracy in automating all of that.

14

Dropbox has been mentioned several times. I’ve been using it on a lot of “packages” (most of the Omni products) without a single problem. However, I’ve shied away from using it with Scrivener due to the apparent complexity of the Scrivener package.

Dropbox is such a simple and elegant solution to what everyone has been discussing that I’m wondering if it might behoove the Scrivener team to officially check it out and turn thumbs up or down.

15

To judge from postings, several, possibly many, members of this forum use Dropbox (including maybe Keith himself), and use it successfully with Scrivener. It works brilliantly for me. I even transfer my files to my PC, although I’ve always taken the precaution of using Dropbox to hold or copy over zipped backups only, not the basic saved Scrivener packages, and of allowing a few minutes after each backing-up for Dropbox to sync before closing down the computer.

Of course synching could be simpler and more direct and have more bells and whistles than Dropbox provides, but other than for Wizlord’s project (and even there downloading or uploading and importing would be fine for me for the time being), when the wheel is round and it rolls, why re-invent it?

H

16

In the past, I’d always been careful to either ZIP or put packages in a disk image prior to transferring to another computer, etc. However, I find that Dropbox seems to do just fine with the raw package. Plus, since it only deals with what’s actually changed, it’s extremely fast–almost instantaneous with anything other than photos, video, etc. So Zipping something prior to placing it in Dropbox seems to detract a bit from the seamlessness of the whole Dropbox idea (if you’re using it for synching and not merely for simple file transfer). I’m guessing it takes a few minutes to synch for you due to the file being Zipped.

I agree with synching backups only, but would be interested in some feedback from Amy or Keith on the need to ZIP.

17

There’ve been a number of threads that have touched on this subject e.g: [url]Recommendations for external hard drive? - #24 by Jaysen]. I don’t pretend to have the expertise to understand all the issues, but I can see that Scrivener’s saving routine could be a challenge for Db.

BTW, she’s Amber…

H

18

As a Dropbox user who has worked with his main project actually stored in the Dropbox folder, therefore synching automatically between the MBP and the MBA, I’ve come to the conclusion that the danger is when you have both machines running at the same time, so they are both trying to sync with the project on the Dropbox host.

Through that I have lost a number of very short paragraphs from one project – translations for editing – but as they are now, literally, old news and I have the exported versions and the original versions that were sent to me elsewhere, I’m not worried, and anyway, I’ve started a new .scriv project for 2009. But it has alerted me to a potential problem.

I also have a .me account and I’m trying that, but boy! is it slow in comparison. Still, I’ll use that for some data files which don’t get updated so frequently.

Mark

19

I don’t think you would want your scriv file in the drop box folder unless you set the auto save to a higher value. Other wise you may wind up with a lot of queued changes and potentially get some overlap. I think if you set the auto save to something in the range of 5 minutes it would be safer.

Just a thought.

20

While I am not an official voice for Scrivener (in Amy-form or not), I’ll add my recommendation to the fray to stay away from the temptation of working off of your DropBox folder with unzipped Scrivener projects. I tried this for a while, and ran into problems at least three times. Once it was my fault—I opened the project twice without thinking—but the other two times were probably something more along the lines of what Jaysen cautions against. With its default settings, Scrivener can generate a hail of disk activity, which when working in DropBox generates a hail of network activity. Somehow, some bits of the project transferred incorrectly and I had to painstakingly correct the project file-by-file using the web interface version controls.

So, theoretically, it should all work—especially if you drop the auto-save rate down to something more reasonable. But if you go this route, do keep frequent backups. Scrivener makes this easy with Cmd-Shift-S. Keep in mind, nearly every single forum post regarding project corruption or lost work eventually boiled down to the usage of Flash media or network sync services like iDisk and DropBox. The remaining few were most likely due to hardware error or system crashes. I’ve used Scrivener for years now, and have never seen a corrupted project or missing data until I started messing with automated network syncing. It’s not an instant recipe for disaster, but the statistics seem to indicate an increased risk; if you are fine with that—then go right ahead. It is awfully convenient. This is true for any complex data package that does frequent writes; it isn’t necessarily a Scrivener problem.