This one’s probably already on your radar, and probably also a big distance off. In my former life I programmed games for Electronic Arts, so I know a little bit about the stumbling blocks involved to support this request–atomic transactions between multiple users and all that.
But, it would be absolutely tremendous if Scrivener could support live syncing in a multi-writer environment, the way Evernote et al, does. I am currently working on a project with another writer and we have to carefully make sure we’re both working on different aspects of the project and then lay out exact details about what to combine every time we get to a point where we want to consolidate our work. This would be particularly useful with the cork board and outlining, where each of us can have a different collection that we adjust / sort / flesh out while putting together the basic story.
Anyway, thought I’d mention it. I haven’t looked through previous posts, but I’m betting this one comes up a lot.
You’re right - it does come up a lot: Search gives 26 pages of posts on the topic of “Collaboration”. Essentially - as I understand it - developing software with “simultaneous” real-time collaboration - if that’s what you’re asking for - is really only a possibility for a developer with the resources of a Google or a Microsoft. For Scrivener it would be even more complicated than it is for those developers, because a single Scrivener project can contain potentially thousands of individual files, all connected and related.
Non-real-time collaboration - serial collaboration, if you like - is possible with Scrivener, if due precautions are observed. It sounds like you’re already making this work. (Here’s a page from the Knowledge Base with advice if you’re using cloud-sync services for this purpose.)
As someone who does work collaboratively, and across platforms, with due care and attention to letting the various computers sync fully before shutting down or before opening the project, it does work using Cubby (in our case) or Dropbox. But we are only two people involved and have a 7-8 hour time-gap between us, so hardly ever are we in a position to work on a project at the same time, which is a no-no.
If you are many, and especially if you—and any of your collaborators—are on Macs, then I’d look rather to using “folder sync” with some kind of GitHub repository, and one of you designated “Chief Editor” responsible for sync’ing back with Scrivener. But have a look at:
and other threads. Mind, you, I think what the OP on that thread is proposing is very complex and risks corrupting the project too. But it’s your decision, and however you do it, my suggestion is snapshot frequently and back up frequently.
Thank you, Hugh. And yeah, that was pretty much the response I expected. I’ve even considered something like CVS, but that seems like such a huge complex project to undertake for something like this. In reality, we rarely ever work on the same data, but it’s still a major pain trying to keep track of what’s changed and what hadn’t so that both of us have the same environment once we “sync,” and by sync, I mean manually updating everything we tracked since the last update… Sigh. And then pray we didn’t forget to track something
Hi Mr. X (I feel like James Bond, talking to the villain)
We too are just two, and there is quite a time change for us as well. I am on the west coast of Canada and he is in Brazil. But I tend to work sixteen hour days, so there is always some overlap. As for snapshotting and backing up–backups are cranked to 25 and I do manual offsite backups often.
Thank you very much for responding to my message. I will check out Cubby and GitHUB. You’re the second person to actually suggest that to me. Frankly, I have no idea what it is–but I will investigate.
That type of sync, where one takes two sources and has a computer attempt to intelligently merge them together into a single product, is something we’ll have limited support for in the future. It’s not going to be super fancy, but if you save off a copy of your project, zip it up and send it to your editor, then get an annotated copy of the project back, in theory you’ll be able to import that copy into your main working project and it will do its best to merge the two binders together and any changed content.
As for twenty people “logging in” to a central server with multi-cursor document editing—eh no, and we’re not working on self-driving cars either. I think for many people something like the above will be more than enough for what they need though.
CVS style systems can work with Scrivener, but I would suggest as a more robust and private form of what services like Dropbox provide. You won’t be deviating from what you get with Dropbox, in a purely pragmatic sense: a cohesive present-tense version of the project kept in a central location. CVS technology all by itself is not going to handle such things as “merge a copy of this project Frodo has been working on independently for eight months with the repository that Bilbo has been writing to”. That’s mainly because Scrivener’s project format is not self-describing from the file unit level, but a group of files described by external meta-data (a hyperlink from one text pointing to another, the master Binder addressing the content files, etc.).
So with cubby… you sync your cubby folder right before you open the project and then sync it again after you save at the end of the day? CAN you work on things simultaneously this way? Is cubby smart enough to mesh the changes? I downloaded it, but the features explanation is a wee bit on the vague side.
Never mind. From what I can tell, Cubby is just a cloud drive, no? Like Google Drive or OneDrive? So if I make changes and sync to Cubby after my collaborater has just made changes and synced, my files are simply going to overwrite his changes, aren’t they? That isn’t the solution I’m looking for. If I want to really do this, I am going to need some sort of concurrent version control. Is that what GitHUB is? Checking that out now…
ADDENDUM: Took a look at GitHUB. Based on the twenty minutes I spent running through their tutorial, I think this is exactly what we need. It should allow us to both work simultaneously and commit and pull changes from the master branch whenever we want to merge our changes together. Does that make sense? Doing this, should we not be able to work completely independently and concurrently without messing things up?
So basically our entire Master branch (and all sub branches) will simply be the .scriv file for the project.
From your first post it sounded as if you were many more than two involved. As you’re only two, don’t bother with GitHub or whatever … just use Dropbox or Cubby and with care you’ll be alright. As I said in
Cubby reports through Growl every time my collaborator’s work is saved to the server, so if I’m on my computer, every few minutes an alert pops up saying a file in the project has been changed, and when she logs out the user.lock file appears in my trash, so I always know when she’s working on it.
If you do try to open it when your collaborator is working on it, you’ll get an alert saying that the project is open on another computer and giving you the option to work on a copy. I’d resist that like the plague as you will be forking the project and marrying the two versions up would be a pain.
No, neither Cubby nor Dropbox mesh any changes. You can only work sequentially as I say above. That is going to be true whatever way you try to collaborate … unless you are willing to do the work and take the risks with GitHub as @glowkeeper in
I WANT to be able to work concurrently, though. And, after yet another day of not writing (my editor and agent are going to have hissy fits soon), I figured out GitHUB and it will do EXACTLY what I want. It’s great and it’s not a huge ordeal like CVS used to be back when I programmed. It works very logically and well, and I think if we pull/commit to the master branch on even a weekly basis, it will keep things all up to date for us. Of course I will also be making manual backups snapshotting the work along the way on top of the cyclic backups Scrivener already does on its own.
I think we’re all good. Thank you very much for your help. Now I better go pound out a dozen pages or so before I have to write off the day. At this point, it’s not going to be Hemingway, but sometimes you just have to settle for Dan Brown
Cheers!
Michael
PS: One completely unrelated thing. You seem to have the same problem with your avatar on here that I do. How the heck do we upload an image to the Avatar thingy in the profile thingy? (sorry for the over-technical jargon there )
Hi Michael, a quick comment on the above before people with greater knowledge than I contribute. As far as I know from looking at this forum from a position of ignorance, no one has so far been able to make concurrent and independent changes to a Scrivener project work - even using GitHub. This may have to do with the thousands-of-files issue I drew attention to above, and also the fact that within a project a document’s meta-data may not be stored in the same file as the document. Nor does Scrivener currently have the capability to merge what would effectively be two projects together (although as Amber says that is planned to change - up to a point.) So you could end up with a mess! But I guess that if you succeed, the rest of us will definitely be interested… Folder Sync which Mr X referred to above is something else and worth exploring in the Scrivener Manual, but is a more limited endeavour.
So far, it IS working, but I haven’t tried it with anything complicated yet. Do you know the issues people previously had using GitHUB? In my mind, it sort of HAS to work–that’s the whole point with GitHUB, the ability to check out a new branch from the master project branch and then, when you’re ready, recombine your branch back into the deployment branch.
Now you have me wondering.
I’ve pointed all the different directories Scrivener allows me to set up so that they are inside my Master GitHUB branch. I guess I’ll find out soon enough I will just make sure I backup before pulling anything from the master branch and vice versa for a while until it we can get a feel for whether it’s working or it isn’t.
The project is actually on hold right now (my partner has other contracts he needs to fulfill) so that is why I am researching all this now. So, probably won’t have a definitive answer for sometime.
Thank you for the warning, though. Don’t worry, I will definitely take it to heart.
Now I have a novel to finish and a proposal to write so no more testing out new technology for me until those to things are on my agent’s desk
The main problem you may find yourself running into when merging branches is where internal addressing is concerned. The main .scrivx file and a few others as well, refer to items internally by a number. When you create a new document in your Binder, the next number in the sequence is assigned to it. Thus if two people check out the same project and both create a new item in the Binder, according to the independent program logic on both machines being unaware of the state of the other machine, both copies of the project will use the same next available integer to address the item. Author A has an 88.rtf file that is some scratch notes written over cocktails, and Author B has an 88.rtf file that represents the start of a new chapter.
Now as the Git person tasked with managing this mess, you find yourself with three .scrivx files, the original and the two forks, none of which are now correct. The old .scrivx doesn’t have any reference to item #88, the new chapter is an orphan file according it—and of course each of the .scrivx files being merged are incorrect as well, since both claim both of the two new items are #88. Your job then is to rename one of the new files yourself to 89.rtf (and 89_notes.rtf, 89_synopsis.txt and 89.comments, depending on how much meta-data and annotation has been done) then go back and merge the #88 section from the .scrivx that describes the now #89 item, fix its ID number and fix any snapshot filenames. That’s just one single file, imagine the fun if both authors add or delete several dozen files.
At that point I wouldn’t even bother with trying to manually merge these two together. I would just open the individual projects and do the merge myself… let the software handle which goes to #89 and so on.
That doesn’t work too well. I’ve seen it fail and catastrophically too using my personal notebook and updating the same note from three different devices: Mac with the OS X application, and iPhone and iPad with the iOS app. The note looks okay until the race condition hits and then the user is left to resolve the multiple conflicts between the notes. All the assistance one gets from the software is demarkations as to when the notes were modified.
This Evernote problem is almost trivial to provoke. Ride the Tube making edits to notes while deep in the tunnels from an iPad. Make changes to the same note later (on the return journey—because the iPad’s battery is flat) from an iPhone in those same tunnels. Wait until all three devices “sync” to Evernote and it looks like all hell has broken out. The note is most definitely broken! So much for “live” syncing.
I would personally steer clear of using source control systems with something like Scrivener. With Git (which is what you use on Github) you will need to do merges frequently.
My personal recommendation is Syncthing. Syncthing is a free open source peer-to-peer folder synchronization solution.
If you use it, here are some tips and tricks that may help:
You can make it ignore files. Your search.indices, ui.ini, and maybe other files should be ignored.
You can configure a refresh interval. I would use something like a minute, if you are simultaneously editing, so it brings over changes quickly.
I don’t know whether Scrivener detects edits on disk and replaces files already opened with the newly changed ones. You’d have to experiment to see. If not then you’d need simultaneous communication to say what’s been edited.
When adding documents or creating notes (which adds files), you’d probably need to close and reopen Scrivener on the other side, because I don’t think it will offer to re-load changes made in the Binder.
Scrivener will warn you that the project is already open, and you’ll need to blow past this warning, or else make Syncthing ignore the user.lock file as well.
If and when you get conflicts, it won’t be necessarily obvious. What will happen is that you’ll see files with an extension like “.conflict” that indicate the device name that it belongs to. You can open the pair of files with a utility like WinMerge and edit them to resolve. Then delete the conflict files. You may also have to use Syncthing’s web interface to make it force the changes over to the other side.
You can direct Syncthing to keep staggered file backups. I recommend you take extreme advantage of this and set it to something like 10 copies per file.
I would also use a tool like FreeFileSync to do regular backups daily.
FWIW, I use Syncthing to synchronize across computers, but it’s only ever me doing the editing,
As far as handling the project merging manually–totally skipping GitHUB and all the rest, just going back to the simple case of sending your partner a copy of the master project, each of you working on separate copies, then merging those later (by opening both and dragging and dropping from the copy into the master) to make a new master and send out a new copy–there are a few things you could do to simplify the merging:
Set a special default status for new documents, then clear that as part of merging and making the new copy. So each time your partner creates a new item, its default status is “NEW!” Then when it comes time to merge, you can easily search for or sort by status to ensure all his new items are copied into the master. (You could even save a search collection for this.) Before you make a new copy, set all those “NEW!” items to some other appropriate status. You can do something similar by using the author’s name for the new status, if you want to do the same on yours and need to keep your new and his new items distinct initially after copying his into the master.
(If you don’t like using the status or label for this, you could set up a document template that has a keyword or custom meta-data with the same info, then make that the default new document type for all root folders in the binder. It won’t catch any cases of making a new item at root level, or in any new folders made at root, but that’s probably a small percentage to double-check.)
Don’t empty the trash except when merging and creating a new copy. This lets you see easily whether items need to be deleted from the master, whether something was accidentally trashed and should be restored (or left in place in the master), and–combined with the above–you can see what of the trash is just scrap notes that your co-author created for temporary use and then tossed. Then you can clean all this up as part of making the new master, so it’s fresh for the next stint.
If you’re focussing on just writing/editing text in specific documents, so that each of you has an “assignment” for the week or such, you could create collections of these documents when splitting the master and copy, so that each person can just load up her collection and work there. It doesn’t prevent you from editing anywhere else, of course, but it’s a helpful way to keep track. Keywords could be used for this too. Or in reverse, if you can train yourselves to always tag an edited document with your name, it will be easy to check edited documents when it’s time to merge back to the master. Sorting by modified date in the outliner can help with that too. Then just clear the keywords/set new ones when you make the new copy, mark the date of the last merge, and you’re ready for the next session.
You may already be doing all that sort of thing anyway, but I figured I’d share.
Hi, I’m a software engineer and use GitHub all the time for code. I initially started using my programing tool, webstorm, to write my novel but it was not very conducive for writing, Hence I am evaluating scrivener. I hope its files format can be diffed by a git repo so I can track my own changes.
Git is not just useful for collab, its great for single authors as its diffing abilities are great, they can help you remember what you were writing the last time.
If a bunch of authors wants some tips for using github, then I’m sure I could arrange a group hangout sometime.
This is great news! For Win/Scrivener?
How does one sort the documents in the outliner by modified (and created?) date?
Does the sorting only handle the documents in a single container? Or range over the entire project … or something in between?
(It would be nice as well to be able to Ctrl-G/S on the date metadata as well … and to do it > and <.)
Outliner sorting is currently only in the Mac version, I’m afraid; I was replying to the OP, who I gathered was using Mac. Sorry for not being clear about that! The outliner sorting will be coming to Windows in a future version but this step you’d need to skip for now. Using keywords to tag your documents and then searching for those would work on either platform.
(The way sorting works on Mac, it’s by container, but collections can be sorted like containers, so you can create a search collection of all the documents with text in the project (or any other collection) and then sort that. Since collections are always flat lists, that sorts the entire project without respect to hierarchy. It’s not currently possible to run searches or such with dates, but this is something on our list as a future possibility.)
It seems there are a lot of people here who believe Scrivener could never play nicely with softwares like git, and while I agree it could be better, I think people here aren’t giving it justice. I’ve been using git for collaborative writing on Scrivener for years now, although admittedly I don’t merge all that often.
But I think Scrivener could be made to work with git quite nicely, and thereby allow for code versioning and collaberative work without having to lift a finger. Well… a few fingers, but it would only need a few changes.
The problem of creating new files on separate machines and having a conflict with the filename could be resolved in a few ways. If instead of assigning a sequential number, it could use a small GUID. It would be a little uglier internally, but it would work. Another way to work it might be to “salt” filenames based on the user who created them. So instead of “89.rtf”, it might be “afletcher_89.rtf”. I don’t think that way is as good as GUIDs since you’d have to deal with potential name conflicts or add advanced configuration options to customize the salt prefix, but, you know, just a possibility.
The other problem is the way the .scrivx files contain both model and view information. If those were separated, so that one file was the “model” file, and contained information about the project hierarchy and filenames, and the other file was the “view” file, containing information about the last selected text and such, then git or svn users could easily set the “view” file to be ignored. That way, collaborative writers don’t have to deal with conflicts about who was looking at what file and such.
The last real problem are the modified dates specified so much in the .scrivx file. Those are guaranteed to conflict every time if they’re all in the .scrivx file, but if they were stored in an ignoreable file, or Scrivener relied upon file system modified dates, then no problem.
There’s also the potential issue of how Scrivener seems to cache a lot of info, such as a search index, or checksums. Those have never been a problem in my git versioning stuff because they’re all ignored. Word Counts get wonky sometimes, but I have a script that blasts those files and forces Scrivener to recreate them. It would be nice if Scrivener recognized when its cached files were expired, but not a requirement for anyone determined to use third-party versioning software.
And I think that would be it. I believe Scrivener could be used on GitHub collaboratively with little to no effort. And it would be super awesome if it did. It already works just fine for personal projects.
