Ability to Sync for Collaborative Writing Projects

[mnott]

I have independently investigated a bit into Scrivener and Git. See over here: http://www.mnott.de/unscrivening/. I’m a Scrivener novice, and just had done some scripting around it to make it work faster and more flexibly with LaTeX (http://github.com/mnott/texdown). So I got to look into the data structures, and I think, the issues are really somehow these:

• Because Scrivener uses all files in one folder, and you can have naming conflicts in Scrivener (and it makes sense to allow for them), some scripting needs to be done to export the Scrivener (.scrivx) path into a Filesystem path, if you want to later get the content back. I think this can be done.
• Because Scrivener uses RTF, diff is easy, but merge is not, on file level. It is potentially possible to work around that, using html or similar as intermediate format for maintaining the formatting, plus plain text for being able to merge, but it is not quite simple. I think this can be done, and if allowing for some loss (which should be fine for LaTeX folks, who do mostly annotative writing), then that may even be a viable option.
• As others have mentioned, even if you solve merge, you’ll have an issue with creating - or concurrent creating - of files. There are actually unique IDs in .scrivx, but I’m not quite sure at this moment how they are calculated. This, at least, would allow us to get away from sequenced numbers.

Now, with that, while I’m a big fan of Git, I think there’s upside potential by doing some more scripting on Scrivener itself. For example:

• Essentially, add a management command line interface. Very much, my little TeXDown script that I started a couple days ago to have my own markdown for LaTeX, and a faster compilation of assets, has already somehow gone beyond its purpose and is doing some management stuff.
• Allow for taking snapshots of stuff from the command line - and for managing them. After all, merging stuff back in, and the “new document added” problems could potentially be solved if rather than adding stuff in all the time, we would just add new stuff in, but other than that, would just re-import stuff as explicitly named snapshots - and then use Scrivener UI to make sense of the changes. I think this can easily be done.
• Allow for exporting parts of .scriv into other files, again from the command line. And for re-importing them.

Essentially, all my focus is on scripting because I know if I’ve scripted it, it just works. If I’ve to remember which stuff to select in the binder to then take a manual snapshot of, I’ll make mistakes.

Fundamentally, Git can not only be used as a direct tool, but also from an API perspective. Just watch this video to see what the OmniFocus people are doing: https://realm.io/news/altconf-wil-shipley-git-document-format/

While I highly like that Scrivener has a very plain text format for most of its assets, I think following that more clever Git route that the Omni people do would add significant potential above and beyond what users like me can do.

[root]

@Hugh

@whiterabbit7844

ADDENDUM: Took a look at GitHUB. Based on the twenty minutes I spent running through their tutorial, I think this is exactly what we need. It should allow us to both work simultaneously and commit and pull changes from the master branch whenever we want to merge our changes together. Does that make sense? Doing this, should we not be able to work completely independently and concurrently without messing things up?

Hi Michael, a quick comment on the above before people with greater knowledge than I contribute. As far as I know from looking at this forum from a position of ignorance, no one has so far been able to make concurrent and independent changes to a Scrivener project work - even using GitHub. This may have to do with the thousands-of-files issue I drew attention to above, and also the fact that within a project a document’s meta-data may not be stored in the same file as the document. Nor does Scrivener currently have the capability to merge what would effectively be two projects together (although as Amber says that is planned to change - up to a point.) So you could end up with a mess! But I guess that if you succeed, the rest of us will definitely be interested… Folder Sync which Mr X referred to above is something else and worth exploring in the Scrivener Manual, but is a more limited endeavour.

Michael, I feel with you since four years is a long time to go without having your questions answered. And, slash or, I hope you have your workflow resolved by now! But if you had to leave Scrivener in order to get collaboration working - or are on some crutches and gaffer tape solution - I just wanted to give you a shout and tell you that you were on to something even if others thought you were not. Also, things have of course evolved since 2016.

The idea, that keeps floating up on this board, that collaboration is only possible if you’re Google, Apple or Microsoft is a very strange one. It was first mentioned by one in the support and has then been spread by others with no apparent software development knowledge. It’s especially funny, since the Google Docs live collab was added by using the open sourced Etherpad engine. Which is the same the small team behind Hackpad used to launch their app. Which was bought by Dropbox and became Dropbox Paper. So just because Google and Dropbox are ginormous, it doesn’t follow that every feature in their stack is developed through their muscles. Actually, Hackpad was a much smaller operation than Scrivener is. (Also, anyone who has worked for a corporation knows that each department and initiative can be smaller and more scrappily funded than many independent operations.)

Anyhow, in lieu of proper collaboration in Scrivener, here’s a workflow that works really well for a bunch of collaborators. And don’t even have to touch the command line!

1. Download GitHub Desktop. (And create a GitHub account if you haven’t got one.)

2. In Scrivener, do the ‘Sync with External Folder’. Set the format to multimarkdown - .md. Also, synch on closing and check on opening. (As a bonus, sync to a folder in your Dropbox or Google Drive for extra backup.)

3. In GitHub Desktop, create a new repository and choose the folder you just synced to. Make sure to choose the ‘Scrivener’ option in the Gitignore drop down.

4. Publish your new repo to GitHub.

5. Invite collaborators to your repo on GitHub and they mimick your setup.

Now, before writing in Scrivener and also when done, you’ll fire up GitHub Desktop (or have it on always) and it will tell you when there are changes to ‘sync’, slash merge.

Added benefits with this workflow are that you and your collaborators can edit files on GitHub in the browser as well… (Hey there, board topic on Scrivener web version! ) Or in a text editor like Atom that has built in GitHub integration. Wait a minute, there’s a thought! With the project on GitHub and collaborators writing in Atom or similar, we have a total collaboration system. Like everyone in software development already has!

And if you find GitHub’s interface less than inspiring for writing, you could set up one more integration. GitBook is a very nice web based writing environment that combines collaboration with version control. With a left panel that has the ouline of what your doing, which you can re-arrange freely. Yes, it’s very much like an online Scrivener! But it also has API:s and webhooks that let you sync both ways with GitHub. So you could even collaborate with people who don’t have Scrivener, but find GitHub a bit too geeky to work in.

[wordchiseler]

Are you still using Syncthing? Do you have issues using it with Scrivener?
I’m just wanting to go between iOS and mac.

@tg2k

Your search.indices, ui.ini, and maybe other files should be ignored.

Which would be the maybe others?