Sharing / Collaborating

Scrivener Wish List
1

Parts of this idea were brought up in an earlier thread called “Version comparaisons / collaborative working” (sic), but there was no official response, and my idea is ever so slightly different.

I’d like to be able to collaborate with my co-author from a project that lives in a central, remote location (FTP directory, MobileMe, etc.). When I open this project, Scrivener will show me what has changed since my last session, using a flag of some kind, or perhaps a color indicator. (Bonus: Scrivener would have a “safe” mode that could bar me from editing a chapter that my collaborator was actively working on.)

That’s basically it. I know that I can already back up to a remote drive if I so choose… but it’s not really the same as a Scrivener that’s truly multi-user aware.

(Plus, then I can get my friend to buy Scrivener instead of forcing me to dump everything out to Google Docs. Eugh.)

2

If you search on the word “collaborating” you’ll find over 40 posts. We’ve discussed this topic often.

My writing partner works downstairs, so our method of collaborating is simple. I write the first drafts, then she revises, and then we pass it back and forth, revising constantly, until we’re ready to export to Pages. (You are apparently constantly revising each other’s work; we do it in stages.)

So we just make regular backups and always choose to work with the file that has the most recent revision date. If I need to see how she changed my golden prose, I look at the version prior to hers.

Once we export to Pages or Word, we do track changes. But we don’t feel that’s needed in Scrivener, since its early work is so subject to constant revision.

3

I’ll look for the prior discussion about collaboration — I’m sorry I missed it as I scanned down the earlier discussions.

For now, I’ll take it that my needs (/wishes) don’t reflect those of the broader community. That’s a nice workflow you’ve devised, though I’m not sure it will work in my particular situation. It’d be nice to have all these features in a single application. Thanks for responding!

4

Dear me. The more I read previous posts on this topic, the more convinced I am that I was right to bring this up afresh. All this talk of zipped files to FTP sites, using Subversion, Google Docs, or collaborators agreeing to work serially? Those aren’t solutions so much as they’re a recipe for insanity. (As an interface designer, I do have a penchant for elegance and simplicity.)

So I think I’ll keep my call for an integrated sharing mechanism on the books, so to speak. The discussion may end here — these things are harder to develop than they are to request, to be sure — but at least I’ve cast my vote.

Cheers!

5

In the next version, collaboration will be much aided by some new tools which have all been announced already. Two of which will probably be of interest to you. The first is that snapshots will now reveal changes. You’ll be able to load up a document, view its prior milestones and compares the differences between them.

Secondly, there will be a sort of “revision marker” that lets you consistently type in a certain colour which matches a revision level. Author A could revise a rough draft with a revision 1 marker, then Author B could review these markings and add changes or notes in a revision 2 marker, and so on. It will be possible to jump through a project by marker type.

Live, multi-user editing of a project will remain impossible. But it would be possible to implement a protocol between collaborators even now, using colour labels or one of the others as a convention. Granted, Scrivener wouldn’t prohibit editing, but so long as everyone played by the rules, this should work.

I hear you on Google Docs. I like some things about it, like two people being able to work on a single document at once, but otherwise its a mess, especially when you have something as nice as Scrivener to compare it to. Synchronous editing is a pretty complicated thing, though, especially over the Internet, because of lag. Have you considered SubEthaEdit? I think that only works if the two individuals are on the same LAN, but if that is the case, it might be you find that better than Google Docs.

Yes, definitely harder to develop than request. :slight_smile:

6

Your post actually gives me hope — those solutions seem workable and reasonable. I will be a day-one upgrader.

(And apologies again for any redundancies to the forum in my initial post. But may this thread help clarify things to others with the same questions.)

7

Yes, this comes up a lot, but as you note, it’s something that is much more difficult to develop than to request. It’s on the list of long-term “would-be-nice” items, but so far I have not come up with a workable solution. For instance, what happens if in your open project you create a document in a folder the other user has deleted? Scrivener would need to keep track of changes to the underlying binder data and keep refreshing the binder every time one is made (which, given the auto-save, could be frequent). And in that case it is possible for a document you are working on to disappear completely. Even if the other user is locked out of editing a document you are editing, what about if he or she trashes the parent folder of that item and then empties the trash? In that case either the other user is refused the deletion or your document ends up in some “restored” folder.

Also, you could edit a document, click on another one, then try to return to the previous document and get locked out because the other user has now clicked on it - it could be a frustrating experience in itself.

How would you do this with a Word document? The fact is, you wouldn’t - you would do it serially. If both of you had the same Word document open at the same time, any changes you made could be lost if the other person saved his or her file after you. This problem thus isn’t unique to Scrivener, it just seems that the expectation of some users is a little different when it comes to Scriv because of the multiple-document setup within a project. But the problems - from an implementation standpoint - are much the same (more complicated, in fact, because of all of the extra structural data).

I’m sure there is a solution down the line, so I’m more than willing to listen to suggestions for how it might work, but I make a point of not implementing anything until I know the solution is right and won’t cause more problems than it was supposed to solve.

All the best,
Keith

8

These problems remind me of what you run into when multiple people are developing code using revision control. If you ever have the time, you should consider learning how to use Subversion, or the more advanced Git. While there is no solution to the issue of when one user deletes a file that the other is simultaneously editing or renaming, the idea is that you both have to communicate the fact that you plan to get rid of said file, or that you’re editing the same portion of that file. Subversion (which I’m most familiar with, though Git is like Subversion++ from what I understand) would simply create a conflict that one of the users would have to resolve before continuing.

I suggest that you look into these tools if you ever get serious about this feature, because they provide all of the functionality that you need (and that is possible) for collaborative work over a network, and they have solved all sorts of other issues that you may not have worked out for yourself. Plus, if you can integrate them into scrivener seamlessly, you can even get rid of most of your snapshot code, since they’ll take care of keeping up with the differences between files too. Well, I think they would at least… code usually has short “paragraphs” compared to prose.

9

Here’s another wacky idea: integrating some kind of Bonjour-like networking, so that changes to a file show up real-time in the other person’s editor (assuming they’re editing/viewing the same part of the same file). I think there’s a code editor that does this… yes, it’s called SubEthaEdit.

codingmonkeys.de/subethaedit/index.html

10

Hi,

Thanks for your thoughts. Yes, I’m aware of SubEthaEdit. It’s all rather a massive undertaking, and not something I relish (or foresee any time soon). There are a number of other issues involved given that Scrivener deals with rich text prose rather than plain text code (as you note).

I doubt it would be possible just to integrate Git or Subversion in any meaningful way. While to the user it may seem simple to have edits in one text show up real-time in the editor of another user, it is actually rather complicated. Each user is really working with a different file in memory, so the only way to update it is to look for changes to the underlying file and then update the editor if it is showing that document.

All the best,
Keith

11

As for real-time vs. all-at-once updating, I think the latter may be simpler and more desirable. Paraphrasing / synthesizing earlier posts: writer 1 would open a chapter located on a remote server, and work on that piece in local memory. The only thing writer 2 would see is some kind of flag that the chapter was open elsewhere, and would either be prevented from opening that chapter at all, or would only be able to open it in “browse” mode (a concept I remember from my mainframe days). Similarly, chapters that writer 1 had changed recently would be marked for writer 2 to see, either at the chapter level (to keep things simple), or, on opening the chapter, at a more granular level.

As for deletion conflicts, the act of deleting could be seen more as a request — delete pending, or some such — than an immediate action. If writer 1 moves to delete a chapter (or a directory), then writer 2 sees a warning flag, and can either sign off on it (thus allowing the deletion to complete), or can dismiss it. Thus, shared projects might only make destructive edits on consensus.

Speaking generally, as someone who works frequently with developers in a production environment, we do often start by asking for the moon. But then we scale back, looking for ways we might gracefully degrade our dream functionality while keeping the essence of the idea intact. The mechanisms above can be very complex… but in this case I think the bare minimum would be a passive flag/color denoting “writer 1 edited this item,” and the rest would rely on spoken agreement between collaborators. It’s not as foolproof as some of the above examples, but I think that captures the essence of the idea.

(I’ll be on the 2.0 train regardless, and my book project will be finished whether or not my partner can pull herself from the mire of Google Docs!)

12

Scamper, have you looked into Google Wave? It might be the ticket for you and your collaborator. It combines the features of Google Mail, Docs, and Talk. You can gather and store links, share notes or drafts with a collaborator, and discuss or make revisions online, in real time. You need invitations; I got mine by saying I wanted to test it for writing collaboration. See wave.google.com

13

I’ve been on Google Wave, and kicked the tires. Personally, I find the UI to be a nightmare of nested threads and counter-intuitive controls. (My friend works at Google, and I’ve given him an earful. :wink:

Wave isn’t really a writing tool (in fact, I’m not sure anyone really knows what need it fulfills!), but I do appreciate your suggestions. Meantime, here’s hoping that our writing tool of choice continues to meet our needs in a most elegant way.

14

Very new here, so please forgive me if I repeat things that have already been thought through…

Collaboration is crucial, in my view. There is always the point when you finally wish or need to show a draft version of your work to somebody else, and I don’t think that this necessarily conincides with moving from Scrivener to a different WP for the “final stages”.

Here is an idea for basic collaboration involving roundtrips from Scrivener to RTF and back, including Scrivener to Scrivener and back (this is going to be rather long and technical, I am afraid):

Why not include an UUID with each text document in the exported, compiled RTF draft – for example as as hidden text, bookmark, or some sort of scrivener-link. The UUID could be associated with the Scrivener generated title for the item and, e.g. feature as an option in Scrivener’s “compile draft” dialog. The UUIDs would define snippets/sections in the exported RTF document corresponding to the text documents in Scrivener. A snippet would start with the UUID-marked title and end with the occurence of the next UUID (or the end of the document). An exported document with the “include UUIDs” option activated would therefore mandate that Titles are included for all items - folders, text containers, and texts.

The UUID would offer a way to check such compiled RTF drafts back in after they have been commented, edited, etc.
Checking back in would include:

  1. splitting the RTF again into the original text documents according to the snippets defined by the UUIDs
  2. archiving the existing version of each corresponding Scrivener document as a snapshot, and
  3. adding the imported snippet as the current version of the document

Re-import could fail gracefully in four steps:

  • If Scrivener finds matching UUIDs in the imported document AND the sequence of the UUIDs in the RTF matches a corresponding sequence in Scrivener, then offer to import the snippets as described above. (Let’s forget about hierarchy here; a matching sequence is enough. See below)
  • if Scrivener finds matching UUIDs in the imported RTF but NO matching sequence, then offer a) to change the sequence according to the RTF, b) import the text documents but leave sequence (and hierarchy) in Scrivener untouched, or c) import as a whole RTF (as it is done now)
  • If Scrivener finds UUIDs in the RTF but no matching ones, offer to import as multiple snippets split accordingly while maintaining the UUIDs (important for collaboration, see below)
  • and finally, if the imported RTF has no identifiers, just add the whole document as it is done now.

This would make the following two collaboration scenarios possible:
Scenario 1: Editing/proof-reading/commenting: export as draft in RTF. The editor works on this document, possibly, no: probably using MS Word and track changes. After you get the document back, either process the changes in your own copy of MS Word and then import the document back into Scrivener, or: thinking Scrivener 2.x and tracked changes between snapshots, just re-import and handle within Scrivener.
Scenario 2: collaborative work between two writers using Scrivener. Writer A starts and sends a draft to Writer B as compiled RTF; Writer B imports and now has a set of Scrivener documents with UUIDs matching the ones in A’s scrivener bundle. The two writers can now send compiled drafts back and forth and all the single text documents contained by the exchanged drafts will continue to create versions on both sides.

Scenario 2 quickly raises a new question: And what if writer B changes the hierarchy/sequence in his version of the draft? How to deal with that? - just ignore it, I would say for two reasons:
First, it will never be possible to “read” a definite hierarchy of sections/snipptets from an RTF document, unless you follow very strict conventions. This would contradict Scrivener’s philosophy of flexibility.
And second, Scrivener is not made to handle/manage a history of a document’s structure (the sequence and hieararchy of the documents in the draft folder), as far as I understand. Keith could, of course, add a “save current structure” command, but that would introduce a bunch of new problems. What, for example, if one would choose to go back to an earlier structure that includes a text document that no longer extists? The added complexity would interfere with Scrivener’s simple, beautiful elegance.
However, the fact that full-scale colaboration would mandate a solution to managing hierarchies and sequences is no reason not to think about more basic collaboration features. If managed hierarchies and sequences are left out of the picture, developing a simple workflow for collaboration becomes indefinitely easier.

15

Every single possible feature in the multiverse is “crucial” for someone, believe me. :slight_smile: (Just take a browse through the rest of the Wish List forum.) Collaboration may be important for some writers, but for writers working individually - which I admit is the paradigm I had in mind when creating Scrivener - this isn’t a fundamental feature. The ability to have all the documents in one place for a first draft and then compile them for sending to an editor or for publication works for most use-cases, and for the vast majority of Scrivener’s published authors. I’m not saying that in the long run I don’t want to provide support for those who do collaborate, but I am saying that, regrettably, it’s not a high priority right now and that, as it is a lot more complicated to implement than most users realise, if it ever does get included then it will only be added when I feel it can be done right. Things like this take time to “brew”, but also I have to be enthusiastic about coding such a big feature either because it’s something I want myself, or because I believe it is something that belongs in Scrivener and the coding challenge it presents interests me. At the moment, I have no time for coding challenges, as the list of things that genuinely has to be done before Scrivener 2.0 can be released is too long to allow for them. So, although I’m happy to hear suggestions and will discuss the if I get the time, bear in mind that I won’t be able to take on board any suggestions regarding collaboration for at least six months or more.

I had already considered the “UUID” idea you suggest, and in fact I started implementing it but dropped it for now (although I may resume work on it post-2.0). It’s pretty ugly, to be honest. Scrivener uses internal IDs to represent documents, and for this approach to work these numbers would need to be incorporated into the exported RTF, e.g:

[ID:198] Title
Text blah blah

[ID:245] Another Title
Text etc etc

The problems here aren’t trivial. You can’t “hide” the ID numbers in the RTF, as I placed them in custom RTF syntax any external RTF editor would wipe them as soon as someone else edited and saved the file. If someone deletes an ID, then the text could end up getting applied to the wrong document when reimported, ignored completely, or created as a new document. A snapshot would need taking of old text, of course, although that’s not a problem. Scrivener would have to try to update titles, but again the end-user could have done all sorts of things to mess this up. But the biggest problem is the one you bring up, the fact that it wouldn’t be possible to restructure the draft to match a reordered document of this type, because there is no way to convert a flat list of documents into a hierarchical one (without flattening it entirely).

Moreover, although this sort of solution would be a nice solution for those who wanted to take their work to a different platform or to send it to a user without Scrivener, it doesn’t solve the issue of editing a single Scrivener file on multiple machines at the same time, which is what the op requested. I’d also argue that it might be more productive just to compile the draft and have the editor make coloured notes and then return it to you so you could incorporate them into the draft manually for this sort of thing, as you would need to evaluate each comment anyway. (Because the export-and-reimport solution would work better for having work edited rather than collaboration.)

As I say, collaboration features definitely aren’t coming with the first version of 2.0. Still, I’m happy to hear all suggestions for possible future consideration. I may not have the time to discuss each one individually, but I do promise to read them all and consider how they would work.

All the best,
Keith

16

Dear Keith,

That makes things much more complicated and ugly, indeed. Especially with UUIDs instead of simple IDs. I was thinking of something like Sente’s embedded \field tags you can create during an RTF scan, but I wasn’t aware that these were wiped outside MS Word.
The options are then: a) include IDs as text, which is possible but ugly, b) make it MS Word specific, which would trigger a flow of support requests from everybody working one anything else, and c) use the titles themselves as “primary keys” – but I am sure you’ve been there before…

I can fully understand that non of these options is particularly tempting.
Thank you for your thorough reply.

Martin

Martin

17

Just thoughts… I don’t expect you to consider them or even answer. But I have to write them down in order to get them out of my head.

You could, however, define the titles as hyperlinks and include the IDs as hrefs, possibly as something like this: scrivener://mybundle/myID
Apple’s rich text engine preserves them, as far as I can see.

As a safety-net you could make the inclusion of scene-breaks mandatory in an “export for collaboration”-mode. The scene breaks, e.g. “#”, could include hyperlinks in the same way: scrivener://mybundle/myID#END

Still true, of course. You could always break it. But titles as hyperlinks would be editable and easy to parse on your end.

Maybe a separate option besides the regular import, something like “Import and update if matches found” could temper expectations: it would not import anything unless a section marked by start and end hyperlinks/tags matches an existing document in Scrivener.

I just moved the entire draft of my PhD dissertation from Pages to Scrivener - much better to jump around, revise, think further, etc. I already feel at home in this little marvel you created :slight_smile:

Martin

18

Subversion gets into a confused tangle with Scrivener, due to the multiple files / rtfd-package format that Subversion insists on populating with hidden .svn directories. Git works fine.

Speaking of which … does anyone know of a good RTF-aware diff-tool for OSX?

19

It depends on what you want it for, but Scrivener will be able to analyse the text content of snapshots for differences. However it won’t produce a diff file that can be later used with other tools, it is rather a visual tool to see what has changed, nor will it analyse style changes. In most cases, that would be the desired behaviour anyway.

20

That sounds very useful.

Can it be pointed at 2 versions of the document (e.g. that I extract from a git repo) that were not scrivener-snapshots?

Will it also indicate when an included non-text file (e.g. a pdf image) has changed? And any chance the binder will include some indication of changed vs. unchanged parts?

Thanks!