My Scrivener journey went a little like this: First, I gathered all project-related materials into Scrivener’s Research folders. Afterward, the Scrivener project files grew significantly large, causing slow synchronisation speeds on Dropbox, even with a fast internet connection. Since Scrivener won’t let you separate the Research Folder and the manuscript, you’ll have to sync everything. I finally realised I could use aliases on Mac for my Research Folder documents rather than moving them into Scrivener. Stupid me! And I eventually stopped using Scrivener aliases, opting instead to organise research material in separate Mac folders! Scrivener would be more usable if its project files were divisible into smaller parts. A helpful improvement for Scrivener would be to save and sync Manuscript and Research folders separately. Projects in Scrivener become too large to be practical. It slows down backups and increase backup storage usage significantly.
But Scrivener projects are already divided into many, many files. A project appears to be just one file, but it is actually a package of files, a disguised folder so to speak. (At least on the Mac side of things; on Windows it’s a normal folder).
And this is for exactly the reason you describe: to prevent the saving and synchronising process from slowing down as the project grows.
I don’t know why it doesn’t work for you. But here is how it should work: Only manuscript and research items that have changed are synchronised, plus a bunch of tiny internal project files which reflect those changes in one way or another.
Zipped project backups on the other hand are actual files and have to be saved and uploaded in one piece. And the zipping itself will take more time when the project grows. Do you use zipped backups and have you set Scrivener to back up zipped projects before syncing?
Multiple projects can already be integrated with a good degree of seamlessness, as demonstrated in this post. And by design that is already the answer to your question of what to do if a project grows too large for your comfort zone (waiting for backups and so on). Maybe in the future we’ll make that a bit better (I can think of a few obvious improvements), but for the moment, I don’t really see how breaking large projects apart into smaller ones is any different than what you are proposing?
I do also agree that something seems a bit off with the technical problem you’re facing with sync. Unless you do just mean it takes a long time to upload the entire .zip at the end of the day, sure. But the project itself, if it is being synced, shouldn’t take more than a few seconds to close. Most of the changes you’d have made during the day would have been saved and synced a long time ago, seconds after you made them. What gets saved on close is a few housekeeping files, like the search index, some settings, the internal binder backups, and any trace content files that were modified seconds before closing.
Even the largest of those files, the search.index on average, is going to be a modest upload in the grand scheme of things.
Thank you for the project management example; I apologise for any confusion. Perhaps splitting the project into two—manuscript in one, research in another—would be beneficial. Now, when I finish or my project closes automatically after 30 minutes of inactivity, Scrivener saves a zipped backup of the entire project, including the Research Folder, even if just one word is changed in the manuscript. This can be inconvenient. I’ve configured Scrivener to store also the zipped backups on Dropbox, in a separate folder, which offers a 30-day change history. Some projects are nearly 1GB, so each time I close a project, a large backup zip file is uploaded to Dropbox.
Suavito, thank you for your comment. You’ll find my reply to AmberV below.
If you have so much research that it is taking more time than you are happy with to create and upload the backup, you might look into Devon Think or Tinderbox as your research repository. They both play well with Scrivener I believe.
Taking Research out of the project is to nullify a major reason for using Scrivener.
Mark
Having separate research projects does help with the backup problem. For one thing, you can go into Project Settings for an individual project, in its Backup tab, and switch it off entirely. That could be okay for stuff that rarely changes—just let your normal backup system handle it (Time Machine, offline HDs, etc.). Scrivener’s automatic backup is more an extended undo system for stuff that changes often. It’s not optimised for efficiently backing up multi-gigabyte, largely static repositories of data (for one thing I like keeping as many in the rotation as possible, the full 25, so that’s 25gb of nearly identical data at all times for no good reason).
And of course while you are there you will see you can also designate a special folder for such projects, instead. That way you could still have that extended undo, but use something outside of any sync service to avoid the long 1gb+ upload times. That doesn’t get around the 25gb problem, but if you have a lot of storage that may not be as much of one.
Aliases are still another solution as well, either instead of separate projects or even in conjunction. I perhaps missed the reason why you stopped using them. You mention that you stopped, and started only using the file system, but with aliases you get both the file system copies (which is indeed very nice) and the integrated access of having this data in your split views and metadata infrastructure, in the project.
AmberV, thank you very much for this useful discussion. I have learned a lot. I will change my backup settings and update them on a project-by-project basis. As you mentioned, the active Scrivener projects will be backed up via another software, so there is no need for duplicate backups. I abandoned alias policies in my projects because the referenced documents were only accessible on the computer where the aliases were created—making the use of multiple devices less useful. One benefit of my approach was the seamless updating of research documents stored in Mac folders; I didn’t have to contend with Scrivener’s project file management.
Wishing you a productive and enjoyable day at work!