Enable iCloud Drive sync using Shortcuts' personal automations

[emilyst]

Hi, all,

If you’re like me, you’d like to use iCloud Drive to sync between mobile devices and your desktop devices. As you know, Scrivener on mobile devices cannot use iCloud Drive, meaning that you cannot use it for sync.

Today I realized there’s a workaround. Using Shortcuts (available on all mobile devices since iOS 13), I created personal automations which trigger whenever the Scrivener app is opened and when it is closed. These automations copy the files back and forth in the background, ensuring they match.

So far, the experience seems pretty seamless. I’ll explain in detail below.

(The Shortcuts app can very useful, but using it is a somewhat advanced topic. I’ve written the advice below for people who have some familiarity with Shortcuts. If you feel uncomfortable attempting to follow it, consider skipping this post until a time when you’re more comfortable with your device.)

You’ll need two automations. First, set up one which is triggered by opening the Scrivener app. Then add the actions below.

• Get contents of folder Writing (do not choose recursive)
• Repeat with each item in Contents of Folder
  • If File Extension is scriv (do not use a leading dot!)
    • Save Repeat Item to Scrivener (expand this action and choose to overwrite existing files)
  • End If
• End Repeat
• Nothing (to avoid notifications having the files attached)

I used the folder “Writing,” which is in my iCloud Drive, but you can use any folder you want. The “Scrivener” folder is the one used by the Scrivener app in “On My iPad” (or iPhone). The result should look like this screenshot:

We need a matching automation for when Scrivener is closed to copy the files back. Create it, and add the following actions:

• Get contents of folder Scrivener (do not choose recursive)
• Repeat with each item in Contents of Folder
  • If File Extension is scriv (do not use a leading dot!)
    • Save Repeat Item to Writing (expand this action and choose to overwrite existing files)
  • End If
• End Repeat
• Nothing (to avoid notifications having the files attached)

This is the same process as before, with the same settings, using the same folders as before, but the origin and destination folders have been swapped. Yours should look like this screenshot:

With those automations added, you should be done, and you can test it. If you’re having an issue, consider these notes:

• Choosing to get the contents of a folder recursively will mean that the .scriv bundles will not be seen as items and therefore will not be copied. This means that this process only works on items in the top-level folder you choose, not its subfolders.
• If you’re talented at using Shortcuts’ automations, you may wish to add a small bit of conditional logic that will check for a file at the destination with the same name and a newer modification date before copying. This would prevent situations where the automation might overwrite a file. Otherwise, it’s assumed that:
  • the contents of the “Writing” folder take precedence on opening Scrivener,
  • the contents of the “Scrivener” folder take precedence on closing Scrivener, and
  • no situation has occurred in which an automation was skipped.
• It’s possible to add notifications or alerts to various places in the actions. This can help with diagnosing a problem or just having a reminder occur that this background process is taking place. You can also have the automation ask permission before running at all.

One other thing to consider is that if you attempt to open a .scriv project file directly from Files, it will create a duplicate due to the copying process. You should try to remember to open Scrivener directly to find your projects.

It’s possible to add automations to trigger at certain times a day as well, if you’d like. I have one that triggers just before sunrise and performs both of these synchronization steps.

I think that’s about it! Let me know if you have any questions or if anything’s unclear. I’ll update this topic if I discover something about this process that I can make better.

[emilyst]

Sorry to bump my own topic, but it occurred to me last night that it would be possible to use this sort of automation to sync Scrivener’s projects between its own directory and any other destination available in Files, such as OneDrive, Google Drive, Synology Drive, or a remote server over SMB/FTP/WebDAV/etc.

If anybody tests this and finds that it works, please let me know!

[narrsd]

Hmm. This scheme is cleverness in scripting, but I keep having a warning sign flashing about it.

The problem is that by circumventing Scrivener’s quite reliable own syncing method, I think you are likely also to be circumventing what makes this fully work.

Scrivener monitors and looks for the kinds of errors which can occur with any cloud syncing when it goes a little wrong, or because of user mistakes that are quite easy to make.

Thus it finds, usually remedies automatically, and in any case carefully preserves original files, for any ‘collisions’ which may occur, when updates on files get out of order as the main example.

It would be very fortunate if it were to do, or be able to do, this kind of checking, if you don’t let Scrivener handle the syncing itself, so I’d be pretty wary of this sort of thing.

It might look like it worked – but then you could get a surprise…

p.s. just realized it was you who wrote this, Emily, and I’m sure then that you understand the issue, when you think about it. Why it’s good to bounce designs around a bit, sometimes I think (other times, not so much )

[emilyst]

Oh, no doubt, it’s a hack, and it carries risks and shortcomings. It’s not a perfect solution at all. I am just trying to avoid using Dropbox if at all possible.

In fact, one major shortcoming that has become apparent to me since trying out this automation is that iCloud Drive can have an out-of-date idea of your files. This can happen if you haven’t opened Files in a while (since iOS resists running tasks in the background). A possible workaround for this is to tell the automation to open the Files app before running the rest of its actions, but this has its own problems—it counts as “closing” the Scrivener app, which then triggers the app-closing automation.

Thanks for tempering expectations for others.

[narrsd]

Good on you – made me smile

[derick]

@emilyst

In fact, one major shortcoming that has become apparent

just reading this now and I was excited to try it out… it seems like it should work…bummed to read about these issues

[emilyst]

@derick

just reading this now and I was excited to try it out… it seems like it should work…bummed to read about these issues

Yeah, my thought is I could make each automation a shortcut which I can trigger on demand instead. (Kinda annoying but workable: Open Files, let it do its thing, then smash the Shortcut, then finally use Scrivener.)

But I haven’t tried that yet.

[narrsd]

@derick, well, it’s nobody’s fault – these issues are the name of the game in any synchronization – they are the recognized technical challenges.

Scrivener’s solution with Dropbox is excellent, and sufficient Dropbox space for writings almost anyone would do, is free.

[kewms]

Readers of this thread should be aware that using these Shortcuts is unsupported and entirely at your own risk.

It’s always a good idea to backup your iOS data regularly, and even more so when conducting experiments like this. Our guide to backups for iOS Scrivener can be found here:
https://scrivener.tenderapp.com/help/kb/ios/guide-to-keeping-ios-projects-backed-up

[anxiousdog]

Thanks for posting this! I thought about doing the same thing considering how many issues I have syncing with Dropbox. I’m on a deadline and had to leave my computer (PC) at home, and work in the car on a long trip. I did a backup and sync before I left, but when I opened my iPad, my previous work wasn’t there. I realized my dropbox was full and since my computer at home was off, I had a major setback.

I have about 5TB of space on OneDrive, so I’m wondering if I can use it instead… Have you had any issues so far? As you said, remembering to download the files and keep them updated before you open Scrivener seems to be the primary issue.

However, since I do a backup anytime I close or save my Scrivener project, I feel like any issues can be recovered.

At this point, I’m considering selling the iPad and grabbing an LTE Microsoft Surface Pro and doing the same thing – letting it sync with OneDrive. Before I do, I’ll try your method and let you know if I have problems.

[narrsd]

@anxiousdog

At this point, I’m considering selling the iPad and grabbing an LTE Microsoft Surface Pro and doing the same thing – letting it sync with OneDrive.

Hmm. It seems USD 10 a month for a 2TB Dropbox is much more attractive than $1500 and bulk, etc…

And, the LTE capable Surface won’t run Scrivener – just looked it up, and it’s not an Intel machine.

Further, as above, you are going to lose something very important if you try anything but Scrivener’s Dropbox method: that it is able to do its checks, quite often automatic corrections, and even if not, complete safety against losing your work, for any issues with syncing.

Like the ones you make when you forget to close on one machine before editing on another.

The Dropbox subscription should protect from whqt just happened to you, at least until you make 2GB of projects without ever archive saving and cleaning off your share(!).

I don’t think there’s any difference in reliability these days with the big name clouds. They will all get slow if you have a bad internet connection, but will always get through to degree they can, and fix up rapidly if needed as soon as they have a good one.

One last point is that a subscription lets you mark other files to be always present on all machines, which can probably save other problems you’ve had since they made that a paid feature.

Anyway, some factors to consider…

Also giving an anxious dog a bone – it’s sure we want tools that don;t make us have to think of too many things, when we need them in the heat of a job

[AmberV]

@narrsd: Further, as above, you are going to lose something very important if you try anything but Scrivener’s Dropbox method: that it is able to do its checks, quite often automatic corrections, and even if not, complete safety against losing your work, for any issues with syncing.

I would just like to point out that copying a project from one place to another (a task this approach attempts to automate) is entirely safe. There is no risk with copying projects beyond the baseline level of risk that all technology has. There is no need for checks, automatic corrections, and you will not lose your work unless you do silly things—like copy the wrong project, or halt the copy halfway through the progress bar.

Bear in mind that a Scrivener project is a folder with files in it. It is no more mysterious than that! It does not take special technology, or piles of sync code, to copy folders with a bunch of files in them between devices and systems. It’s one of the most basic and rudimentary capabilities of any file system.

It is also the only method I ever use to get data on or off my device. I wouldn’t trust Dropbox (or any other sync) nor do I have need for that level of complexity for how I work. There are many safe ways to copy projects around between devices, and many of them are documented in the Scrivener use manual for Mac/PC. We even have an article on semi-automating through copies, which describes a method that is far safer than using live sync, precisely because it makes use of the dirt basic technology involved in copying projects between systems rather than syncing them.

Thus: these are all officially supported methods, and in fact could be argued to be the default methods, since you have to go out of your way to set up sync.

Now one might quibble that the transport mechanism one chooses to use for this copying is sync based, but if you’re using it to transfer whole copies, or to even better yet transfer zipped copies, then the specific mechanism is almost irrelevant. We aren’t “syncing” in the sense that one folder and its contents is being updated in pieces by multiple devices in a sequential fashion. That is where things get complicated in terms of technology, and error-prone in the sense that it is easy to mess up at a human level.

With copying, you copy “A” from iOS then all of “A” arrives on the Mac and becomes copy “B”. Edit it and copy “B” over to iOS and it becomes “C”. If you overwrite, at that point “A” ceases to exist because it is fully replaced from top to bottom with “B”. But when one uses the copy method you do not need to overwrite—unlike sync users, you can keep a date stamped log of copies on each device, just for a little extra safety net. I can make sure “C” copied over well, and then choose to delete “A” if I want.

Like the ones you make when you forget to close on one machine before editing on another.

That isn’t a complication when you are copying projects back and forth (nor is it really one for the mobile version, as it is designed to be safe to work on a project left open on the Mac/PC, specifically because many people use it to quickly jot some idea down while walking around the house or whatever).

@anxiousdog: I have about 5TB of space on OneDrive, so I’m wondering if I can use it instead… Have you had any issues so far? As you said, remembering to download the files and keep them updated before you open Scrivener seems to be the primary issue.

There were some noted issues with using it specifically as originally described, on account of bugs in the iOS file manager that make it so it can get out of date with the sync server. The other issue I spotted is that the trigger, as originally described, happens whenever you multitask. Depending on how you work that may be copying things around far too often, and with larger projects, may even cause issues if you switch in and out of things so fast that it isn’t getting a chance to finish copying.

I would say a manual trigger, run from Files, is a lot safer all around.

Do check out the article I linked to above in this post. It’s the safest approach, is not too much less convenient than built-in sync, and can be used with everything from USB cables to whatever sync service you prefer. It works hand-in-hand with your backup on close approach to working.

That method, with the exception of using a much safer “stack of zips” instead of replacing one single copy over and over is essentially what is being described though, and could be automated if one chooses to. Personally I’ve never bothered as dragging the latest .zip over to Scrivener’s folder on the iPad, and tapping it to unzip, really doesn’t put me out.

[kewms]

@AmberV

That method, with the exception of using a much safer “stack of zips” instead of replacing one single copy over and over is essentially what is being described though, and could be automated if one chooses to. Personally I’ve never bothered as dragging the latest .zip over to Scrivener’s folder on the iPad, and tapping it to unzip, really doesn’t put me out.

Using “one single copy over and over” is very worrisome to me. In theory, it’s completely safe. Out in the real world, though, I would be very worried about accidental overwriting of the “correct” version.

[JimRac]

@anxiousdog

I realized my dropbox was full

A free DropBox account is around 2 GB, isn’t it? That’s a lot of space from Scrivener’s perspective, unless you’re projects are full of photos & videos.

If you’re keeping stuff in DropBox besides Scrivener projects, can you move that other stuff to OneDrive? Also, if you’re keeping Scrivener zipped backups in DropBox as well as your live projects, that’s an “all your eggs in one basket” scenario–you should move the backups to OneDrive.

Just some ideas in case you’re open to staying with the Dropbox approach.

Best,
Jim

[JimRac]

@anxiousdog

However, since I do a backup anytime I close or save my Scrivener project, I feel like any issues can be recovered.

While I find Dropbox more reliable than OneDrive, I consider OneDrive good enough for syncing Scrivener and would use it myself for that. Other posters here do.

But —

Be disciplined and careful no matter what you use. Sync errors can be insidious, in that you may not discover them until long after they’ve occurred. In that recovery scenario, you’re not doing a restore from backup so much as a document by document recovery. Tedious and painful.

[emilyst]

@anxiousdog

I have about 5TB of space on OneDrive, so I’m wondering if I can use it instead… Have you had any issues so far? As you said, remembering to download the files and keep them updated before you open Scrivener seems to be the primary issue.

In the end, I decided that the automations made me a little nervous, and I instead created shortcuts that I just try to remember to run manually. Putting these on the home screen or in the dock on my iPad help me. It’s possible to call these shortcuts from whatever automations you like on whichever device, once added.

I also dramatically simplified the actions in those shortcuts after I learned a bit more about what I’m doing. Below, you can see links to add these shortcuts:

• Sync Writing to Scrivener: Shortcuts
• Sync Scrivener to Writing: Shortcuts

As for why automations made me nervous—or more to the point, to answer your question about whether I’ve had any issues so far—the primary issue is that I don’t know how to forcibly trigger iCloud Drive to sync before firing a shortcut or automation. In my experience, it works fine. But it gave me a little more peace of mind to make sure that I could check the files myself before doing the copying, and I did not find this manual process (look at files, hit a button) to be untowardly burdensome.

---

I see there’s some really interesting discussion elsewhere on the thread, so I’ll follow up on that in another reply.

[AmberV]

Using “one single copy over and over” is very worrisome to me. In theory, it’s completely safe. Out in the real world, though, I would be very worried about accidental overwriting of the “correct” version.

I do agree! I would never use that method myself, even though I take daily backups of everything. Using Scrivener’s automatic backup system to instead provide a list of date and time-stamped .zips is much safer, and much less error prone since you can see right in front of you if the iPad is out of date with the server, as the copy will be yesterday’s date or whatever.

[emilyst]

I am super grateful for all the interesting and lively discussion I see here. It adds some additional context to consider for me or anyone else considering this “alternative” route to sync. I want to add some thoughts.

As I mentioned in my previous comment, I find myself in violent agreement with @AmberV that triggering the copying process manually seems to be preferable. The automations route seems to be a little finicky, as iOS considers simply switching away from an app to be considered “closing” it. (It is for this reason I can’t simply tell an automation to open the Files app to trigger a quick iCloud Drive sync—it causes a kind of behavior where the Scrivener app is opened and then closed in the same instant.)

In addition, using automations means you can’t take into account when you have no access to Internet, or sluggish access. If you have large projects, it may take some time to complete as well, adding an undesirable delay.

In my previous comment, I added a link to two shortcuts I use to accomplish the task now. I hope these are useful. I find them to be very helpful, and I trigger them manually from my home screen on the iPad.

I also find that I agree with everything else that @AmberV said. Thank you for your well considered reply. I suggest reading that reply if you’re at all unsure or nervous.

I would also like to elaborate on why I’m not using Dropbox. There are a couple of reasons.

• I’m already paying for iCloud Drive. (Even if I weren’t paying, I’d still already have its no-cost offering.) I don’t want to run a second sync solution just for Scrivener projects.
• iCloud Drive offers a kind of system integration that Dropbox cannot. Dropbox cannot (AFAIK) replace the entire Documents folder on my devices, for example.

---

As a final note, I worked on the shortcuts I mentioned above for a while to try to foolproof them, only to realize that Shortcuts simply isn’t up to the task. I found I couldn’t compare file modification dates, or trigger or interrogate iCloud Drive’s sync, or determine a course of action based on whether a file exists. It was honestly maddening.

I even considered attempting to write an entire application to solve the problem, only to realize I’d have to sideload it myself from Xcode every week or go through the entire application submission process with Apple.

(If ever Scrivener itself wants to attempt to make this sort of functionality possible as a stopgap before the next major version, it would work far better than a shortcut. From the user’s point of view, Scrivener could ask “Do you want to keep files in ‘some folder’ up to date with your Scrivener projects?” Then it could take responsibility for enumerating files which are newer or missing and copying them back and forth in a much safer way. But this is just a wishlist item on my part, and I would find it very understandable for the developer not to bother.)

I considered deleting this entire topic a few times because I felt squeamish about suggesting something that could be risky in certain circumstances, but ultimately it seems like there is value in offering it. I appreciate the discussion around this, which ensures that others finding the topic will have a fuller understanding of what they’re getting into and how it works.

I’ll continue to monitor this topic and make updates as need be.

[creech]

Thanks Emily - i’m surprised at anyone who uses any apple product not begging constantly for iCloud file sync, and so any post telling of attempts at staying off Dropbox shared here are highly valued.

To your last comments, please never delete these posts - especially if you change your mind and don’t want to promote them - otherwise someone else like me will start the same thing over again because Files access or sync in Apps is basic stuff nowawadays for the Apple dev community. You’re helping all of us with your example and this thread is awesome to read!

iCloud and The Files App killed Dropbox for everything for me except Scrivener - which i keep leaving over these basic file issues -but i keep coming back to because i like Scrivener so much. If someone was to feature match them a bit more though….

I actually wonder at a simple iOS app that only accessed files and checked things and moved them based on modification date checking, then launched Scrivener wouldn’t be that hard - plus you could simple load it from TestFlight and share accordingly …

Trying to tell myself i dont have the time LOL - thanks again really appreciate it with the very slow/no progress from Scrivener on this critical feature !

Anyways thanks so much again,