Odd hang with binder.backup and binder.autosave

Scrivener Scrivener for iOS
,
1

I’m not sure if this is a macOS matter, or an iOS one, but it only began emerging with iOS 18. If I’ve opened and edited a project on iOS, Dropbox seems to get hung up on trying to sync either binder.backup or binder.autosave, with the desktop menu bar app showing a “waiting to sync” note on one or the other of those files.

The result seems to be that my changes aren’t shown in the Mac side of the project until after I’ve opened it, quit Scrivener, and reloaded the project.

Most of the time when I do that, Scrivener loads the changed files as expected; but sometimes I get the “conflicted versions” dialog, and earlier this evening, everything in one of my text files was gone, tucked away in a “recovered items” folder.

It doesn’t seem to matter what project file it might be, how many items it contains, or how large it is. Whatever I’ve recently opened and synced seems to exhibit this behavior.

Opening Dropbox’s web control interface and manually deleting the hung-up file seems to resolve the problem.

Are there known or even rare problems with iOS 18?

2

This is actually expected behavior. iOS Scrivener writes its changes to a separate “Mobile” folder within the project structure. When the project loads, Mac/Windows Scrivener reviews that folder and incorporates any changes. Use the File → Sync → With Mobile Devices command to update the project without reloading it.

To avoid exactly this issue, we recommend closing Mac/Windows Scrivener before switching to a mobile device, and then ensuring that Dropbox has finished syncing before you re-open it.

2 Likes
3

Please understand this is new behavior in my experience, which I believe to be considerable. I’ve been using Scrivener for well over a decade, before there was even an iOS version of it, have written something on the order of one million words using it, and have found the overall integration via Dropbox to be smooth and largely trouble free.

4

Possible culprit or cause:

With iOS 18, you can use fingerprint ID to lock out any app from opening, theoretically preventing unauthorized access, if you happen to leave your device lying around unlocked.

As some of the material I work with in Scrivener is fairly sensitive, I enabled that.

That was when the misbehavior I’ve reported here began, and when I disabled it, so far, it’s seemed to settle down.

I can’t be sure I’m not committing a correlation-causation error here, but that is the only readily identifiable variable I’ve introduced to my Scrivener workflow in years.

What I’m wondering is whether iOS is hanging Scrivener somehow, maybe by adding a layer of authentication that isn’t satisfied to iOS’s specifications, and keeps those autosave documents from being marked as finished.

1 Like
5

Have you tried resetting the Dropbox cache yet on your Mac? Persistent failure to sync certain files seems like a symptom that would indicate a problem with its mechanism.

But who knows, there are many mysteries these days in sync. It used to be extremely straight-forward at a conceptual level, and even a technical one, but Apple has been making it increasingly opaque and complicated (which yes, impacts Dropbox because they were strong-armed into using Apple’s technology instead of their own). Maybe locking your phone the wrong way can in fact jam files on another computer! If so, that seems like a cascade of failures that shouldn’t happen though, even still. Imagine a share folder with 15 people using it, and one person doesn’t set their phone down right. :laughing:

6

That, I have not yet tried, no. I made a surmise about a new feature in iOS 18 — the per-app option to require fingerprint verification on load or refresh — which was a feature I’d turned on for Scrivener. I’ve since turned it back off, and so far I haven’t seen a recurrence, but it’s only been one day and one project.

1 Like
7

It will be interesting to see if it stays fixed, and if someone else has this happen we can have them check that. It is very strange though, since as noted above the iOS version won’t be changing 99% of the original project (just some files in the Settings subfolder, that it uses almost exclusively for itself). It has no knowledge of how to create or update these binder files, nor the features to do so safely.

Plus, the bespoke Dropbox client the mobile version uses shouldn’t be uploading any files that haven’t changed on the device (otherwise it would be a lot slower). That’s why it is so weird—that what amounts to an OS level lock on an application could influence how that application’s custom binary executes certain commands, while it is locked and thus not syncing, via an API that must be manually triggered, so specifically as to jam files on a third-party sync service and halt the functioning at the server level so that another device cannot sync. If all of that is gibberish, that’s like a flashlight being turned on against a wall causing a machine in another city to turn on and start making chicken nuggets. There shouldn’t be any plausible path between A and Z.

8

Indeed. It’s entirely possible this is a correlation-causation fallacy on my part. But my troubles only began after the iOS 18 update, and after I’d enabled fingerprint authentication, and they happened on two different projects. Since I’ve disabled that fingerprint ID setting on the iPad, I haven’t seen a repeat performance, but it’s only been one day and one project so far.

And yes, it’s puzzling how a system setting that essentially (seemingly) puts a blur over an app’s screen could cause a third-party FUBAR. But if any part of the app’s activity is suspended by that lockout until authentication is accepted — and particularly if it persists as a background process, which seems to be the case — who knows what kind of gremlin could be spontaneously generated?

iOS looks simple enough, but I know enough about its back end, and about coding, to not be deceived by that. And complexity, combined with a brand-new feature, is a sure mix for esoteric errors.

9

Security software in general tends to be somewhat heavy-handed. And it’s not actually surprising for a failed authentication to prevent the application from contacting the internet (or interrupt internet tasks in progress), since the internet is how malware reports back to its masters. So mere correlation it may be, but it’s a persuasive correlation.

2 Likes
10

What I’m going to do, now that the creative dust has settled a little, is see if I can replicate this error. If I’m able to, I’ll post steps to repro so others can try it and maybe something can be chased down and isolated as a cause.

If I’m unable to replicate the error, I’ll report that as well, since negative findings can also be useful.

This may be related to files vanishing from within binders after moving a project from DB to an iCloud-synced folder, depending on what is happening on the back end with sync in general. This is only an inference based on the fact that the topic I’ve posted at the link above, and this topic, both began manifesting in early October. Two weirdnesses affecting related systems smacks of an introduced bug, though whether it’s Dropbox’s or Apple’s, I’m not sure.

@AmberV noted, upthread and in passing, that DB has been forced to adopt some of Apple’s sync tech, which is becoming more poorly documented for developers, and I’mm’a bet an entire cupcake that this is at the root of it. There is file f*ckery afoot in the Hallowed Halls of Jobs, and we’re seeing esoteric and abstruse collateral effects.

1 Like
11

Yeah, definitely. If you can find a clear on/off situation with this new feature we’ll take a look at it. But I suspect it might be beyond our reach since our use of Dropbox on the mobile client is purely through API calls, and while it downloads everything from the project, it only initiates “puts” through the API on the things the mobile client has modified or created during the session.

Perhaps there is a mistake somewhere in there, but in theory it shouldn’t ever be trying to do anything with those two files with the API, so whether the software itself is locked or not shouldn’t make a difference one way or another, theory. :smiley:

12

It’s not the touch ID that’s causing it. I suspect it’s Dropbox, and I suspect the problem I reported here is a DB-generated one as well.

To attempt to replicate the difficulty I’ve been having, I created a new project on my Mac and banged out a couple sentences in it. Then I saved the project and closed it, opened the Dropbox menu icon, and saw this:

stalled_sync
stalled_sync859×1279 109 KB

As you can see, there are a couple files at the top that are “waiting to sync.” As you can also see, the Dropbox icon is lacking the sync flag (circle with arrows in it). So one part of DB is waiting for content to sync, while another part of DB apparently has no idea there’s content that needs to be synced.

I kept the window open to see what would happen. Those items did eventually sync, but after a four-minute delay.

As far as I’m concerned, that makes it glaringly clear touch ID on the iPad app has nothing to do with it, as it hadn’t even seen the file yet.

The Dropbox app for macOS, version 209.4.3661, shows a last modification date of Oct. 4. This was likely when it was most recently updated (DB updates silently). And I first started noticing this problem in early October. I don’t believe in that level of coincidence.

Pretty sure their latest update borked something somewhere, and until and unless they address this, we’re going to continue seeing it. And as Dropbox will not allow you to submit a bug report or support request unless you have a paid subscription (no way am I coughing up $10 a month just to do them a favor by reporting an error in their code), it might be a while before they pull up their socks and address it.

We’re so close to having an alternative, though, as I’m able to import existing .scriv projects from my iCloud drive into Scrivener for iPad. Unfortunately the only two choices are to store it “on my iPad” or in Dropbox, according to the dialog that opens; and once it’s in there (“on my iPad”), changes I make are not reflected in the project in the iCloud folder. This presumably is what “import” means, heh.

It would be so very nice if I weren’t relegated to those two choices, if I could instead have Scrivener for iPad treat external .scriv files in a way similar to what the desktop app offers — opening the file opens the app, editing happens, it’s saved without needing to import or export, etc. — though I imagine the coding might be fraught, as iOS in general really isn’t designed to work that way.

The other option would be to allow user-selected folders for syncing, thus relying exclusively on iCloud to make it happen, if users wanted that option, and bypassing Dropbox entirely. But that’s yet another fraught exercise, I suspect.

13

Thanks for the update.

Right, hence the iCloud-level quality I referred to. Status that doesn’t seem to work right, sync that can take minutes or longer to start, etc. It used to be iCloud was clearly worse than everything, but now everything feels a bit like it does, and even iCloud seems appealing to some, I’ve noticed (all according to plan, I’m sure).

It’s something we’ll have to keep an eye on in our support involving sync. Just to double check, did you ever get around to resetting the Dropbox cache? That can often clear up “jams” like this.

Otherwise, there are alternatives, such as Maestral, that sync your content in a more straight-forward manner. No weird proxy folders that seemingly only exist in the Finder sidebar, no “smart sync” that removes all your data from the disk and makes your own local backups worthless, no greedy policies that only allow you to keep one single folder “offline” at a time (that this word should even be used with a sync service is indicative of how lost these companies have become as to their original purpose), or only install on three devices. It does use the Dropbox API, like Scrivener itself does, so it will be slower than you are accustomed to. They throttle that connection to cripple developers like this, who come along and make demonstrably better clients than they do. :wink:

It would be so very nice if I weren’t relegated to those two choices, if I could instead have Scrivener for iPad treat external .scriv files in a way similar to what the desktop app offers — opening the file opens the app, editing happens, it’s saved without needing to import or export, etc. — though I imagine the coding might be fraught, as iOS in general really isn’t designed to work that way.

It’s getting… better. Sort of. At least so long as you don’t edit packages (which Scrivener needs to) over cloud accounts. iOS still struggles with the concept that one program might upload more than one file at a time, at least in my testing. But they are at least working on fixing past mistakes.

We’re so close to having an alternative, though, as I’m able to import existing .scriv projects from my iCloud drive into Scrivener for iPad.

I guess you could think of it that way, as importing and exporting, but if you go into Files.app you’ll see it’s just copying things around on the file system. This is the only way I work in fact, and honestly would probably do so even if I could do the above. I would always prefer the safer path when it comes to sync. I think of the tech more as a thumb drive replacement, not a file server replacement.

14

I’m unable to find anywhere to reset the Dropbox cache. Doesn’t seem available through the menu icon, or online.

I wasn’t aware there are sync alternatives that will work with DB. Or was the Maestral reference more along the lines of a general recommendation?

The only reason I have DB now is to support Scrivener sync. If not for that, I’d hardly ever use it.

Re the “iOS still struggles with…” part: Well, heh, there are a lot of things it struggles with…

15

Go to Scrivener’s section in the iOS Settings app. Scroll all the way down to the Reset Scrivener menu. Clearing the Dropbox cache is one of the options on that menu.

16

Ah.

Done, though considering that this seems to be happening just on the Mac/DB side, it’ll be surprising if that does much. Anyway, thanks.

17

I meant resetting the DB cache on the Mac, though it never hurts to do the mobile as well (just makes it slower for a bit).

As for Maestral, it is a third-party open source Dropbox client, meant to be used instead of corporate’s official client. It is clean, simple, and does only what it needs to. And it doesn’t make your work online-only by default, which is a problem with the official client, especially on free accounts.

18

Ah, I see.

There is no .dropbox.cache folder in the DB folder, but only this evening I got a “new” notification telling me my passcode was [digit string], and I could finish setting up my new device, which could only be the G10 iPad I got about a month ago.

Well gosh, guys, thanks for telling me on the day of…

19

Although that’s probably all it is, I’d go into your account page and make sure that’s actually your device trying to link up to your account! I’d be a little wary of a message like that coming out of the blue.

20

Did, and there’s nothing there I don’t recognize. And interestingly enough, it has a deep backhistory, including machines I’ve long since decommissioned and retired.

It’s also amusing to see the device locations change, with Apple’s “Private Relay” mini-obfuscations.

And since I got that odd and belated notification, sync seems to be settling down and behaving more as expected, which was not an outcome I would’ve predicted. However, and to stir things up, DB updated itself on Oct. 16, as well.

I think this one is apt forever to remain a mystery.

1 Like