Documents copied between projects don't necessarily copy over the most recent version

More than a few times I’ve noticed that when I drag a document from one project to another project, the document will show up in the target project as an older version. For instance, sometimes if you create a document, type three paragraphs, then decide that the document actually belongs in another project, the document will arrive in the target project with just two paragraphs.

I believe I know why this happens and can even work around it. However, you have to actively plan for this bug and if you forget you can lose data in the transfer. The reason it happens (I’m guessing) is that Scrivener copies over not what has actually been typed but rather what existed at the time of the last save. If you manually save (control-s) before dranging the document, then I think (emphasis on think) this will circumvent the bug.

Nonetheless, it is a bug — at least in my version of Scrivener. I’m running MacOS Catalina and have Scrivener version 3.2.3.

I don’t know if that would be an easy one to solve, since the only way of copying the data over between projects would be to correctly manage all of the different aspects of data that are stored in RAM, rather than what is on the disk. The overall complexity of this is larger than the text content itself, as an individual binder item can represent quite a number of actual files on the disk that need to be transferred over to the new project. Some would be in RAM, some would have to be files, and I can see that getting pretty messy and potentially buggy.

At any rate, in the meanwhile just make sure the little dot in the red window title bar button is gone (that indicates auto-save has run), or as you say, hit ⌘S if you don’t want to wait the two seconds for that before dragging.

1 Like
  1. Perhaps mdmullins has increased the auto-save wait time to something higher than the default 2 seconds?

  2. Ensuring a doc being copied over is auto-saved seems like the disciplined thing to do, but shouldn’t the app itself have this discipline? I suppose that in the course of drag and drop, the app must process a request to deliver certain files over, so it seems like there is a catchable and sensible moment where the app might-should run auto-save on those files before continuing. The app should not be willing to copy over (or duplicate) files that do not represent the current state of them. The only reason I can think of not to internally invoke auto-save is producing an unpleasant moment of lag in the drag-and-drop. But isn’t it the same momentary pause the disciplined user would be waiting anyway while autosave finishes?

3 Likes

That’s a good point, @mdmullins, I’d check the General: Saving preference pane and see if your auto-save idle period is significantly higher than the default of two seconds. Given you were able to type an entire paragraph without triggering it, in your original example, that could be a negative factor here.

Anyway, I suppose that in the course of drag and drop, the app must process a request to deliver certain files over, so it seems like there is a catchable and sensible moment where the app might-should run auto-save on those files before continuing.

Yeah, that’s a thought I had as well. There are actually two places where this can occur, the other is Documents ▸ Copy to Project, which also operates solely upon the disk state and can be used prematurely.

I don’t know the ramifications of doing so, but it would, if possible, be the most elegant solution. Well—one I can think of is just what we were speculating on: users that have cranked auto-save up to a level where it rarely fires on purpose, as a way of having more manual control over saving, might be upset to find mere drag and drop saves everything. Hmm.

1 Like

There is that, but I think one could still argue that for even such a user it would be their intent to have copied over the actual current state of the docs, not the disk state.

1 Like

Yeah, but then we’re back to the potentially very messy quasi-disk/RAM drag. Currently I believe the drag event is very simple: send a UUID to the receiver which then uses that data to extract all associated files from the source project. Can a window extract RAM data from another window though? Would that be safe to do? Would the OS even allow it, since messing around at a low level like that is often the source of crashes?

1 Like

I’d say copying the expected data (read: all of it) outweighs the potential inconvenience of having to wait for an “unwanted” autosave.

Alternative: Warning dialog when unsaved files are involved (Scrivener should be aware of those).

2 Likes

@AmberV @November_Sierra
Hi. Yes, I did increase auto-save because, at the time, the document was large enough that the lag of the document auto-saving was disruptive.

I agree that the discipline should belong to Scrivener: when a user drags a file, they naturally expect the file they see to get dragged over and not the file they saw at some point in the past. However, an automatically triggered auto-save at this point and any resulting lag would, in my opinion, be trivial. Lags while dragging and dropping seem somewhat natural?

I think this is correct:

1 Like

Yes, I think we can all agree that there is an idealistic concept here that nobody would argue against. :slight_smile:

That’s not really the point; as is often the case, the devil in the details, and those are as yet not fully known but being speculated upon.

So at any rate, thanks for the report.

If it were possible to employ multithreading so that auto-saving occurs deep in the background, then the lag caused by auto-saving large files would disappear? I of course don’t have access to the code base and probably am not fluent enough in programming to make sense of it, but it seems to me that concurrency may be a solution?

I’m not a programmer, but the idea that you could save a file without interrupting the user’s access to it seems problematic to me.

There’s a bottleneck, in that only one copy of the file exists in memory at once. To save it, you have to either save that copy directly to disk, or clone it and save the clone. (The “clone” solution is pretty close to what actually happens, due to disk buffers and what not.) Either involves temporarily “locking” access to the file.

As far as I can see there would be no upside to trying to do this in some tricky way (gleaning changes from RAM or multithreading).

To reiterate a point made earlier: saving the relevant files to disk prior to sending the UUID to the receiver causes no additional overhead – because anyone who wants to drop files over wants the real thing copied and hence /must/ either let autosave complete anyway or must manually invoke it (if they have set a long autosave).

Not to put too fine a point on it: having the app do the save before drop-copy takes time, but costs the user no additional time, and, from the armchair at least, would seem to be programmatically simple and unproblematic (as autosave is already a thing).

So, if you are putting something into the hob about this, I should think tapping autosave (for the targeted files at least) is the thing to put in the pot.

“Copy on write” filesystems do exactly this under the hood, but at the block level, not the filesystem level – when a write to disk happens, the new or updated data is written to new disk blocks. Once the new blocks are written, the links between blocks are updated to point to the new blocks.

Depending on your OS filesystem routines, this may already be what is happening at the file level, too. When the program calls the “write file” routine, the library may simply be locking the original file (if it exists) to read-only mode, writing the new version of the file to disk, then updating the filesystem pointers to point to the new copy of the file and release the lock. Doing it this way is simple, clean, and performant.

Unless the application developer is willing to write their own low-level code to interact with the filesystem (and that is taking on a HUGE task that usually is only worth the time and effort for very specialized scenarios), they’re stuck with whatever the behavior of the system library is.