Scrivener 3 macOS ↔Windows: All binder aliases are broken when reimporting back to macOS

I use Aliases extensively in the Binder. Now I do not expect the alias to work cross-platform, but I do hope that when a macOS project that was then edited on Windows comes back to macOS that the aliases would work again. However, currently all my binder aliases in a project returned from windows are broken; Scrivener now treats them as a normal file (e.g. tries to show a PDF using the alias file itself). The alias file is binary identical before and after the roundtrip, but the macOS Finder filesystem metadata is now missing.

Here is the macOS metadata for a broken and working alias to the same PDF file:

➜ ls -l@
total 8
-rw-r--r--@ 1 ian  staff  1104 30 Nov 16:31 content.pdf
	com.dropbox.attrs	  26

➜ ls -l@
total 8
-rw-r--r--@ 1 ian  staff  1096  2 Feb 12:26 content.pdf	  32	  42
	com.dropbox.attrs	  26
➜ xattr -l -p content.pdf
00000000  61 6C 69 73 4D 41 43 53 80 00 00 00 00 00 00 00  |alisMACS........|
00000010  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  |................|

I assume being lost is the culprit[1]. It would be good if macOS Scrivener could “rebless” these alias files on the roundtrip back. This would require either Scrivener to store the fact it is an alias in the Binder, or it could test the file itself on opening.

[1] … he-finder/

For anyone that faces this problem, I created a small script that traverses the Scrivener content files and first tests if they are binary alias files[1]; if so then it writes the correct metadata back. Only metadata is rewritten using Apple’s xattr command so the files themselves are untouched, but no guarantees are provided! … Aliases.rb

Download the script and make sure it is executable, then run it from Terminal passing in your scrivener project folder.

> fixAliases.rb "$HOME/Dropbox/Paper/Test/MyProject.scriv"

It would still be nice if Scrivener could work round this. I do see that Scrivener’s XML binder data does contain the fact these are aliases: Yes

[1] I read the first 12 bytes using xdd and match using the hex 626f6f6b000000006d61726b (book…mark in ASCII)

Hmm, I just ran a quick test, and these results may depend upon the transport method used, or perhaps some activity done within the project that might impact those items that I missed. Copying the project around with SMB, and doing things to it that would modify the .scrivx, don’t really seem to have any detrimental effect. The metadata remains intact.

But despite not having a natural way to reproduce this, I can certainly do so by hand. So I did that, and in tandem with a workflow that I anticipated would repair the damage. The result looks promising, give this a shot:

  1. First archive the project on the Mac (I used Apple’s modified tar, which creates AppleDouble files for storing metadata like so: “._content.pdf”, but Compression Utility’s zip should also do this, using a different method).
  2. Copied that over to the PC and extracted. I see the dot files as I browse through the content, so in theory when I archive this back up and then untar it, Apple’s tar will piece it back together.
  3. On the Mac, working on a copy of the content.pdf file I use xattr -d content.pdf to destroy the metadata.
  4. Copy the damaged content.pdf over to the PC, and replace the good copy with that. This may be unnecessary, but I don’t know for sure if tar actually splits the file or stores the metadata redundantly into the double-file.
  5. Finally, ark the test copy (using 7zip to create the tar), copy it back to the Mac, and untar (simple double-click in Finder).

Result: good alias, and the content.pdf file has the correct metadata. I ran a quick counter-test to make sure I did indeed damage the PDF correctly, by tarring another copy on the PC, and this time deleting the ._content.pdf double file from the archive. On extraction, the content.pdf has no metadata, and displays improperly in Scrivener, much like I suspect you’re seeing.

So that might be worth checking out as a workflow. I think overall tar is going to be a better tool for the job, based on the mechanical differences between zip and tar in how they handle copyfile data while on the PC. The tar method interleaves metadata into the original directory tree, and so is largely invisible and resilient to changes to the tree. The zip method depends upon a directory mirror in a top-level sibling folder that must be archived back with the project in the end—and while in theory Scrivener won’t be changing the existing tree, I don’t know what will happen if the tree changes dramatically (say you import 100 JPG files on the PC) and the toplevel __MACOSX file looks way different than the directory it’s supposed to match. It seems more fragile to do things that, and Apple probably intended it more to be an invisible mechanism that Mac users never see when sharing .zip files around.

But maybe it works fine. It could be worth testing, since using Scrivener’s backup functions, along with native Windows zip compression, would be a lot simpler than making tarballs.

I was using 7zip archives (my preferred GUI for archiving is Betterzip) to send to my student and my student was zipping back using some generic windows app to return to me. Indeed that __MACOSX folder was either not extracted by his zip program or he didn’t add it back (he doesn’t remember seeing it). I didn’t know the exact differences between zip and tar for metadata transport, as you say the differences between the intermediate transport for zip and tar files are :

[attachment=0]Screen Shot 2021-02-03 at 12.54.50.png[/attachment]

I do think tarballs are going to be more robust, in that the ._ files are next to their links rather than in the __MACOSX shadow directory that is easy to lose. Nevertheless I have tested using 7zip on Windows (which I’ve recommended my student to install) and BetterZip on macOS and can get round-trip transport that preserves metadata. I told him to preserve the __MACOSX folder on extraction and include it when he zips up his version.

Using tar, one small caveat is that it seems you need to extract (-x) as root for metadata to be preserved according to the man page:

This is going to be harder for less geeky people to follow, and so I still wonder whether Scrivener should try to fix aliases when they are broken (as that is easy to test/do)? At the very least it should be mentioned in the documentation about how to round-trip scrivener projects between macOS and Windows.

Is that something that has changed in more recent versions of macOS? The word ‘metadata’ doesn’t even appear for me. I had to look up how to even exclude the AppleDouble files since it is undocumented for me (it’s –disable-copyfile by the way, at least in 10.14). So by implication it should always split and combine the dotfiles automatically, and that’s the experience I’ve had with it. Like I said, I only needed to double-click the archive in Finder for it to work properly.

The again, come to think of it, I may have set Keka (the GUI replacement I use for Archive Utility) to take over all compression tasks. But I did also test using tar on the shell with a simple tar xf as a normal (admin) user.

No debate there! Just talking about tar at all is probably a branch too far! And it does look like it would be straight-forward and safe to repair the metadata if necessary. I would imagine in most cases, Windows users either will not extract the __MACOSX folder, or not think to include it in the archive if they do. And of course most Mac users will be entirely unaware of that folder since it never shows up using the standard GUI tools.

Well, at least there is a solid workflow for now, even if a bit geeky on both ends. 7zip on the PC isn’t too difficult to use, but it’s certainly not as simple as Explorer’s zip tool. I’ve added the rest to the list for investigation.

Thanks for digging into the details of what it takes to fix broken archives!

Hm, my tar version in macOS 11.1 is:

➜ tar --version bsdtar 3.3.2 - libarchive 3.3.2 zlib/1.2.11 liblzma/5.0.5 bz2lib/1.0.6

The date at the end of man tar says “February 25, 2017,” the info on –mac-metadata comes on line 236 of the man page, and i don’t see a mention of –disable-copyfile at all :open_mouth: I am surprised the CLI interface would have changed so much between 10.14 and 11.0…

Wow, yeah they took a while in upgrading bsdtar it seems. The version that ships with 10.14 is 2.8.3, man page dated Oct. 2009!

Maybe you were thinking 2017 was a bit out of date. :slight_smile:

Well that’s kind of annoying that they made it less seamless in handling metadata through archives. Extracting data in the same state you archived it in three years ago shouldn’t require administrative rights.