Search by UUID

Greetings

I think it would be helpful if the UUID, the unique identifier that Scrivener uses to locate and maintain the docs and folders within its projects, were deemed an honorary custom metadata field for search purposes.

Yeah, it’s for those occasions when we’ve found a Scriv document through the Windows file system, and would like to view or maintain it within Scrivener. (Bad practice? Scold away, but this is our data, and we’re free to parse it as we please.)

Cheers - Jerome

You’re free to parse your data as you please, but we’re free to decide what approaches we will support.

Accessing the contents of a Scrivener project with any tool other than Scrivener itself is unsupported and entirely at your own risk.

Katherine

Wow, Kewms !

Having the exact file name of the document / folder as a read only metadata or being able to display a table of equivalence somewhere between file name and document name is also in my wish list.

There are some legitimate and non avoidable use cases for manipulating files generated with the Scrivener software outside of the Scrivener software, such as: folder comparison, file comparison … Also any source control software will work at file level and not treat the Scrivener project as a whole.

These workflows are very common when working with content and not offered by Scrivener (which is entirely understandable) so I am working daily at file level inside Scrivener projects outside of the Scrivener software. As you said, my risk. But displaying this information would make it easier. What do you expect? , the files are non-binary and not encrypted, so of course users will peek.

For instance, having this information hidden forces me to very heavy-handed selection in my source control software:

Finally end-users’ enhancement requests may be unsupported, contrary to the product vision, orthogonal to the software’s internal architecture, too costly, whatever, but they are always interesting because they are a sign of interest, and also they give good pointers to how users use our products (never quite how we thought ! ) .

So, I take the risk to second the original poster’s wish… :slight_smile:

S.

Why would we want to make it easier for users to corrupt their projects?

If you have the knowledge to figure out Scrivener’s project format for yourself, you probably also have the ability to figure out how to access the component files safely.

As opposed to the user I once helped who used Finder on his Mac to manually rename every single document inside the project, and then was outraged because Scrivener couldn’t read the project anymore.

Katherine

Displaying this information will only allow users who have no idea what are they doing to brake things up. Which will end up in a shitstorm of complains.

This is a very bad idea.

Regards,
M

Busted, I’m a big bad user intent on breaking everything then …
I was just saying … the original poster’s request seems not illegitimate to me, as you are already using an open format (txt, rtf) and made the choice not to use a proprietary binary format and so the content is not protected from exterior intervention, so to speak.
I also have some idea of the constraints of support, as I myself am a software developer.
I’ve worked on projects with proprietary sedes where nearly everyone was reluctant to modify the serialization code.
And also on projects with open formats protected by file and folder checksums.
It is just a choice on your part, and a “wish” on mine. Please, just be cool about it.
S.

We use an open, non-encrypted format because we don’t want to hold user data hostage. Your data will still be recoverable if Scrivener ceases to exist.

But as long as Scrivener does exist, we don’t want to encourage users to poke around in unsupported and potentially damaging ways.

Katherine

They are being cool about it. “We don’t recommend you mess with the individual files inside a Scrivener project. If you do, you’re on your own to fix the damage you create.”

The Scrivener project format is, in certain places, one of those. There is a checksum on the primary data folder, as well as on the main binder structure and a few other places, and tripping those will cause the software to drop to a rebuild state when you open the project the next time. A little search index rebuild and scour for orphaned files isn’t the end of the world in most cases, particularly if you know what you’re doing.

Ultimately our stance on the matter is what has been said above: that writing formats should be transparent and open, and easily modifiable by those with the skills to do so. I have a few scripts of my own in fact that are designed to do various things to a project that there are no features for in the software. The XML is human-readable enough that I don’t even think I’d say one would need to be a programmer to examine the files directly. But we do encourage scripting, and will provide help to those that need it. A number of interesting tools have become integrated with Scrivener as a result of that. So your outrage is a bit misplaced, I would say, saiph.

As for the original request: there is an easy way to look up binder items by UUID, and I think it’s easy enough to not need user interface for it. Just open the search.indexes file in notepad++ and search for the UUID. That’s the simplest and easiest XML file to read, and since it has the full text right there, it’s the best tool for making sure you have the right result. Works well in the other direction as well, if that’s what you need.

As you say, it’s your data to parse. Not everything that involves that needs to be in the GUI, and I think for the kinds of things that go beyond the basics, sometimes that additional “just open the .xml file in notepad++” barrier is good enough! Anyone that even knows what “UUID” means isn’t going to be scared of doing that, and those that are scared of doing that probably should be! Problem solved. :mrgreen:

Folks, there was a time not that long ago when Scrivener took great pains to obscure the identifiers that connected binder docs and the file system. I’ve always respected that design choice in the forum, while circumventing it privately in my AutoHotkey script. Scrivener isn’t for data geeks; it’s a popularly-priced program for ascendant writers, and it will not remain so if its support team is attending to user-inflicted file damage.

But the obscurity approach has become antiquated. Into 2015 my script would find the file identifiers by bit operations against a couple of variably offset bytes on the HTML clipboard. Then came the ScrivLnk, which made them accessible to non-programmers. Today we can retrieve these identifiers in plain text via a menu operation: Copy Document as External Link. And the User Manual for MacOS (section 10.1.6) shows advanced users how to construct links around UUID.

So UUID has evolved from an internal data reference into a partially supported field. And its ease of discovery on the outbound side is what might tempt a user into an unsupported data operation.

But I’d argue that supporting UUID as an identifier on the inbound side would have the opposite effect. Internal consistency requires that we update our docs solely within the Scrivener ecosystem, even if we’ve identified those docs by external means. Whatever the use case for inbound external links, that case is further advanced by tools that allow for recognition of UUID in interactive Scrivener operations. Scrivener allows us to search for documents by their metadata. UUID is the metadata field that all projects have in common, representing each document’s sole unique key.

Interactive support of UUID would actually spare users the need to create and maintain a redundant unique key within metadata, and make it easier to maintain all the elements of a Scrivener project within Scrivener itself.

Cheers - Jerome

Scrivener is already considered a pretty complicated piece of software by many users. And although I think it’s the best software for writing novels on Windows, its learning curve is steep, and a lot of the concepts it uses are foreign to the most ordinary users.

All this talk about scripts, UUIDs, external-internal links and so on, scares people even more. I know that in the forum, the most vocal users are people who have IT background, but Scrivener team, IMHO, shouldn’t lose what Scrivener is all about: An organizational tool for a non-technically adept customer who doesn’t want to spend his time programming instead of writing.

Adding more complicated options may be useful for a few hyper-advanced users, but in general, it is a bad idea.

You don’t have to believe me, just take a look at the latest software products that come out of the big companies like Google, Microsoft, Adobe… All of them are trying to simplify the UI even at the cost of cutting off some functionality.

This is precisely the way I would like to see Scrivener go. Simplifying and making things easier to understand.

Regards,
M

Such options also add a non-trivial technical support load. Saying “if you really want, you can open the .xml file in notepad++” will attract a different group of users from “choose this option in the Project Search menu.” Once we expose the functionality to non-technical users, we have to be prepared to help with whatever those users decide to do with it, however misguided that might be.

Katherine

You bring up a good point: the hyperlink mechanism in v3 is a kind of interface like you’re asking for. I realise it’s not quite the same as pasting into Quick Search or whatever, but if you have a link set up somewhere and drop in the UUID whenever you want, then it’s pretty easy to do. Project Bookmarks strikes me as a good place in fact.

For myself, I use a system-wide tool that lets me launch custom searches—typically you would use it to launch a web search of some sort, it works like any browser that lets you create custom searches, with a wildcard in the variable spot of the URL. You’d have to create a few different searches for your commonly used projects, but that would be a handy approach if you have a tool like that.

Couple of belated thoughts. Searching for UUID in Notepad++ would be pointless for users who are seeking the minute-to-minute capability rather than the occasional retrieved value. Anyway, I already have the UUID; I’m looking for the easiest way to deploy it to open the document.

It’s true that as an experienced Scriv scripter I can work around the absence of UUID among Scrivener’s meta fields. My script maintains its own unique search key in custom meta, BinDocID, which has its roots in the integer DocID used in Scrivener 1. With almost 11,000 docs in my main project, it’s helpful to have a friendly integer handle for each.

UUID in metadata would enable most of the scripting operations to work on any Scrivener project, not just the one I’ve seeded with a redundant unique key. But my suggestion is founded on UUID’s advantage for writers, based on my experience with BinDocID.

I’ve used BinDocID in Export and Compile to capture the fragment’s provenance. That’s how UUID would be used as metadata. And Scrivener 3 acknowledges that need in its compile options: Insert links back to Scrivener in each section, and Insert unique document identifiers only. These insert UUIDs into the compiled output, either as x-scrivener-item links or as plain text entries. And they demonstrate as well that UUID is a working field for writers, no more an inordinate support burden than any other aspect of Scrivener’s design.


So now, if UUID is a plain text field in compile, devised to give the viewer an indication of each section’s provenance, doesn’t it make sense to be able to paste that key field into the project’s search bar? And doesn’t it make sense to be able to insert UUID and its corresponding x-scrivener-item as fields on Export as well? So for example if we’re exporting to brainstorm among our docs in Freeplane or OneNote, we can likewise have a live link back to each doc in Scrivener. These are among the advantages of treating UUID, and the matching x-scrivener-item link, as honorary metadata.

Thanks again for considering

Cheers - Jerome