Metadata is lost after split

EDIT/UPDATE: I’ve created an AutoHotKey script to duplicate the missing functionality, and shared it here.

Both Scrivener v1.0.1 or v1.2.1.0 running on Windows 7 64-bit Home Premium exhibit this behavior:

Page 96 in the manual: “When splitting documents, all meta-data will be carried over to both documents.”

But when I split text-only documents that are NOT snapshots, NO metadata (status, keywords, etc.) is carried over to the new document according to the inspector pane
Happens with both Ctrl-K and Ctrl-Shift-K after selecting text

Example: I create a blank new project, type in a few words to a document, change status, and create and assign a keyword to that document
Inspector pane (keyword view activated at bottom) shows keyword and status accurately
Place cursor in middle of text, Ctrl-K
New document shows no keyword, and status shows as x No Status

Yeah, I can confirm that. I think this is a known issue, but I’ll check the bug list and make sure.

Snapshots, however, should not be split out. They are not considered meta-data. They should remain attached to the original item.

Wow, really? All metadata always lost on split? That makes Scrivener 99.9% unusable for me, and I spent about 6 hours setting up several projects in it.

The manual being very clear about that, and maintaining and not losing metadata obviously being a key feature, this seems less like a bug and more like something that shouldn’t be in the manual. In fact, what’s the point of metadata if it is lost by using a key feature?

Sorry for ranting. If this is the case, obviously I’ll be asking for a refund, and I’ll just have to rationalize losing the six hours. However, I still find it hard to believe this is so. Creating metadata and then wiping it out every time something is split? I’m still hoping to hear something else here…

(Also, thanks for clarifying—that’s why I specified specifically that I wasn’t using snapshots.)

In case that sounds a bit dramatic :slight_smile: it is unfortunately the case for me :frowning:

I’m creating technical documents from mostly imported materials. Process is to tag all imported data for source, type and status, split it up to match its source outline, and then start adding keywords for priority and sub-project, splitting key parts out as needed, recombining via search and sorting into filtered key projects and then edit and add new material as needed.

Obviously, if every split destroys all metadata, I’ll end up with next to nothing left…which is what I see has happened, particularly after setting the keyword for source before making dozens of splits to the imported document. How in the world can it say in the manual that splits retain metadata? Was it ever true?

I’m not sure if perhaps we are talking about the same thing? I’m referring to the Documents/Split/ sub-menu. The reason why I’m a bit confused is:

  • This isn’t typically considered a crucial feature, much less the meta-data carry over. Agreed, it is awfully handy when you first start using the program and have a bunch of long documents you wish to convert over to the “Scrivener way” of doing things—but after that transition it’s one of those things you use fairly rarely.
  • It’s pretty easy to bulk manage meta-data. To add keywords to a bunch of documents, select them all, then drag the keyword from the Project Keywords window onto the selection. Done. To set a bunch of items to a particular label, right-click on a selection of items and choose the label from the contextual menu. Done. Same goes for status. The only forms of meta-data that are a bit slower in this regard are the synopsis and notes, neither of which tend to be used in a bulk like manner.
  • It isn’t actually lost. It’s still attached to the first item you split from. So it’s not like you are losing assignments you’ve made, they just aren’t being applied to new items split off from the originator.

I’ll definitely agree with you: it’s less convenient if you have a chapter, say, that you’ve split into eight scenes. If meta-data carried over you’d be done, but with a few clicks you’re back to where you wanted to be. Less convenient, as I say, but not deal breaking.

So this:

Isn’t quite the case. It isn’t wiped out or lost, it’s just not applied to the new stuff. That could be a matter of semantics; I say it isn’t applied, you say it is lost, but either way it can be easily applied to the new stuff in a matter of seconds.

So given all of the above, I would recommend you import, split up, and then worry about meta-data. Assigning it in a bulk fashion, selectively, to the items that require it. You’ve actually not added any extra work in doing that. You’d have to apply that meta-data either way—this way you are using the bulk tools to apply it, rather than the fine-grained tools.

It should have been true, and in fact I thought this was fixed some time ago. It either regressed, or the fixes never happened. Like I say, I’ve checked the bug list and have added this as an issue for the developers to look in to. What likely happened is, when I wrote the original draft of the manual for that section, I asked about whether it would be fixed and then left it in with the assumption that it would be. There are a lot of places like that, and in most cases things worked out okay, but sometimes things didn’t get addressed and so the manual is looking at a spot where the software should be, but isn’t. Given how huge the manual is, and I’m the only one taking care of it, yeah… these things happen.

But again, like I say, I don’t quite follow why this is such a huge deal breaker because it is so easy to correct, and splitting is the sort of thing you do in the first few months of using the program. Once you start drafting and working in the program from day one for new projects, I find most people rarely split at all. Everyone is different—you might anticipate needing it forever, and far be it from me to say you can’t—you can. I’m just saying, this isn’t something that I had pinned at the top of my to do list as a massively crucial thing that needs double-checking.

Thanks for such a complete and thoughtful reply. I certainly see your point that most people don’t use information in small chunks like I do. I hadn’t considered quite how different my approach is, and assumed that non-persistent meta-data (that you can admittedly work at reincorporating) would be a more major issue for most people. It really is a major issue for me, though.

I actually use databases to write, so that I can use my custom setup to apply a parent’s metadata to child text when it is split out.

For example, three current project(s) require third parties to go through and pull out hundreds of pieces from existing information, and each piece needs to retain their status and source as they are broken out, while adding keywords to facilitate combining into new documents. I began creating training materials for using Scrivener when I realized the metadata issue.

Most of what we do after collecting research and adding in various transcriptions (a lot of it from me creating voice notes throughout the day) is to break things into every smaller chunks. It would be a nightmare to ensure the correct metadata persisted—we would spend a lot of time moving metadata around, proofing metadata, with no clear way to restore if we lost it (where did something come from that lost it’s metadata? No way to tell. Can’t use a quote when the attribution is lost, can’t get all parts into the document if some lost their label, etc.)

It’s only after all the collecting and breaking into small labelled chunks (and filtering and sorting) that we start writing. So while we could spend the time and accept some errors in reapplying metadata to chunks that lost when split from their parent, Scrivener becomes a cause of metadata work and problems, rather than a solution to metadata persistence. And I need to never lose the metadata.

So perhaps my only real quibble is that Scrivener doesn’t do what the manual says it does, and what I bought it to do. It sure was an exciting program for a bit there though!

If there’s some hope metadata could be made to persist in the very near future (preferably without a lot of follow-on bugs that probably can’t be promised) I would keep Scrivener. I have used it for around 18 hours of writing so far, and over 90% of that I can export accurately without having to deal with the metadata problems that have been created in the writing.

I’ve read everything, and started my own manual of how it works to teach it to others, so it’s a big disappointment personally. My social media links get clicked on around a half-million times a month (I’m @TweetSmarter on Twitter, for example), so I was looking forward to talking it up a lot, but no worries, I won’t be saying anything bad about it. I was just excited that I was going to be able to talk it up a lot—I love finding solutions and talking about them.

Have you considered using Documents/Duplicate instead of Documents/Split? There may be more editing involved depending on how you were splitting but it does carry the keywords forward.

Thanks for sharing a bit of how you use the program. I always find that stuff interesting. I wouldn’t say that most people do not work in small chunks. It’s probably safe to say the split is roughly equal. There are many people who prefer to work in larger chunks because that is how they write—they don’t need anything smaller, or that is what they are used to, working in Word in chapter length files. Then there are those that have grown used to how adept Scrivener is at handling smaller pieces, or those that take a liking to it immediately like yourself. I myself write in very small pieces as well. Larger works can number way up in the hundreds, even approaching 1,000 pieces in the draft outline, many of them only a few paragraphs long.

I think the main difference is less that, and more how people assign meta-data and how they end up with small chunks in the first place. I’d say most people who are starting a new work in the software end up with pieces that don’t get split or merged at all. They build pieces at the size they need in the first place. Just to apply this to an example, when I build out a new section of a book, I tend to either create a series of cards for each topic I know I need to work on, knowing ahead of time that each card that I create is for one atomic piece of information. If I need more pieces, I make more cards. When I’m working this way I have no content yet, so splitting is wholly irrelevant—I just have bunches of empty cards. The other way I tend to work is when I’m actually writing without a pre-existing structure. Then organising takes the back seat. When I’m working that way, when I reach a point where I’m not longer discussing the topic at hand, I create a new card and continue writing. Either way, the concept of applying meta-data is particulate to the pieces I create, less so than being in batches. Batches do happen, but when they do I assign them after I’ve created a bunch of pieces.

Splitting implies that (a) you either already have a huge chunk of text you need to break down or (b) you don’t break into new cards as you write. I’d say the latter case is something a veteran user might run into more often if that is how their mind works. But give how pieces = atomic topics more often than not, the concept of having meta-data carry over is, for me anyway, more often an inconvenience. Why? Because the reason I broke off to a new card is because I’ve shifted to a new topic—hence, new meta-data. Of course, what you are describing sounds entirely different to what I’m describing. It’s a usage I did not anticipate when considering the importance of this feature. To me, carrying over meta-data is one of those very small conveniences, gravy if you will, not a core component of splitting, let alone of the whole software platform.

Another difference between the workflow you are describing and how I think most people use the software is the importance of the meta-data. I don’t think most people depend so heavily upon it because their structures are a lot more simple than what you are describing, which is almost more like a database. Most people are just putting down 25 chapter files in their Draft, or maybe breaking that out into scenes or sections as the case may be, applying a little meta-data to pieces as necessary. They aren’t using meta-data as a tracking system for origination, bread crumbs, and so on. That’s really cool what you were doing, wish it worked out for you.

And for that I do apologise! Like I said before, there are so many different topics in the manual, and it is literally impossible for me to completely audit the software constantly (keeping in mind I have to do this for both platforms!) so sometimes errors like this creep in.

Yeah, it seems to me you’ve taken the software in a very specific and specialised purpose and unfortunately without auditing whether or not is was working before getting hundreds of items down the road. That’s really frustrating for me because I try to avoid that situation when documenting the software.

Well, I hope you can find a way to salvage the problem with the bulk meta-data tools I described and maybe find a way to make the software work better for you. I’ve done what I can to elevate the issue to the developer, and I’ll be fixing the manual as well. That’s the most I can do. I’d appreciate if you are just too frustrated to continue working with the software though.

I thought almansur’s idea to duplicate rather than split sounded really worth considering.

You’d stop losing metadata, and in fact would gain the chance to tune each fraction to have exactly the context(s) and metadata you want.

In terms of real-world meaning, this may often be very appropriate. Doesn’t information often separate with overlap of contexts and frames, rather than splitting down some imaginary line, so that you actually need this ability?

I’m thinking also that some original information may need more than two segments, which a duplicate and prune approach supports, especially as frames will have even more tendency to overlap.

I make something like this pattern in all the information I collect, whether with Scrivener or tagged Evernote items, or by other means-- in fact tags beautifully illustrate the principle at least on my work. I wanted them for years before they were possible in anything except an obscure piece of Lotus software which James Fallows used to write about in his salad days.

Maybe this is actually what you want to teach?

Regards,
Clive

Thanks, narrsd and almansur, for emphasizing the duplication feature idea. By creating a keyboard macro to add a control string (such as “~~”) at insertion point, duplicate, then remove all before and inclusive of the control string, I can re-create the missing split with persistent metadata feature.

However, I noticed that the supposed keyboard shortcut for duplicate, Ctrl+Shift+D does not work (it opens scratch pad instead). It’s I believe the third keyboard shortcut I’ve tried that didn’t do what it was supposed to do.

I can of course use alt d-a-o (then enter Ctrl-tab Ctrl-F ~~ enter escape) to find a “~~” control string and then Ctrl+Right and Ctrl-Shift+Home and delete to remove the parent data (and there’s probably an even more elegant way), but I’m starting to get concerned here.

I almost always encounter some bugs in software. Years ago I provided support for Microsoft desktops, and there are a whole slew of wrinkles in MS software. I know how to prevent Word outlines from expanding or collapsing after a save, for example. A dozen or so very familiar issues with Outlook, etc.

But I’ve never encountered features documented that haven’t worked in quite some time or multiple mis-assigned keyboard shortcuts while I was still learning the software. What else am I going to find? I’m concerned about committing most of my workflow to a program that feels closer to an alpha than one out of beta. Scrivener seems to need an audit of some kind.

That said, I love what the program may be able to do, and have to admit I’m as yet undecided if I want to take the risk of committing to it and seeing what else I may encounter.

I expect I’m most likely to find that I’ll have to create my own shortcuts (which I tend to do anyway) to be the the power user I would like to be rather than that the program doesn’t perform other essential (to me) functions as advertised, but still, it gives me pause.

Yes, thanks to both of you for that helpful advice. I meant to bring up duplication as well and forgot to draft that in—I suppose in part because it sounds as though OP has already done a lot of instantiation already using Split and has given up on the software.

I’m a huge fan of tracking originators when building nebulous non-linear structures out of concepts. In fact I like to track the Prime originator as well as all incidental originators along the way so that any single piece of information can be traced back along its ancestry, no matter where it ends up in the network of info. I use unique identifiers for this in my own work. Each atomic piece is identified uniquely, and so anything instantiating from it has its number in its meta-data. Thus the discovery of a lineage can be found at its end, middle or beginning and unearthed completely with a single search. Likewise, and just as easily, forking chains can be wholly recovered simply by searching for the central Prime they all came from.

And I do use duplication for this, because I rarely find that I want to subtract data from the original node. In fact I am, you could say, ideologically opposed to subtracting data from the original. If I want to subtract data from a data node, I will extrude a duplicate and then subtract from the duplicate, binding it back to the Prime as a revision of it and marking the original as inferior. I prefer that to marking the new one as superior (and thus representative of the strand of versions), because it requires less retro-editing of meta-data later. You need only mark the things that are definitely inferior, and doing so is in a way a kind of deletion. If a new superior comes along, I can mark the older one as inferior if necessary but doing that only requires one action, whereas marking a new superior would take at a minimum two actions to demote the old superior node.

Eh, I’m rambling at this point. :slight_smile:

Anyway, I like tags for some of this. Since I use unique identifies instead of themes for tracking, tags aren’t quite the right tool for the job, but they work great for other things (like the inferior bit).

I ran into something like this as well and found there was a registry bug—I think if you used the betas (but maybe not exclusive to that). Try deleting the [HKEY_CURRENT_USER]Software/Scrivener/Scrivener/Options/Shortcut key folder with the software closed and re-open. That cleared up some errant shortcuts for me.

RE: Resetting keyboard shortcuts, that restored Ctrl+Shift+D but the correct Scratch Pad shortcut Ctrl-Shift-► no longer works now.

And that’s in
[HKEY_CURRENT_USER\Software\Scrivener\Scrivener\Options\Shortcut]
on my machine, but removing it worked for restoring Ctrl+Shift+D.

Thanks :slight_smile:

Eeks! Edited my post so nobody does damage. (Sorry on a Mac at the moment and was just pulling that from grey matter).

I think the scratch pad shortcut is Shift-Ctrl-Alt-P now. That’s what it shows in my menu after the reset, and it works for me.

Thanks, Ioa. That indeed works. I’ll be sure to check the manual against the menus if I have problems in the future and try whichever I haven’t tried.

E.g. p. 231 in the June, 2012 Rev 1.2.1-01b manual (section A.7 “Tools Menu”) still gives the scratch pad shortcut as Ctrl-Shift-► (I tried both right arrow and > for the ► character, but I assume it means right arrow).

I’m just using Alt-1-6 from the number pad (under Windows) to create the ► character to approximate what it shows in the manual, as the manual seems to use a different unicode character that doesn’t copy/paste to this forum successfully.

Yeah, like I said I got snagged by the same problem you did. :slight_smile: I didn’t find out about it until after the manual went out. I did also catch the problem with the menu separator creeping in. I don’t type those in by hand while writing, but use ‘/’ instead; a script looks in all blue-text and converts slashes to the triangle. Thanks for pointing these out though.

Besides ► Alt-16, my other favs:
• Alt-7
· Alt-0183
… Alt-0133
— Alt-0151

:wink:

I’ve created an AutoHotKey script to duplicate the missing functionality, and shared it here. I’ve also included some other AutoHotKey scripts from other Scrivener users, and asked folks to share any they may have.