With all the various ways to search for the presence of a property in Scrivener, is there any way to search for the absence of a property/characteristic/word? For example, is it possible to find all the documents/cards in a project that do not contain a particular property? If so, how? Thanks.
Not that I know of (other than the obvious: performing a search for…, and then refining it). As far as I know, you’d need a more complex search feature with a Boolean “NOT”, such as a dedicated data manager like DevonThink has.
H
OK, thanks. I test-drove the DevonThink demo awhile ago, but decided it was overkill for my work at the time. So, I never got around to purchasing it. I seem to recall that it has the “Search for Absence” (aka Boolean NOT) feature, but before going to the expense and trouble, it would be helpful to better understand the workflow that would be involved between Scrivener and DT. Would DT be able to search all of the individual document files contained within a Scrivener project merely upon importing the Scrivener project (project.scriv file) into a DT database, or would each individual document file have to be imported separately? Thanks.
To be honest, I have no idea. I bring material into Scrivener from DT, not the other way around. But you could always test-drive DT some more to find out. Bear in mind you don’t have to import folders into DT - you can index them externally.
And there must be other Mac applications you could try that use Boolean search terms.
H
At what level are you looking to “not match”? Then you are too nebulous on the “property/character/word”. The idea of a search is that you are looking for a specific something as a specific level. There are tools that will do this, but if you want to include scrivener meta data you may be out of luck.
To a degree. One could export from Scrivener with full meta-data options enabled, to text files, and then operate on those files with much more finesse. It wouldn’t be something you’d want to do every day, though.
OK. Thanks, all. But it’s still not clear to me whether Scrivener has the capability to search in the manner I wish, or whether there might be some other app (such as DevonThink) that would be useful to simplify the accomplishment of my task. Perhaps such lack of clarity is attributable to my rather ill-defined description of what I’m trying to accomplish, or to some gene missing from my DNA that leaves me unable to see what’s hiding within Scrivener or to understand other solutions being offered thus far. On the off chance that it’s the former (ill-defined description) please bear with the following elaboration of the problem, its context, and some specific questions for which I seek assistance.
First the context: The digitized data that has found its way into the documents within the Scrivener project file in question has been derived from sources already on my hard drive, sources on the www, sources scanned in, and cards/docs generated from within the Scrivener project itself. Pretty typical, no doubt. To complicate matters, though, my muse rarely stands still long enough for me to accurately categorize/tag each index card, or synopsis, or note, or other type doc as they are created and deposited into the stew (chili??). So, my project, as it stands, consists of thousands of index cards, synopses, and doc fragments at varying levels of categorization – mostly created within Scrivener – and each competing to find its way into a less-than-finished framework of story lines. Still rather typical, I suspect.
Now, the problem (simplified somewhat in an attempt to be clear) is this: About 20% of the cards/docs in this project were fairly easy to assign to scenes/events and other logical elements of story structure. However, the remaining 80% (about 4000 cards/docs) are much more problematic since it is not yet clear to me where or how or even if they fit into the final mix. So, at this point my challenge is to figure a way to sort/sift and winnow them down without overlooking or discarding whatever hidden jewels they might contain.
It occurred to me that one way to to proceed would be to go thru remaining 4000 (+/-) cards/docs in the project one by one, adding keywords or other searchable annotations that later would allow me to use Scrivener to sift and sort as necessary. But not only would this be a pretty daunting task, I can not know beforehand all the keywords that might come to be useful. Rather I expect the annotation rubric will emerge in the process of poring over the stew (meaning that annotating/sifting/sorting/winnowing could well become several iterations of daunting tasks – ugh!!).
So, I’m wondering whether there is some other way that Scrivener might be used to minimize the chore. It would be helpful if it could search and find, say, all docs which include any mention of character Green, set them aside into their own “sub-stew” folder, AND simply allow me to exclude them from the remaining 4000 – then scour this smaller set of any mention of setting, say, Waverly Park, setting the results aside and removing them, and so forth. This would allow me to try several easy and simple iterations to see what might take shape, and easily start over if unhappy with what it produces. Unfortunately, I have been unable to figure out a way to do this in Scrivener. Can it be done? If so, how?
Alternatively, it has been suggested that, if such a task cannot be accomplished in Scrivener, I might benefit from using a dedicated data base organizer app such as DevonThink, which has the capability to do sophisticated searches including Boolean NOT (a.k.a. “Search on Absense”). Pursuit of such an alternative makes sense to me only if it is easy to get the docs/card files which were created in Scrivener into DT (or other such app) for it to do such a search, however. Being only marginally familiar with DT, I inquire of those with greater facility with it to advise whether importing the Scrivener project file (project.scriv) into DT would allow all of the docs/cards within the project to be thoroughly searched by DT, or whether each file would have to be imported separately before DT could properly perform such a task?
Many thanks. Sorry for the length.
Scrivener can not do this. I do not know enough about devon to comment.
DevonThink cannot see inside a Scrivener package file. (I just checked.) You would need to export all of the files from Scrivener, then re-import them to DevonThink. Not difficult, since both programs can handle large numbers of files as a group. However, since I haven’t done it myself, I don’t know what would happen to the Scrivener metadata during the transformation.
Once you got the files into DevonThink, everything you’re suggesting should be well within its capabilities. There are some very helpful people in the DevonTechnologies forums who’d be able to provide more specific advice.
It would also be possible to export all the files out of DevonThink and import them back into Scrivener once you’ve organized things to a point where Scrivener is again the best tool. However, you will see some loss of DevonThink’s metadata when you do that.
I wouldn’t advise a workflow that requires you to move large volumes of data back and forth between the two programs on a regular basis.
As a side note, TinderBox also claims to be able to handle this sort of task. It’s more visual than DevonThink, so might be a good choice if you’d like to spread index “cards” out on a virtual “table.” However, I haven’t used TinderBox’s advanced features enough to say exactly how you would apply it to this kind of problem.
Katherine
If I understand your problem correctly, I believe this is something that can be done with Scrivener. The problem is basically this: you have a large, uncategorised data pool that you want to selectively and gradually categorise, and as you do so you want to eliminate those items from the uncategorised data pool. Once you look at it that way, you can approach it from a positive search mechanism, rather than a negative.
Yes, this could be done in Tinderbox, or DTP, or a number of other data analysation tools, but in each case you are going to have a problem with meta-data getting lost, and since this is a problem which fundamentally revolves around meta-data, I don’t think you’ll gain much by using an external tool. We don’t yet have a good common standard for meta-data (though this is something we are getting close to) like we do for say, rich text. It used to be applications couldn’t share formatting between each other, but with RTF we can now do that. There really isn’t anything out there yet that works for meta-data. OpenMeta is getting close, but not everyone has adopted that and there is no indication that it will certainly become a standard. Scrivener doesn’t use it anyway so even if you found an application that does use it, it wouldn’t help here.
Since we want a Scrivener project that makes some sense of all this, we should stick with Scrivener as best we can. The first thing to look at is: which Scrivener meta-data method is the most transparent, and by that I mean how easy is it to set and see at a glance. I would say labels are the best for this because not only can you set a label to a thousand items at once, you can also rapidly scan through cards and Binder items and see their label by using the “Tint Icons with Label Color” option in the View menu. You could use keywords, or status, or even annotations typed into the documents, that is up to you, but the rest of this explanation will presume you are using labels.
Once the bulk of the categorisation is done, you can easily switch to another mechanism and free up labels for something else (which you most likely will want to do) because searching by label is easy. Another plus for this task.
So, the first thing to do is open the label manager (Cmd-Opt-,) and see what we have. You’ll note that, in a default blank project, “No Label” is the bold entry which means it is the default. When you imported everything into Scrivener, assuming you didn’t change that already, everything should already be set to “No Label”. If you started with a template, or have already done some modifications here, you might have different results, but chances are the vast majority of your project is set to the default label. If it is something other than “No Label” then substitute whatever that value is in the rest of these directions.
Perform a project search in the tool bar for “No Label” and set the search constraint to “Label” in the magnifying glass drop-down. You should now have ~4,000 or whatever search results. Using the magnifying glass again, save this search and you’ll get a little purple folder in the Binder. When you double-click on that this same search will be performed again.
I suppose the first thing to do is exclude the ~20% you already have categorised. Open the label manager and add a new label “Ignore”. I’m not sure how you’ve done that, but do another project search for those items and once you have those (or a good chunk of them) in your search results, press Cmd-A to select them all, right-click, and set their label to “Ignore”. Now when you double-click on the saved search in the Binder, your result list should be shorter. In a sense, you’ve just done a negative search by using a positive data pool search. You could go ahead and devise a set of more indicative labels, but we are pushing for speed and efficiency and the main task right now is to cut up your project into large chunks that can later be analysed with a finer comb.
No doubt you can see where you should be going from here. For each large grouping, say “Green”, create a new label, run the “No Label” saved search, and assign items to it. They won’t disappear immediately from your search as that is just a static one-time search short-hand rather than a smart folder, but if you have the tint icon option set, this should help you as you work through the list. The next time you run the search they’ll be eliminated from it.
The rest of your job will be laborious, but I think it is worth mentioning that it would be laborious no matter what application you use. Something like Tinderbox or DTP could certainly help reduce the amount of manual labour by a bit, but not much. With this particular problem you’ve presented, there is going to be a large amount of human intervention since only you can read the material and decide what it should be categorised as.
Once you have large divisions set up, you can then do a search for them selectively and break them down further with keywords or whatever mechanism you prefer. Remember that with keywords you can use the HUD to easily assign multiple keywords to multiple documents at once by selecting the target cards with the shift and command key, and then selecting the desired keywords likewise, then dragging the keyword group to the multiple selection.
Again, I’ve used labels as the fundamental divider here, but you could use any mechanism you prefer so long as there is a positive match in the data pool that essentially says “I don’t know yet”. If you wanted to stick with keywords the whole time, just to give a brief example, you could drag an “Uncategorised” keyword to every single document at once (excluding the ~20% naturally), and then work from there with your saved search, instead of using labels. This would probably entail more work though as a label can only be assigned once, and all we really want right now is the ability to say “This document is now categorised”, while keywords allow a document to paradoxically be set to “I don’t know” and “Green” at the same time; you have to remember to remove the “I don’t know” bit as well as saying, “This is that”. What the category is is not important yet to the exclusion search, just that it is or is not categorised. Keywords may provide too much flexibility for this simple task, but will certain prove useful down the road for a project this large.
Many thanks, all! Your suggestions have proved to be very interesting and helpful! As I’ve read the responses to this thread I’ve been led to see and explore possibilities within Scrivener and without (such as DT and Tinderbox) that would not otherwise have occurred to me so readily – leading not only to an upgrade in my understanding of Scrivener, but significantly elevating my appreciation for its ability to respond when challenged. Thanks!
Amber,
Thanks for that fascinating post. It made me think about a basic problem I have: tagging. I don’t tend to use tags because I can’t conceive of names or phrases that make data useful beyond a specific project. Are there some good sources out there on the art of tagging? I’ll go look for myself, but if anyone knows of useful sites or documents, please let me know.
D
FWIW I can offer several basic categories that I find useful (expand or contract to suit your needs): character name, age, sex, race, hair color/style, story line, plot point, scene title, chapter title, sequence, act title, date, weather, day/night, time of day, inside/outside, event, issue, conflict, obstacle, location, POV, story stage (per Writer’s/Hero’s Journey), story through-line (per Dramatica), and the like.