Devonthink and Scrivener collaboration for academic workflow

This is partial complement to the subject found in this thread:
https://forum.literatureandlatte.com/t/opml-export-the-others-do-it-why-scrivener-doesnt/13213/1
about an eventual support of OPML files in Scrivener. Maybe much people know that Devonthink can import an OPML file as a folder/documents structure, so it’s possible to get in DT a clone of a folder/documents structure in Scrivener. But this is not completely true:

  • The last level of the OPML results always in a text file, not a folder;
  • Scrivener actually doesn’t export to OPML (it’s possible to have an OPML file with some trick involving apps like Novamind… please, refer to the preceding link if you want more about).
    The purpose of having two structures cloned in both apps is to let Scrivener only for the work-out of texts documents and Devonthink to keep, maintain, search, link, etc. the huge bunch of heterogeneous files (text, pdf, media, snippets, etc.) related with the research.
    This is a only my personal preference. In my field, the Research folder in Scrivener, yet very useful and easy, cannot compete with the great facilities of DT in information management. Any way, I recommend to give it a try, cause it has really nice features for quick and light information management.

This is what I do with both apps:

  • I outline my text structure in Scrivener. For outlining Scrivener is much better then DT: user interface is a delight, and the outline can grove out just from creative text and thoughts brain-storming sessions, that is natural for me (I prefer to think in sparse items before and in structures after, DT obliges me to the opposite…). Devonthink allows to compose text, even in full screen, but Scrivener is much better.
  • Once my outline is complete, I convert all its levels to folders: this is the key move. For Scrivener docs and folder are quite the same, but outside folders are real folders and docs are real docs,… follow up.
  • In Scrivener I export the selected folders (ex docs) to Files… I get a folder in my desktop, containing all the folders of my outline (in other words, all my outline items appear as real Finder folder items).
  • In Devonthink, I import the root folder of my outline… That’s all!

In the following step I can decide to convert back the folders to docs in Scrivener, if needed. But now, I have a container structure ready to work for me in Devonthink. This is the basic principle: I’m always on the Scrivener side, my research material is always on the Devonthink side: both two we collaborate, when one calls to the other… I can “feed” every folder with the relevant material from DT powerful searching, ordering, linking features, and compose my own text in Scrivener docs, merging, citing, summarizing it, just right where I need. No fuss about what is mine and what is not.
If needed, in some case, I can feed DT folders with my own material out of Scrivener too, for some kind of recursive/auto-citing/auto-summarizing purpose. I’ve found this useful when working with translated material, for example. This is done simply dragging and dropping the item between the windows apps (it doesn’t work with Scrivener folders, only docs). UPDATED: The last is a mistake, drag&drop of items out of Scrivener is not working at all; it works only between two Scrivener windows. You have to export them as files. Any way I keep the mistaken phrase as a wish for a future update, maybe? :unamused:

Last useful tip: some people in the forum have yet remarked that every Devonthink item can be copied as a link to the clipboard (Edit>Copy Item Link… or Cmd-Alt-Ctrl-C) and pasted as clickable link in Scrivener.
It’s a world of powerfull possibilities!

Thanks for sharing your workflow! I also use both DT and Scrivener and I think the tip for exporting the Scrivener folder structure to DT might be really helpful for me.

I’m just wondering–I understand you to be saying that you drag Scrivener documents directly to DT? I would love to do this but it has never worked for me–I always have to export them from Scrivener first. I’m curious about this, did I misread you, or am I missing something really basic? :smiley:

Thanks again for the great post.

Bluebird wrote:

Bluebird, you’re right, the mistake is mine: really it seems direct drag&drop item from Scrivener to Devonthink not working. Maybe, in my trials, I was dragging item between two open windows of Scrivener, believing one of them being in DT… Ouch! I’m sorry for the inconvenience.

Any way, I can suggest to let the Devonthink Inbox icon appear in the Finder sidebar: in menu Devonthink/Install Add-Ons… check “Global Inbox in Save Dialogs”. When you export items in Scrivener, just choose the DT Inbox icon as the destination, and then move the resulted files to the relevant folder.

I see there’s an active thread on this forum on the same subject:
viewtopic.php?f=15&t=12702&start=0

Hi - I’m curious about the choice to use a DTP folder structure for supporting research materials. Why not just put all supporting materials in a single DTP project folder (or even in a research support materials library) and use metadata + simple search and/or smart groups to locate what you want whenever you want it? Thanks, Keith

Thanks, Keith for your interest, I love this kind of methodological questions. I try to explain my case. I’m musicology researcher and teacher, and collect a huge quantity of material about early music: pdfs, scores, academic literature, books, web pages, etc. In DTP I keep also my own material (summaries, notes, mindmaps, more or less completes documents, snippets, etc.). All this is ordered in folders and, eventually, tagged (this is quite new in DTP and I’m not completely convinced about, I tend to keep myself to folders). This is my personal library, contained in a DTP database. It’s quite general, but it comes from years of study, research and collecting. It’s my treasure, and I keep it regularly backed-up (!)
When I have to actually write something about a specific subject (a paper, an explanation for my classes, a thesis, etc. i.e. something complex and structured, which need a working-out), I have to extract from my DTP database the relevant information. In this moment I use the powerful search capabilities of DTP and eventually organize the material in a more focused structure, related with the scopes of the particular activity or sub-activities. I use here replicated items, one-use folder structures, informal outlines, etc. It’s no matter if I keep this structure for the future, as my general material stay in its place.
It’s here where Scrivener comes to my scene. When I get the content of my text more or less clear, I outline it in Scrivener, defining its parts and purposes, following the conventions of the required style or of the particular circumstance. Scrivener helps me with its synopsis, metadata, views, quickview panels, labels, etc. Once finished my outline, I feel the need to reorganize the relevant material in DTP in a structure adhering every part of my Scrivener text, like a shadow. It’s like to have two parallel bodies: one is my actual text, and the containers (folders) in DTP are the “memory” or “background” (all I have read about, all I have speculated about, etc.) for every part of my text.
I keep my Scrivener project open on the right and the relevant part of my DTP database ready to the left. From here it’s all a matter of traveling from one app to the other, summarizing, more outlining, composing phrases, citing, linking, using copy/paste services, etc. And I see how my text comes to life in Scrivener.
I have created some auxiliary folders in Scrivener, out of the draft folder. One is a Casual Thinkings folder, another an Agenda folder, another one a Journal folder. I try to keep track of the evolution of the project, and to fix an agenda with its “next actions” in red. If I have to leave the project for a while, I have to be able to resume it looking at my Agenda folder. If I need to schedule something, I actually copy and paste relevant text from my Scrivener agenda to Busycal to create dated tasks: my schedule is automatically synched to MobileMe and Iphone. But cloud synching, iPhone, etc. are huge other matters…
I have to say also that all my bibliographic information is kept in Sente and linked to the library in DTP. Following a metaphore I like, Sente is my index card archiver, DTP my library bookshelf, and Scrivener my work table. I try also to have all the items hyper-linked, and to use clicks of the mouse to go from one app to the other, if needed. The three apps allow this, and maybe this is one of the reasons for I reached them and discarded other ones.
What are the benefits:

  • I use the best from every world: DTP for collect and search, Sente for indexing, Scrivener for writing (well, using Scrivener I can really WRITE, non only write);
  • I don’t alter my information general structure: the bookshelf doesn’t change following the particular researches;
  • I focus myself on the actual work: the Scrivener project is my table, the DTP clone structure is all I have taken temporarily from the bookshelf and it’s lying on my table. With Sente I dispose of all my bibliographic wells in one place and written only once for ever.
  • I prefer not to use the Research folder in Scrivener in order to keep my creative writing space clean, and keep really strictly separated the source from my redactions. In DTP too I keep well separated my own product from the sources. But this is really a personal habit and maybe it’s related to the actual material I use.

Hope it helps.

Because DTP’s AI works better if you give it some information about what items belong together.

And also because the “project” distinction becomes less and less meaningful as time passes, while the support materials themselves may be valuable for years.

Katherine

It occurs to me that tags for individual projects would be helpful here. Using a succinct tag to identify all of the research that you deem useful for a given project would allow you to avoid duplicating documents in Devonthink, and the “AI” in DT might even find other materials that might belong to that project based on the commonalities of what you have tagged. When you are done with a project, you can either keep the tag for that project, or delete it from all your documents to keep your tags “clean”.

It’s similar to your current methods, but instead of using folders when you mine your DT database, you use Tags (which I assume you are not using otherwise). Tags have the advantage of making it clear that a given DT document is already part of your new project, or not, so you don’t end up with 3 copies of the same document in your project’s folder structure, or miss-remembering that you already added a copy when you haven’t.

The way this all sits in my mind seems similar in nature to Robert’s comments.

Katherine, I understand your comment about the AI, but I wonder if it might not be better for the user to be putting additional meaning into the documents via metadata directly rather than depending on the programmers to do that for us. That is, the AI engine takes the folders to be tags of a sort and determines relationships based on those relationships (at least that’s what I think is going on). Although that it is a neat idea in the absence of my direct contribution of tags, will it capture all of the related meaning and connections that I’d like it to? It may capture additional things, but can’t it do that based on my tags rather than my folder structure? And, my guess is that the whole process will be deeper and more rich if I get good at tagging than it was when I managed folder structures.

Even if you have the same person organize the same information into a folder structure at different times, he/she will at least sometimes come up with different results - choose hair color as the top level folder and height as the subfolder vs. height as the top level folder and hair color as the subfolder - whereas there really isn’t any additional meaning being derived from the work done of picking and arranging a particular structure (I don’t know if the AI engine thinks there is additional meaning in such a case). Instead of taking the folder structure approach, what if we just add hair color and height as tags to each file stored in a single bucket and then search for all the people 6 feet tall or less with brown hair when that list of information is of interest.

As we get better and better at determining metadata and implementing search processes and tools, our results are more efficient and effective than the previous methods that relied on time consuming management of folder structures. And, I suspect that the overall situation is better than if we rely increasingly on AI engines in the absence of better metadata habits. I don’t know exactly at what level of complexity (# of files, # of organizing concepts, etc.) this happens, but it seems to me that when a DTP database gets to a certain level of complexity, the old ways will be too clunky and cumbersome. I happen to think that we are either there or close enough that we should switch. Although I’m not a programmer, it seems to me that it is the same thing programmers went through when the level of complexity of programming tasks and teams led to the evolution of thinking resulting in going from procedural to object-oriented programming.

To bring Scrivener back into my comments, I’d like to sit down at my computer and be able to very quickly call up whatever resources I want to work with - a website, article, application, … , or Scrivener project. And, it seems like the process should be very similar for all of those things - and, ideally, slick like LaunchBar.

As always, I really appreciate any feedback!

Keith

Agostino’s system is similar to, though not the same as, one recommended by the historian Steven Berlin Johnson. He too suggests keeping a “shadow” research structure in DT*. But I think Agostino’s idea of exporting a Scrivener folder structure to use as a container hierarchy within DT is an interesting and useful refinement.

  • Edit: in response to Keith’s post above, the tags versus folders debate will probably always be with us (although perhaps it’s a false distinction). My understanding of Steven Berlin Johnson’s reasoning behind the building of his "shadow’ structure in folders in DT is purely to save time. The structure of the argument can be perceived in the structure of the research; writing then almost becomes a matter of connecting the argument manifested by the ordering of the pieces of research. Time is saved. My guess is that Agostino thinks similarly. It’s my view too.

That doesn’t invalidate tagging, and searching on tags, to build a pool of research from which to construct the argument in the first place. But maybe it depends on your subject-matter and academic discipline?

Yes, I really like the idea of a shadow structure in DT. I’m going to test that out. If I’m working on chapter 5 of my dissertation, I can see that it would be really handy to have a place in DT that was not only for chapter 5 stuff (I do have a chapter 5 tag) but also chapter 5, section 1Aic.

I currently have a shadow structure, more or less, in the Research folder in Scrivener. But I do find that things get a little cluttered; I also feel the need to distinguish better between my own notes and my source material; and once I’ve pulled research material from DT into Scrivener, the DT “See Related” function is no longer at my fingertips. So I think the method Agostino is suggesting has a lot of advantages.

I can definitely see pulling in things from DT more selectively, one at a time, to use on one side of a split in Scrivener. But right now I have a bajillion notes in my Research folder and it’s not as easy to manage them in Scrivener as it is in DT; the two apps just have different strengths.

I’m not a big tagger; and I guess you could do the same thing with tagging, but I don’t feel too excited about setting up tags for chapter5sectionA1cii, or whatever collection of tags would be specific enough to get that subsection of material grouped together. Importing the folder structure I’ve worked out in Scrivener seems like a really nice way to go.

I’ve also done something vaguely like this in my last project (roughly mirroring the structure in DT and Scrivener). I’m sure a certain amount depends on what sort of research and writing one is doing, but in my case I found that having material in groups (folders) in DT helped me to think about the subject. As my understanding of the subject became more sophisticated, I noticed that the structure of the group (folder) hierarchy became more complex and altered to reflect my development in thinking. Put simply, I was moving items around as I came to see how they linked to each other. This sort of development is also facilitated by DT’s ability to use replicants (items can appear in more than one place if necessary). For me, just being able to find things was not what made DT interesting – it was being able to use it as a tool to understand the subject, or topic I was researching. This is one reason why I would tend to have a separate DT database for each project.

Martin.