Creating subsets of materials in Research folder: aliases, links, collections, labels, filters, ...?

[jandavid]

I have a large Research folder with a complex hierarchy and am trying to find a way to make material therein accessible by topic. Specifically, I’m wondering two things:

(1) Is there a way to get Collections to display the hierarchical structure of subfolders/-files?

(2) Has anyone found a way to replicate more or less the behavior of what is called an “alias” of a folder in Finder on macOS.

Here’s my scenario: The research folder has the following subfolders: Notes, Events and workshops, Conference Presentations, Application materials, and Archive. Each of these has subfolders organized by topics and appropriately labeled (and colored): So there are subfolders of Topic X under notes (for random notes and brainstorming regarding topic X), under events (events that had to do with topic X), presentations (that I gave on topic X) and so on. In turn, the actual content is nested under them in multiple levels of sub-sub-folders and -files.

I have labeled them Topic X, Topic Y, and so on, and each has its own color for easy identification. However, my research folder has grown super large and with many dozens of topics showing up in multiple places it’s become a bit unwieldy. So when I’m working on a paper or chapter regarding Topic X, I’d love to find a way to isolate only that material regarding Topic X.

It seems that Collections are meant for that purpose. But there is one crucial limitation: A selection doesn’t show me subfolders. Whether it’s a Saved Search or a manually created collection, it just dumps everything with that label on the same level and there’s no way to expand or collapse as I am normally able to in the regular Binder. And with hundreds of files/folders/subfolders it’s even more unwieldy.

Let’s say in the (regular) Binder under Research I have:

Notes
 |—Topic X
    |— Discussion with Justin  
    |— Brilliant idea on the train 
 |—Topic Y
 |—Topic Z
Events
 |—Topic X
    |— March 30  
    |— October 18-19
    |— November online workshop
 |—Topic Y
 |—Topic Z
Archive
 |—Topic X
    |— Outtakes from paper in journal A  
    |— Outtakes from paper in journal B 
 |—Topic Y
...

Then, when assembled in a collection, I’d love to see that hierarchy, but simply without Topic Y and Z etc. (bc. there are way too many topics )

Unfortunately (in the case of a Saved Search), if the label Topic X is applied to all files/folders, I end up with:

 Topic X
 Discussion with Justin  
 Brilliant idea on the train 
 Topic X
 March 30  
 October 18-19
 November online workshop
 Topic X
 Outtakes from paper in journal A  
 Outtakes from paper in journal B 

On the other hand, if the label Topic X is applied only to top level folders (Topic X), the collection looks like this:

 Topic X
 Topic X
 Topic X

Neither of those solutions are desirable.

In the second scenario, when I activate Scrivenings mode, the subfolders will show up in the editor – but depending on the subfolder-depth it is still too much information.

As an alternative I thought, perhaps there’s a way to manually build something like a collection in the regular Binder that would still show the hierarchy via document links or bookmarks – replicating what in the Finder is called “Alias”. But I couldn’t figure out how it might work. I tried to “Copy a Document Link” to a Folder in Scrivener, which gives me x-scrivener-item:///Users/… and then tried to use the “Add Webpage …”-function under a different folder, pasting the link from the Clipboard into the dialog box – alas it gives me an Error Message – I guess that’s not how it was intended.

So I’m wondering, are there any other solutions for my scenario?

Or perhaps others have come up with better ways of organizing their Research folders such that they don’t run into this problem?

Any ideas much appreciated!

[xiamenese]

It seems to me that you need to move your research into a knowledge base like Devon Think or Tinderbox, which will allow you to organise it how you want and set up links etc. between items. Both of them interface with Scrivener.

N.B. I don’t use either of them, but there have been many threads here, particularly concerning Devon Think.

Mark

[rms]

I use DEVONthink and Scrivener for my writing projects and I agree this sounds like a better solution for the OP @jandavid. Especially for the long term.

[fto]

I don’t really have a good idea either … maybe something like this: In addition to the structure in the binder, you can also create a (visual) structure with keywords.

This has the advantage that the two structures can be created completely independently of each other. And if you display the window with the keywords next to the binder, you have both side by side. Mm, Devonthink is better

@jandavid

I have a large Research folder

A note: Scrivener can handle large amounts of data well. However, there is one thing that eventually stops working satisfactorily when the number of documents becomes too large: the search function.

[SCN]

@jandavid

So I’m wondering, are there any other solutions for my scenario?

You’re running into the limitations of a hierarchical organizational system. It could be that a system that lets you organize hierarchically but also break free of its limitations would be a solution. I’m fond of Obsidian, but there are others, as well, some of which have been mentioned.

[AmberV]

@jandavid

Is there a way to get Collections to display the hierarchical structure of subfolders/-files?

There is a kind of way of doing that, though it isn’t specifically related to collections. Collections can help you get to the point you need to be at more efficiently, however, which I’ll come back to.

The trick is to use the Outliner view in the main editor, and to select only those groups that relate to the topic you are interested in (if you select any non-group items it will go back to showing a flat list). This is a similar approach to doing the same sort of group-only selection in Corkboard mode, where there it stacks each group’s child list in a sequence of corkboards. In most cases you would want to use the outliner for this kind of thing though, as you can expand and collapse sections to any depth. But, if your sub-topic stuff is all one deep as you depict here, maybe stacked corkboards would be even more efficient as everything is already essentially “expanded”.

So where collections come in: the theory is to aim for the goal of only marking the containers relating to Topic X, so that your search collections only gather groups. You can then easily and simply Select All within the collection list to work with them in the outliner as hierarchy.

The only real downside to this is that multiple selections are a non-persistent entity. They don’t store any metadata for themselves meaning the moment you dismiss one, your expansion states are lost. The only exception to that is history, where the multiple selections become semi-persistent, and thus have a place to store a little extra information. If you navigate to a child item and then hit the Back button, you’ll return to the multiple selection in the state that you left it in.

It’s not a huge downside in my opinion, since as noted above, the entities in the History queue are semi-permanent across sessions. Such “saved” multiple selections will remain so long as you don’t navigate them out of existence (always navigate forward, not back beyond it and then branching off; same as in a browser, you nuke your forward history when you do that).

Yes, there is other software, it’s mostly a matter of whether it is of a substantial impact to be able to do this, over all of the tight integration you get otherwise. The latter can be mitigated by external links though, and particularly software that also supports external links. But that is not the only form of integration that exists of course, there is also one-shot search results, shared metadata, being able to compile notes, etc. I very rarely “outsource” my research and notes, no matter how good other software may be at doing so, because I enjoy a rich network of information that is closely bound to the material I’m writing, too much. Where I make exceptions, the other software has to be truly exceptional at something Scrivener is not, like Logseq. When to outsource is going to be a personal equation.

[jandavid]

Thanks all for sharing your insights and thoughts.

I’ve been playing around with Obsidian for a while, for note-taking, task management, and other stuff. There are indeed some examples of using Scrivener and Obsidian together. My main concern with additional tools is to have too many tools doing similar things, too much information in too many different places. I’m a minimalist. And for a piece of text it’s not always clear at what point it ends being a “note” and becomes a paragraph in a piece I’m writing, i.e., at what point would it move from Obsidian to Scrivener?

For over a decade, everything has been living in Scrivener (Obsidian also wasn’t around when I started with Scrivener) and for writing my PhD it was the best experience ever. What AmberV writes totally resonates, having everything in one place, “a rich network of information that is closely bound to the material I’m writing.” And I do mostly like the hierarchical organization of the binder (hence my question here in the first place) – Obsidian could probably be set up to replicate that more or less, but moving over fully I fear I’m going to miss all of what I’ve grown used to in Scrivener.

But I also see that there are limits what Scrivener can do. Working mostly on one topic the research folder is still fairly consistent. But as my research has branched out and entirely new things have started, I’m running into the issue described above more and more. I guess I’ll have to experiment with a few options …

[kewms]

@jandavid

But I also see that there are limits what Scrivener can do. Working mostly on one topic the research folder is still fairly consistent. But as my research has branched out and entirely new things have started, I’m running into the issue described above more and more. I guess I’ll have to experiment with a few options …

This is precisely why I use DevonThink, not Scrivener, as my primary research repository.

[rms]

@jandavid

having everything in one place,

I fully agree with @kewms’s comment above. Scrivener and DEVONthink are my main tools for so much of what I do.

And from my long-held perspective, the “one place” is my MacBook – not the individual apps. That’s how modern computers are designed to work.

[I was introduced to computing using mainframe computers via terminals where the idea of “one place” made sense to me at the time – but contemporary computer interfaces with windows eliminates that way of thinking for me.]

[SCN]

@jandavid

My main concern with additional tools is to have too many tools doing similar things, too much information in too many different places. I’m a minimalist. And for a piece of text it’s not always clear at what point it ends being a “note” and becomes a paragraph in a piece I’m writing, i.e., at what point would it move from Obsidian to Scrivener?

Scrivener constantly surprises me with its capabilities. I completely understand your desire to contain your writing project in this one amazing writing system. I bet you’ll find a way to do that.

I, too, have been on a quest for one app to do it all, so your thinking resonates strongly with me. I’ve found, though, that I get further using best-in-class apps to do what each is best at. Of course, each of us has to figure out which apps those are. For me, they are Obsidian for story bible and related information, Scrivener for writing and revision, and Aeon Timeline* for outlining and tracking the many story elements—essential for writing my fantasy series.

For what it’s worth, I don’t think these three tools do similar things. Rather, they harmonize beautifully, like a musical triad.

*Aeon Timeline syncs beautifully with Scrivener so that one could think of the combination as a single app. In my writing workflow, they practically are.

[ViktorSirin]

Allow me to add the solution I am slowly evolving towards.

I create Scrivener projects that I consider my research “folders”. It’s so easy to make a link to open another project, and for each project I make a “dashboard” that contains list of what’s in the project, and explanations of the logic I used when creating that project, as a reminder to myself. One can open a lot of Scrivener projects and cycle between them so easily.

My biggest problem with Scrivener (and almost all Mac apps) is the way tables are handled. I spent the vast majority of my life crunching numbers in Excel Spreadsheets, and for me, so much data is more easily stored in tabular form. So I use Apple Numbers and use File>Import>Research Files As Aliases, which allows me to look at the data in Scrivener, plus I can either open the file from inside Scrivener, or Numbers if Scrivener happens not to be open, and everything works fine.

My last complaint is that Numbers biggest shortcoming is its lack of ability to link to other Numbers files, or other files of any type; it links only within the Numbers file you’re in, or to email or the web, or a phone number.

[GoalieDad]

A research project open when writing in the main project is a way. Or you open a quick reference panel to write in and have a research collection even based on topics like worldbuilding, magic, pov’s, etc. If set up that way can open the “Folder Collection in the Outliner and use the expand all or collapse all commands to show the research hierachy and set it up in a split editor to view any document clicked on in the outliner and even save as a layout. below is an example from the novel I am writing.

The collection unexpanded.

A modified view using the expand all icon on the toolbar (added) to show internal folder structure.

And finally a split editor view to view your research as needed while writing on a floating QRP panel.

I don’t use Obsidian, and on windows not use Devon think, but this might get you closer to what you need while inside Scrivener.