Large-Scale Info Managers and Scriv

Well ultimately it comes down to what works best for you, the data you are working with, and your relationship with it. It is difficult for me to say.

But I can say on this, where in example A you have a folder hierarchy ending in a simple filename: that’s the typical way of handling files and folders. You’ve got broad folders at the top level and things get more and more specific as you drill down, and finally you start running into files. The files themselves are named fairly simply, because they don’t need to be unique amongst many; they don’t need to communicate much for themselves when the folder hierarchy has already said much.

The reason that I myself started developing something that eschews that way of working is because I grew tired of:

  1. Having to decide which folder everything should go in; especially the ambiguous ones
  2. In the case of ambiguous ones and wanting to use aliases/symlinks/etc having to do all of that manual labour in order to keep things consistent
  3. Not being able to easily scroll through a list of everything because I don’t remember which folder it or its aliases are in
  4. Not being able to see items in context with one another even if they are unrelated

So, lots of busy work throwing files around; making folders; and if one is ambitious, aliasing them around so that your mind has multiple access points in the future in case you think of one hierarchy and not the other.

The latter two points are more discovery based, and are important to me in that upper equation. My data and my relationship with it is such that it is very useful to look at a list of things from “around then in 2006”. Even if I’m only looking for one thing, finding that thing amongst all of the things that I did at that particular point in time (and took the trouble to record) is to myself, extremely useful, because I’m the type of person that gets insights from and ideas from stuff like that.

I can find an old recorded notion, but I can also see everything that went on around that time that may have led to the notion. It’s not just a file sitting in the end of a long chain of pre-supposed descending categorical statements. It’s a chink in a chain that is my entire recorded life. Being able to see, just to pull out an example, the development of this system itself as it was recorded in I-Theory files amongst all of the personal observations; journal entries; e-mails & chats; creative work; photography—everything else that makes up my life, is pretty cool and exactly what I set out to accomplish.

So for me, the primary premise of this method was to build a better record; because I’m big on that. If your relationship with your data is more needs based; more results seeking; then some of these more existential properties might be of less importance. The chronological theory might still be useful to you, but for entirely different reasons.

So that is why I prefer no folders at all—and the only reason I started using quarterly folders is, as said, to keep the computer from getting slow. Finder takes an age to open and display a folder with 10k items in it. Over time I’ve come to appreciate the quarterly slices as a feature in and of themselves, too. Like I said earlier, it’s pretty easy to pin down something by quarter and year. You might be foggy on precisely when, but late 2008 is pretty easy to pull out of memory.

And that is of course why I felt the need to develop a formal system for naming the files themselves. A long chain of files with no categorical assignment needs to be able to convey quite a lot in the filenames.

As to which method is better, who knows. I know what works better for my self+data+relationship model, but that’s about all I can speak for. But it is interesting to note how Andreas and I are essentially operating from the same stance, your method needs to improve your thinking—but to him a chain of files is a dust-bin. To me that’s how project and categorical folders are because once the project is done, the files within it cease to live amongst the rest of my thoughts unless I specifically drill into that folder and muse through it—but even then it is divorced from time and the history of my life. It is just a categorical statement, isolated from everything else. I could turn on Finder’s modified date column and visually cross-reference things, but that’s very inefficient. So that system doesn’t work for me and my thinking.

I mentioned grep as a tool in conjunction with CLI usage because that is how one would approach the problem of paring down a long list of file names if they didn’t have a GUI. I don’t think it is necessary to learn that skill on a Mac unless you find yourself running into limitations with Spotlight’s fuzzy-results system, or cannot find a search tool that works for you otherwise. grep uses the complex regular expression query language. You can Google ‘regular expression tutorial’ to see whether or not it is for you; if so you needn’t take yourself to the command line to use it. There are a number of Mac tools that use it as well. Geeks like it for its power, so you tend to find it lurking in the “advanced” sections of programs here and there. EagleFiler, for instance, allows you to perform regular expressions in smart folders.

There are so many ways of finding something, but the chronology + categorical filename system very rarely fails me. It did for a long time feel really weird (to the point of panic) to just throw everything into one spot, I fought it for a long time; trying to build super-layers of categorical assignment via keywords or what have you on top of everything. But then I realised I was just replicating the busy-work parts of my old problem. So I pared down these super-layers (which I ironically never used, not once, to find anything) and let the information flow naturally as it was experienced, written, and fixated.

Innovations since that point have been entirely to mechanisms within it. I now use a recorded-search method. If I look for something and it takes me more than a few seconds to find it, or if I look for several things—I write that down! I create an index file of my discoveries, the holes in my theories; areas I feel still need to be documented, and then I file that index record in the system. I’ve been doing that for several years now, and I’ve already benefitted from it. A good example is this discussion right now. For a while I had my thoughts on that matter scattered all over—but because I’ve periodically revisited it due to forum interest, or been requested a list of links to the topic, I now have extensive index records and search history files on this topic. I can pull up lists with dozens of articles, e-mails, forum posts, and rich back-research into earlier attempts that I’ve voluntarily explored in free time. I now have a very broad and clear understanding of my efforts on organisation, thanks to the organisation system itself. Sometimes that’s very simple. For example, I remember some idea I had about the philosophical properties of beauty, look for it, find it in an old 2004 transcription from shorthand on a bus commute, so I write a new article back-referencing these sources with my updated conclusions. Now there are two points of discovery on this topic in my archive. It’s now twice as easy to find the thoughts. If I happen across the original (which doesn’t forward-link due to my immutability guideline) I can merely search for its ID—and the more modern file comes up because it used the original’s ID to back-reference.

So my refinements have been to methodology within the structure itself, not to the structure itself.