Do I Need to Stop Using Mail as my Filing System?

Latte Other Software & Development
1

Background
Except for large reports that I have to download I receive everything on my MacBook Air by e-mail, with or without attachments. The attachments are mainly MS Word, Excel, PowerPoint and PDF files. I file all the e-mails, which average 20-50 per day in some 260 Mac Mail mailboxes that are nested 2-4 deep, most of them 2-3 deep. These data now amount to 10.8GB. But all this is simpler than it’s sound because, while 260 folders sounds like lot, all this is for five companies that I deal with, of which two are inactive; and the folder structure is very similar for the five companies. Indeed, I could archive some 3GB of data from the inactive companies, as long as I had an easy way to search the archive, which I would only need to do rarely, say, some 3-4 times a year.

As the folder structure is by subject matter, such as, Accounts, Budget, Shareholders, Fund Raising, Board, and so on in the nested folders as well, I usually can quickly find the e-mail that I need through a Mail search facility, particularly as Mail now has “threaded” messages, in what the program calls “Conversations”, in which e-mails with the same subject title are batched together under one heading that can be expanded.

Do I have a problem?
The main problem I have is that, since I file the attachments with the mail messages, it sometimes takes too much time to find a report or memo that was attached to an e-mail. On the other hand, search for attachments is not something that I have to do every day, A related problem, is that reports and documents that I don’t receive by e-mail but that I download, are filed in the OS X folders and it’s sometimes difficult to remember whether a document is likely to me in the Mail filing system or outside of it. But,this is something that happens more rarely than trying to find an attachment in Mail. So the main question is, would there be a better solution for me than to use Mail as my main filing system?

Possibilities?
Now, I read through the last few pages of a long thread called Large-Scale Info Managers and Scriv, and learned a bit about DevonThink, Hazel, NValt, PostBox and ImportExportTools. While DevonThink seems to be powerful in its database, organizing and query functions, I am reluctant to entrust my e-mails and attached documents to a closed system, from which it does not seem that easy to extract the data. Also, the idea of sending each piece of mail to DevonThink, after having filed it in Mail, seems like a pain — I should explain that I now file each e-mail that I read or that I write by control-clicking on it in Mail and selecting the mailbox folder where it should go. I have similar concerns about having to file twice with Hazel/NVAL and Postbox/ImportExportTools, although I still don’t know exactly how those tools work, but they seem to have the advantage of placing the data in the OS X folder system rather than in something relatively closed like DevonTools. Basically, I am beginning to think that my problem of searching for attachments in Mail may be not that major to warrant the investment of time in moving to a new filing system. Any thoughts and recommendations, on way or the other?

Advantage of Using Scrivener
I should add that I have been using Scrivener for a month. Basically, I used it for documents (notes to the board and memos and letters of some complexity, for which I need to refer to “reference” reference material. All of this is for current issues that I deal with and helps me a lot because I can find all the documents any of these issues that I need for conference calls and meetings, which, since I have all this available in the Scrivener Reference facility, means that need less time preparing for these calls and meetings. Also, having the documents on current issues readily available as Scrivener Reference means that I need to spend less time searching for them in Mail, which is what I would do if I didn’t set them up a Scrivener Reference, simply because I would be less organised in this respect.

—Mitch/Bangkok

2

If I had that much data, I would definitely use Devonthink. Scrivener is not suited to the job – it’s for writing, not large-scale data management. It doesn’t have the search tools that Devonthink has, for example. (It can search for a word that is, say, within ten words of another word, next to another word, etc, etc.)

I wouldn’t call Devonthink a “closed system”. You may have been reading about an old version of Devonthink, which stored material in a database that was to a great extent “opaque” to anything outside Devonthink. The latest version uses a different system. If I were you, I would (a) download a copy and try it, and (b) go over to the Devonthink forums and ask for advice. They are the experts on Devonthink.

Having said that, you might want to investigate MailTags, and MailActOn:

indev.ca

Cheers, Martin.

3

Martin, just to be clear: I was not suggesting to use Scrivener for database management, but only meant to illustrate what I do use it for.

I’ll have to look at MailTags and MailActOn. Perhaps the latest version of DevonThink is not that “closed”, but I still have the concern for filing in Mail and then exporting and filing elsewhere, and wonder whether it’s worth it considering that I don’t do that much searching for attachments, as I mentioned.

—Mitch/Bangkok

4

Martin, I’ve looked at MailActOn and MallTags. MailActOn doesn’t do anything for me because (1) I cannot set rules based on the Sender because I am dealing with three companies and the most of the Senders work with all three, nor on the Subject because thus changes and even e-mails with the same subject often have content that requires filing in different folders. Also, the list of folders shown in the MailActOn palette is flat (un-nested), which isn’t useful when one has 260 folders — much easier to file by doing a Control-Click in Mail and getting a Move-to list that shows nested folders.

Not sure about MailTags but, since tags can act as keywords that can be used to select message viewed, I could out a tag on each e-mail that has an attachment, which would make it easier to search and find e-mails with attachment, that being one of my problems as the attachments are not shown the collapsed view in Mail in messages that I send out; they are shown in received messages.

—Mitch/Bangkok

5

Each person has to work out their own system based on their particular needs, of course, but I find it useful to think in terms of processes – what processes do I go through most often (and can any of them be automated), what slows down each process, and what leads to errors creeping into the process(es)? To my mind, it is only by carrying out that initial analysis that one can arrive at a good system.

Sometimes one has to try out a few tools, because it is only by using them that one can really discover their potentialities and limitations – and each potentiality and limitation may be tied to the particular things one is trying to do. They may not be apparent to someone doing something else. I can only suggest that you try Devonthink – it may be ideal for you, or it may be useless for you.

Tagging is certainly something that can be very useful, and OpenMeta tags are system-wide, so you can search for them via Spotlight, Devonthink, MailTags, and many other programs. You could have a look at these blogs:

brettterpstra.com/some-suggestio … r-tagging/

brettterpstra.com/on-sorting-tag … r-nerdery/

And by the way, you might want to look at the documentation for MailActOn more carefully – there is more to it than might seem.

Cheers,
Martin.

6

A point about DevonThink that you may not be aware of: if you have DT Pro Office, you can send entire mailboxes of messages to DT via a single command that DT inserts upon installation in your Apple Mail menu. In DT Pro Office the mailboxes can then be searched using DT’s extensive search features. Subsequently you can update the mailboxes now in DT with individual messages, again via a (different) single command in your Apple Mail menu.

More widely, I endorse what Martin has written above about DevonThink. If you have a lot files (messages or others) to capture, store, recover and perhaps later export again, Devonthink is as good a solution as you’ll find for the Mac. As Martin says, a DT database is no longer a closed black box; it can, for example, be indexed by Spotlight. And it works happily with Scrivener: you can drag whatever subset of files you’re working on at any one time from DT to Scrivener’s Research folder.

At least it would be worth trialling it.

7

Martin and Hugh, thanks for taking the time in responding and making suggestions. Please don’t think that I am ignoring what you’re saying but, so far, my conclusion is that I can basically go on doing what I have been and continue to use Mail as my filing system. But I’ve made some progress and have learned a few things.

You may recall that my main problem was difficulty and slowness in finding documents that were attached to filed e-mails, particularly e-mails that I had sent out because, in the collapsed view, there is no way of knowing whether a sent (and filed) e-mail has an attachment. Now, after reading and thinking about your posts, which, when I looked in to “tags”, stimulated me to learn more about searches and “tokens” in Mail. Indeed, I’ve found that there is a “token” for searching for attachments, which can be compounded with other tokens (e.g.,Sender, Subject, Keyword).That essentially solves my main problem of finding attachments. Also, compounding tokens also speeds up finding any e-mail. The other thing I found that the structure of my folders in Mail, that is the structure of my database is good in that I know where to file any received or sent e-mail without having to agonize over where is should go. That also means that I know where look in my searches. At this point, I concluded that “the best is the enemy of the good” and decided to continue what I am doing, but with greater skill and speed in my searches. The answer, thenm to my original question is, “No, I don’t need to stop using Mail as my filing system.”

If I were to use DevonThink, I would use the database structure that now have in Mail; but I would have to take extra steps in filing e-mails in DevonThink. Sending the e-mails to DevonThink could probably to automated to some extent, but the time to set all this up and debug would be excessive; and I would gain little because I don’t need to explore or use other relationships between my e-mails that DevonThink could identify. I’d be running two database systems, with only a small gain in ease of searching.

As for generally automating filing of e-mails in Mail, I need to file manually because, as mentioned earlier, I receive the bulk of emails from a team that works for three companies, which means that neither the Sender nor the Subject can be used to determine where to file. Finally, I did look at DevonThink some ten days ago — granted, too briefly — and imported one mailbox of some 500 e-mails and then just stared at them and wondered what to do. What I saw was that the effort to set up the DevonThink database would be too much for too little gain in my case.

—Mitch/Bangkok

8

It’s not uncommon for discussion to lead one back to the place one was before – but with the advantage of being better informed and more certain of what one is doing. That is not a bad result – and certainly not a waste of time. Far from it.

I’d only add that it might be worth keeping an eye on developments in tagging, and occasionally look around to see if someone has come up with new ideas on the subject. I think it is likely to become more widespread. And it does have the advantage that it can slowly be added to what you are doing already without really disturbing the present workflow. It just adds an extra possibility when it comes to searching for things or grouping them together. Also, be aware that you can use a syntax in Spotlight like “tag:money kind:email” to conduct quick searches without even opening Mail. You might have a look at HoudahSpot to improve your search capabilities:

macupdate.com/app/mac/21333/houdahspot

or even Devonsphere Express, which brings some of the search capabilities of DT to the desktop:

macupdate.com/app/mac/41880/ … re-express

Cheers, Martin.

9

Martin, yes, I feel that I’ve made a lot of progress and improved how I’m using Mail: relatively little effort for a large gain in efficiency — and also in gaining some peace of mind from no longer fearing, as I start searching, that I won’t find what I’m looking for.

You make a good point about continuing to look at developments, and I’m trying out MailTags now. Thanks for suggesting HoudahSpot and Devonsphere Express. I’ll look at these when I have some time, perhaps next weekend.

—Mitch/Bangkok

10

I’m glad it helped.

I’ve just tried Devonsphere Express with email by searching for a key phrase from one message – it found 33 related emails instantaneously – and I mean so fast that it had brought up the list before I could move my eye from the part of the screen where I was typing to the place where the list came up. I really do suggest you try it!

Best of luck,
Martin.

11

Martin, I’ve just downloaded, installed and indexed it, and shall by trying it out all this week.

—Mitch/Bangkok

12

Martin, I have now tried Devonsphere Express to do some searches in my Mail database and find that it is fast, but that search in Mail is just as fast. However, using the search facility in Mountain Lion (and Lion) provides somewhat more specificity, and hence effectiveness, for my purpose. One example: I did a few searches for all MS Word attachments sent by a specified sender. The results were that Devonsphere Express always listed some more e-mails than the Mail search. The reason is that in Devonsphere Express I could not find a way to specify that I wanted to find attachments with “.docx” in the attachment title, which Mail does, so that the Devonsphere Express result list included some e-mails in which the body of the e-mail contained the name of a file with “.docx” in it, but which had no attachment.

Another thing that I found is that Devonsphere Express is a memory hog. The Activity Monitor showed that Devonsphere Express uses almost 400MB or “Real Memory” while not performing a task and as much as 850MB when doing a search. Using a MacBook Air with 4GB of RAM makes me concerbed with this.

I haven’t yet tried HoudahSpot, but a brief online video that I saw on it makes it look like it offers substantially more search power than Devonsphere Express, but I don’t know whether it would be any better for my purpose of Mail search than Mail itself. It is also interesting that it is a front end for Spotlight, which should mean that it uses much less memory than Devonsphere Express.

I continue to find MailTags useful, only moderately so at this stage, but I can see that the usefulness may take a leap as I have more experience with it. One thing that could be very useful is if I could find a way of automatically filing outgoing mail using a tags from MailTags together with MailAct-On; but that will take me some time to figure out because my filing criteria are fairly complicated and not limited to the Sender or Subject. One problem with these Indev programs is that they only have quick start guides but no manuals. More information is available on the Indev website through searches of discussions on Support, but that is laborious and timemconsuming.

—Mitch/Bangkok

13

Mitch,

I was guilty of being a bit brief in my comments. The strength of Devonsphere Express is precisely that it will do searches that find related material. If you already know what you are looking for, this is not so useful, but if you want to find a load of things that are related to each other (because they have similar content) it has a role to play. Moreover, you can use the extended search parameters that you get with DT Pro (for example, you can enter “napoleon NEAR bonaparte” and get hits only when the first word is ten words or fewer before the second word). There is a list of extended search terms in the Help file. In short, I find it a useful extension to the search capabilities of Spotlight / Mail – if I need to look for something specific, then Spotlight / Mail will do the trick. If I need to bring up material that might be relevant, I can use DSE (my paranoia is that I might miss something that is important). Different horses for different courses – but of course you might not have the latter requirement.

As for memory, at the moment DSE is using 48MB on my system (I’m also on an MBA 4GB). I did a search and it used 96MB. Not sure why it would be using so much on your system.

Anyway, it seems you are edging closer to a system that works for you, which is good!

Best wishes,
Martin

14

Perhaps it was busy indexing Mitch’s hard disk? I’ve read that high memory usage for that purpose is a temporary phenomenon but can last a little while. (I believe that DSE was a big, permanent memory hog when it was first released, but now, at version 1.5, that issue is said to have been addressed by the developers.)

H (not a DSE user, but considering purchasing it)

15

Actually, the high memory use figures were long after DS had finished indexing. Later, I did a web search and found a good number of complaints of it’s memory use, although a few people reported figure that are as low as the one’s that Martin cites.

—Mitch/Bangkok

16

Mitch,

You might be interested to look at this, which is a lot of Mail-associated software at half price (including MailTags and it partners):

getback2mail.com

Martin.

PS: there is a product called MailHub, which I didn’t know about, and it looks as if it might be useful for your scenario with lots of folders (though I haven’t tried it yet). It seems cheap in the half-price sale, too.

17

Martin, thanks for the head up. I’ve been trying out MailHub for the last two days: it does exactly what I need in terms of automatically filing a sent message in the folder of the message that I am responding to (“Send and File” facility); it also has the option of “Send and Delete”, which I use for thank you notes that I don’t need to file. Moreover, MailHub facilitates filing because it allows you to find in its search field the folder you want to file something in — you then click a button to file a message and a thread. All this save a lot of time when your dealing with 50-100 e-mails daily.

I’ll now also buy MailTag, which I find useful. I was interested in MailAct-On, but learned that with the current version of the program one can implement “Send and File” only with Smart Mailboxes and an Apple Script. Invev customer support will send me instructions or an example of how to do this. I would have liked to do this with MailAct-On because it integrates with MailTag. I think the next version of MailAct-On will have this facility, but for right now MailHub is a no-brainer, particularly at half-price.

Thanks so much for putting me onto all this, which will save me a lot of time and effort. Thanks to you, I’ve made a lot of progress since I started this thread. I now know that continuing to use Mail as my filing system is viable, which has been made more efficient through the use of MacTag and MailHub.

—Mitch/Manila