That’s the longest break I’ve ever taken from blogging: 36 days without a post. At first I was just too busy for a week. Then as time went on, I had more and more things I was thinking about blogging — the pile of ideas kept getting bigger. Finally the sheer size of the pile and number of possible choices became overwhelming. I was a victim of too many choices.
I decided to break the stalemate by putting aside all of the ideas and starting with one new one, which is to talk about the tools I use to be more efficient and productive at work. What I use might not be right for everyone, but perhaps you’ll see some idea here that can help you with your job or your hobby, and be happier with your effectiveness. I was inspired by John Poelstra’s recent post on The Drug of Distraction to write about this subject.
Email is the lifeblood of most information-oriented jobs now, and especially one revolving around open source or a globally dispersed company. Both of these characteristics are part of life at Red Hat, of course! It’s really important to have a way of dealing with email that makes your work more efficient. For me that comes down to two major considerations. There are others, of course, but they’re not quite as important as getting these two right. That’s my email client, and filtering.
First is your email client. For a long time I was a relatively happy user of Evolution, the GNOME personal information manager that helps you manage email, calendar, tasks, and addresses. It’s a robust, full-featured program and I still think it’s a wonderful piece of work even if it has its issues. And honestly, what software doesn’t? However, what I found was that there were a couple of factors that were causing me to lose a lot of time in reading and processing email. By “processing,” I mean performing whatever task is appropriate for a specific piece of email. That might mean deletion, or it might mean responding immediately, or something in between.
Evolution allowed me to have several windows open at once where I could be drafting many different emails. I could make a start, then put the compose window aside to work on something else, or process another more important email before finishing. And for me, that turned out to be a drawback. It encouraged a lack of focus on what I was doing, so that by the end of the day I had a bunch of unfinished responses — and the concomitant feeling that I was a failure because I wasn’t getting the responses out. I was letting my tool usage affect not only my output, but the way I felt about my performance!
(You could of course say that the fault here was mine, not my email application. After all, why can’t I just focus better? And that would be a perfectly fair assessment. However, to change very deeply ingrained behavior takes a lot of time and effort, which I felt I could better spend simply doing work. And it’s probably arguable that changing my behavioral reaction to a windowed environment to optimize for how I deal with email at work wouldn’t necessarily be a net benefit in the general case of my overall computer usage. In any case, I don’t want this to come off as an indictment of Evolution, which I think is a great piece of software.)
Another factor that was hampering me was the way I was accessing my email through the client — IMAP in this case. Each time I wanted to read or reply to email, I was hitting the network. I have a great broadband connection at home using Cox Cable — around 10-12 Mbps downstream and something like 2 Mbps upstream. But even so, latency delays (seek and retrieval) on the server is more of a concern than transmission time, and there was a delay of seconds every time I accessed email. And with hundreds of email messages a day I was looking at — after filtering those I didn’t need to read — I was wasting a lot of time. With 500 email messages and, say, 3 seconds per email on a good network day between retrieval and response (this is a conservative estimate), I was losing 25 minutes a day assuming response time was good. Over the course of a week, that’s over two hours!
Two tools helped me wrangle my email to the point where I could reclaim every bit of those two hours and more. First was the Mutt email client, which is a text-based email reader. Like most text based programs, it has a bunch of keystrokes that you have to learn if you want to use it quickly. It took me about three days of use to get really accustomed to its interface, and most of two weeks to optimize my configuration. But once that was finished, I had conquered the focus problem in dealing with a GUI-based, windowing email client. I was responding to an email in a terminal as a single foreground operation, and when I was finished with my response, I would send it to return to the mail reader. Not seeing other email messages waiting for me (or arriving) as I composed helped me focus on the task at hand, finish it quickly, and return to processing other mail if needed.
The second tool — or rather a combination of tools — that helped me immensely was offlineimap, for syncing email to local storage, and postfix for sending out email from my local system. Using this toolset meant that my network delays were a thing of the past. Recalling email from its stored location is instantaneous, and when I send an email, the response from the server is instantaneous as well. The email gets sent out as a background operation and if anything untoward occurs I get notified as with any other email service.
The other major benefit of offlineimap is like other integrated clients in that it gives me a choice of when to sync my mail. I can either run the synchronization as a periodic background process, or as a one-time operation. I’ve come to greatly prefer doing this as a one-time operation because it keeps me from “riding my Inbox” all day. Nothing will kill true productivity faster than being on email all day, waiting for that next “important” message to arrive. Instead, I try to check email only a few times each day: (1) all mail in the morning after completing any other critical work; (2) important folders only, right before lunch; (3) important folders only, as an afternoon break and limited to no more than a specific time like 10 minutes; and (4) (optional) remaining mail before I leave for the day. This type of routine has helped me restrict my email processing time per day to a very reasonable amount of time — leaving me more time for work requiring heavy focus, deep thought, in-person contact, and/or dedicated writing or composition.
The second consideration for email was filtering. Server side filters are the most effective in my opinion, because among other benefits they don’t rely on me having the same local configuration wherever I’m retrieving or processing email. I have a lot of filters in place both with my GMail account and my internal Red Hat email, to sort each list separately, and prioritize specific kinds of mail such as those that come directly to me from specific people (like my manager).
These filters help me prioritize my email work, so that I don’t end up spending an amount of time on each list proportional to its list traffic. Certain lists, and of course my direct Inbox, are more important to me, and I handle those first. If I’m pressed for time I can defer all the rest. In addition my Mutt email client helps with this process because I have a routine to elevate specific mailboxes to the top of the list through which I cycle. I also practice judicious use of functions such as “mark thread as read,” avoiding spending too much time on discussions that don’t require my attention or input. Every six months or so I also make a practice of archiving old mail so that my mailbox searches, which usually are focused on the recent past, stay fast.
More generally, using this combination of tools and routine helped me conquer an internal focus problem as well as an external latency problem. Again, while my choices may not be right for everyone, they really helped me.
I still highly recommend running your own mailserver; it’s the best change I ever made to my workflow. You can aggregate mail from all your accounts into one account accessible from any system, and have one point for filtering (including spam).
You’re halfway there with postfix, that’s the outbound bit (though the neat thing is to set it up to act as a secure relay so you can send mail through your server from a cellphone, and so on; you can also set up filters so it will forward mail sent from each of your identities through the appropriate SMTP servers, which is important to avoid it being filtered on remote ends because it looks like spam, as it claims to be from a redhat.com address but didn’t come from Red Hat’s servers, say).
for inbound I use fetchmail to retrieve, procmail to filter, and courier to serve it out.
It gets a bit fiddly dealing with firewalls and ssl certificates and getting on the Red Hat VPN for RH incoming/outgoing mail, but it’s definitely been worth it for me.
You mention postfix for sending out email from your local system. I am curious to know how you have implemented this with so many anti-spam systems moving to reputation-based filtering. Is your local postfix sending thru HQ? Or what?
@Adam: @Jamie: Both of you hit on one of the risks of running your own mail server — although one you can negotiate without too much difficulty. I’ve done what Adam mentions, which is to forward mail through the appropriate SMTP server based on my sending identity.
Since I trust my email providers’ storage and backups more than my own, I’m OK with leaving my mail there for now, although I have considered running my own mail server. (I actually do have one at my domain but I don’t use it heavily or advertise it.)
I have a small NetworkManager dispatcher script that manages the “defer_transports” setting in postfix, based on the available routes when an interface goes up or down. I run my different email transports over separate named postfix services. That way I can defer email to each SMTP server separately. It’s working really well; postfix holds onto mail for the deferred transport until the right network is available, and then flushes the queue. Makes email handling on the airplane really snappy!
@Adam procmail is apparently no longer the recommended filter to use with fetchmail — mailfilter is, and it is actually easier to configure.
I’m using pretty much the same setup, mutt+fetchmail+mailfilter, although I do fetch mail every 5 minutes. I’ve come full circle, starting out with pine, then mutt, then evolution, thunderbird, then back to mutt. I tried mutt’s IMAP support for a bit, but it was just too slow when switching folders.