Understanding Nothing

One of the other things that’s changed quite dramatically in the past three months is the way in which I deal with my mail. I’ve been through a number of different mail clients and setups in the past, and have never quite found any approach that quite fits my needs. I was a long term fan of MH on unix, mostly because there was no real client getting in the way. Everything was just simple files on disk with a series of commands to allowed you to slice and dice them in interesting ways. (Actually it was really only one big command that was hard-linked to be called as different commands, changing its behaviour depending on the name it was called as!)

This meant you could chain your mail commands into a normal unix pipeline, allowing you to manipulate them in simple ways – as long as you’re comfortable with the unix command line, of course! And if MH didn’t offer something you needed, you could just write your own as simple shell or perl scripts. One of my first perl scripts was a recursive grep through my mail archives.

Eventually I was persuaded to move to mutt, the one true unix mail client, and it has served me well for past 7 or 8 years. Its scoring and colour coding became very useful during the years I started drowning in spam, and once I got spam detection tools working, the ability to set up keybindings to integrate mutt with those made my life much easier.

I’ve had a rather strange mutt set-up though. I never really mastered procmail, or Mail::Audit, or equivalent, so rather than filtering on ‘From:’ lines, I’ve always filtered on ‘To’ lines. (Having my own domain means I can have an infinite number of email addresses). Each of the many mailing lists I subscribe to, and each commercial site I have to sign up for, gets a unique address, which then get gatewayed to their own folders through a series of .forw.d files. I then have a little script triggered from my mutt startup, to inform mutt of their existence. I then read ‘groups’ of mail in turn, rather than having them all mixed up in one big box.

I’ve experimented with other mail setups from time to time, and have been using Thunderbird via IMAP as by secondary mail reader for a while – particularly for my commercial mail folders which tend to mostly be in HTML these days (and, though some will be horrified to hear me say it, usually better for it), and for dealing with attachments.

But two things have changed in my mail set-up recently, and it’s making me rethink the whole approach again.

Firstly, I’ve switched my spam detection to the new service we’ve set up in UNITE. This is proving much more accurate than bogofilter (which had gotten stuck around 85-90%), and is currently running at 98.8% on the mail that arrives in my box (and a much greater amount of spam is also stopped upstream, never even reaching me). It also operates a web-based quarantine, so I get almost no spam arriving in my inbox. All my fancy mutt bindings to tag, report, and delete spam are to very little avail any more, and with a much clearer inbox, my scoring and colour coding loses a lot of its value.

Secondly, I’ve moved almost all my mailing lists and commercial email to Gmail. As a general mail client, Gmail is disappointing. I don’t think I could use it as my primary tool. But its ‘conversation’ approach works surprisingly well for mailing lists. With judicious use of labels and filters, I can not only direct each list to a different “folder”, but actually allow multiple messages to be in multiple ‘folders’ (so that, for example, Class::DBI discussion on the Maypole mailing list can also appear in my CDBI list, along with any CPAN uploads with Class::DBI in their name). This also allows me to stay on some of the mailing lists I was going to unsubscribe from. There are a variety of lists I subscribe to but rarely read (although I still occasionally skim the threaded subject lines for the occasional post that looks interesting). These are usually related to technologies that I use from time to time, and whose lists don’t have good archives. Often, when I have a problem, I can then just search through my own archive of the list. Gmail, of course, makes this even easier than before.

With these two changes, I now have a situation that I haven’t seen for close to 15 years, where almost all the email arriving on my desktop is either personal or work-related, and almost all requires some action! This theoretically enables me to massively simplify my mail set-up (and then probably add a new layer of complexity in a different way!)

I’m currently thinking of reverting to a single inbox, and doing all my filtering after reading rather than before, probably to a variety of categorised ‘TODO’ folders, leaving my inbox as a real inbox, ideally emptying it every time I access it.

The fact that it’s taken me 15 years to get close to the position that the majority of email users have as their normal set-up probably says something interesting. I just don’t know what that is yet.

Six months ago, after a six month hiatus, I started blogging again for a couple of months. Then the events of life caught up with me again, and I’ve been silent again since then. This doesn’t mean I haven’t been blogging. In fact I’ve probably been blogging more than ever. It’s just that these days almost all of it happens behind the firewall, rather than to the wider world. And even then, a lot of it, isn’t strictly speaking “blogging” any more, but “wikiing”, (or whatever people are going to call that) as I’m finding it a better and better way to structure information.

There’s a certain irony in that, as it was a discussion with Dave Winer about wikis back in 2002 that started me blogging. Wiki software has come a long way since then. A lot of the wiki extensions are certainly interesting, and probably generally useful, although a lot of them seem to misunderstand the “wiki way”. But two features have completely changed the way I use wikis. Firstly, RSS feeds. I always knew that this would be useful on a wiki, but I hugely underestimated just how useful.

Once of the first things we did upon taking over the running of UNITE was to introduce a ticketing system for support enquiries, blogs for staff to narrate their work, and a wiki as a repository of general company information. These are, in many ways, now crucial to the functioning of the business. However, with three disparate systems, there’s a certain amount of information overlap. Not everyone looks at every support ticket, so wider information out of that has to usually either be blogged, or put on the wiki. Any information that may need to be known at some stage in the future in a specific context gets put on the wiki, hopefully in a sensible place. But any information that people might find more generally useful now gets put on a blog (everyone has to read the ‘Announce’ blog, but most people tend to at least skim everyone else’s personal blog as well). Until we switched to a wiki with RSS, some information would end up having to be copied around all three systems.

Now that’s much less of an issue. People don’t need to duplicate information onto the blog if they think it might be of random interest. The people who might be interested can now pick it up from the wiki. Where before you couldn’t ever rely on someone stumbling across it, now people can subscribe to the RSS and notice it passing by. Being able to see every change scrolling past also means that the wiki gets ‘refactored’ much more readily and regularly, which leads, of course, to a much more useful knowledge base.

The second wiki addition that has qualitatively changed how I work is the “discussion” page. I don’t know if Wikipedia was the first wiki to add this, but it’s certainly the most prominent example of it. There it allows articles to maintain their feel as bring part of a real encyclopaedia, even when the content is under heavy discussion. On Ward’s original c2.com wiki the discussions all took place in situ and on emerging topics it was often difficult to follow what was going on. Behind our firewall, this feature has found had a different application. On our wiki we store lots of correspondence. The ‘content’ / ‘discussion’ split allows us to have a page for a specific letter or email, with comments on it neatly separated out.

Switching one of our internal wikis to MediaWiki rather than Kwiki has also convinced me of the disproportionate difference it makes to be able to have “real names” for pages, rather than “WikiNames”. My software development background means I can live quite happily with CamelCase, as can most of the staff of in an ISP. But the switch to MediaWiki style naming has been hugely freeing, and I now find it really frustrating every time I have to find an unnatural way to make a WikiName on our older wiki.

As to future developments, I’ve been watching Jotspot quite closely since I saw the demo at Web 2.0 last October. (Jon Udell’s screencast gives a great overview). The ease of adding structured information to the unstructured (or at best semi-structured) wiki data has huge potential. It’s probably still a little too cumbersome for widespread adoption at this point, but it’s certainly something to watch.

I have some big ideas in this whole area, so I expect I’ll be talking a bit more about wikis soon…