I had a fascinating discussion yesterday about Freedom of Information statistics. Apparently, in the country in question (which shall remain anonymous as the actual details here are less important than the general concept), the percentage of requests that are being rejected has gone up significantly over the last year. It seems that many people think that this is a bad thing and shows that transparency is on the decline. My immediate reaction, on the other hand, was exactly the opposite: I assumed that this was a good thing, and transparency must be much greater now than before!

To explain, let me first step back to my days at BlackStar — at the time Europe’s largest online DVD retailer. Amongst my jobs there was tracking the response times to customer support emails. But, the flip side of that was to keep a close eye on what sorts of emails we were getting, to work out what we could do to no longer need to get those emails. Every week we would see which thing was taking our care staff most time to deal with that could most easily be minimised by making the site work better in the first place. Lots of people emailing us to see if they could cancel their order? Well, then we needed to push the ability for them to cancel it themselves out as late as possible: not just from when it started to be packed, but right through until the Royal Mail delivery truck actually picked it up. Lots of people emailing us to see if we were selling an item that didn’t seem to be listed on our site? Then we found and licensed a database of every video and DVD that had ever been released, even if they were no longer available. Lots of people now emailing us confused as to why were were listing an item on the site if they couldn’t buy it? Then we came up with a service to track down used copies for people. This process repeated itself time and time again and led to us having the most customer-friendly ecommerce site in Europe. However, the natural result of this was that over time the average customer support email got more complex. The low-hanging fruit were constantly being picked off so that no-one would ever again need to ask those sorts of questions. So, by definition, most of the requests we were getting were becoming significantly harder to answer, would take longer to deal with, and had a much higher chance of being unresolvable.

I view Freedom of Information requests in much the same way. Having to make a request should be an exception. Each one is highlighting an area where a government department should have been more proactively transparent. A perfectly functioning system would never require any Freedom of Information requests, as everything that would be released on request has already been published anyway. Most countries aren’t at that stage yet, but there’s a definite trend in that direction, helped greatly by initiatives like data.gov in the US and data.gov.uk in the UK. Both of these countries (and, presumably, any others doing likewise), aren’t entirely sure yet what the most important datasets to release are, though. Patterns of Freedom of Information requests can provide great insight into that. If certain types of information are being requested more than others, that’s probably a good sign that that’s something that would benefit greatly from pre-emptive disclosure.

So, to return to the original question of statistics: what would it mean if a department (or indeed entire country) rejected 100% of all Freedom of Information requests? Well, I guess it could mean that it was the most secretive, refusing to release anything at all. But it could also mean that it’s the most open, and each request made is being rejected either because the information is already publicly available anyway, or because it’s amongst the subset of information that is exempt from disclosure. The raw numbers tell us almost nothing. As always, the truth requires much more digging behind the statistics.