Understanding Nothing

I run several sites off Mediawiki, and the spam problem has been getting progressively worse recently.

Making the wiki ‘registered users only can edit’ helps a bit, but most of the spambots can get around that now. In fact most seem to get around the ‘easy rollback’ of recently registered users changes by creating their spamming accounts quite some time before actually using them to spam.

Most of the spam I get is of a particularly useless kind. It’s wrapped in a div set with a style of “overflow:auto;height:1px;”, which makes it invisible on the page. However, as most recent releases of MediaWiki also default to adding a “nofollow” to external links, this means that the spam links get no googlejuice either.

Although useless to the spammer, it’s highly irritating to wiki admins. However, there is a particularly simple fix.

In your LocalSettings.php, just add one line:

$wgSpamRegex="/overflow\s*:\s*auto/i";

Now any attempt to include text that matches in an edit causes the edit to fail.

I’m sure this is documented somewhere, but I only came across it on one of the mailing lists. So I’m offering it out here in the hope that some other plagued mediawiki admin can cut back their admin burden too.

Dion Hinchcliffe’s Web 2.0 Blog has an interesting post on “Ten Ways To Take Advantage of Web 2.0″. It’s full of great advice, but I find point 2 particularly interesting:

The read/write Web is about making users co-creators of content on a massive scale. Armed with foreknowledge of the effectiveness of the Wisdom of Crowds, you can take advantage of the fact that none of us is as smart as all of us. Wiki sites turn this editable dial all the way to the right for example, and let every page be editable by anyone who is allowed. Far too many sites don’t take advantage of the fact that you can give people an ownership stake, and get them immersed in working on improving what you offer, all just by letting them have the ability to change an appropriate level of content.

I’ve been writing here about wikis since I started this weblog back in early 2002, and it’s fascinating to see them getting tagged as Web 2.0!

I’m certainly no rabid purist on this, distraught that a 10 year old technology is getting caught up in some newfangled hype. As far as I’m concerned, the more things that promote the usefulness of wikis the better.

And it’s great to see that Joel finally gets it too!

In your robots.txt, make sure that as well as the traditional

Disallow: /wiki/index.php?

you also include

Disallow: /wiki/Special:Random

Otherwise you’ll get some rather confused visitors…

It seems that someone was actually considering staging “Jerry Springer: The Opera” at the Waterfront Hall in Belfast.

It is, of course, possible that they were figuring that having the DUP et al picket the production, with all the accompanying publicity, would be a good thing, allowing them to slash their marketing budget by 90%, and that this was actually a well thought out proposal. But if so they overlooked one rather important fact. Belfast Waterfront Hall is actually operated by Belfast City Council, rather than the private sector, so the Council has to approve everything that happens there.

And, as one might expect, they refused, “in the light of the potential of the production to cause grave offence to a considerable number of the citizens of Belfast”.

I’ll leave the social commentary to one side, and instead focus on one other major benefit to having the minutes in a wiki that I missed in my earlier post. The “official” minutes, whilst theoretically available on-line, are very far from easy to link to. They’re hidden behind a log-in (it may be one which automatically logs you in without you really knowing about it, but it still has an annoying habit of logging you out when you idle to long, and forcing you to “log in” again), a clumsy search engine, and are in Microsoft Word format, and although the search engine can display these as HTML, it’s all controlled via a JavaScript interface which makes it pretty much impossible to link to directly.

On the wiki the minutes become first class internet citizens.

Maybe in time the official minutes system will allow for this, but given the difficulties the Council appears to have with internet technology, I’m not expecting this any time soon.

James Stewart picks up on my rant about Belfast City Council’s minutes system with lament on e-Government in general.

He makes the interesting comment that:

Properly modelling government structures is a complex business, and my experience so far that it’s a lot of work to build the critical mass required to make a wiki work in a context like this. But given the service the council provides, it won’t take much to shame them, and it’ll be interesting to see how the project develops.

This is, of course, true, but at this point I’m not really expecting this to build up any sort of critical mass. In my experience, very few people in Nothern Ireland are sufficiently interested in what local government actually does. I don’t need to fund a focus group to realise that the size of the interesection of that small group with the also rather tiny group of people who are likely to contribute to a wiki is going to be very close to zero.

However, putting the content on such a wiki does still have some benefits.

Firstly, it helps me keep aware of what the Council is up to. By adding pressure to myself to spend 15 minutes a day doing something on the wiki, just to keep it ticking over, I’m making sure I’m paying attention to what’s going on. Without the wiki my good intentions here are more likely to get swallowed by all the other things I’m meant to be doing every day.

Secondly, the content will get picked up by Google. Currently the minutes, although theoretically public, are so well hidden behind a complex search tool that in practice they’re invisible to the search engines. By breaking them out of this constraint, it’s more likely that someone else will be able to find what they’re looking for.

Thirdly, by making them part of a wiki, it allows others to comment on them or help tidy them up, by making relevant cross-links. I’m not really expecting this to happen much, but it’s better to have the option open than not.

Lastly, it provides a framework for anyone else in NI who’s interested in this to do likewise without needing to have the technical ability to set up a wiki. If someone wanted to put, say, Carrickfergus Borough Council minutes online too, the barrier to doing so is now much lower. Again, I’m not really expecting this to happen any time soon, but again, it’s better that the ability is there.

Of course, in an ideal world, lots of people would find this immensely fascinating, and we’d have a steady stream of contributors, properly cross-referencing and cross-checking everything, and helping provide the checks and balances necessary for good government. But this is, after all, Northern Ireland…

My remark in the previous post on how the biggest problem with introducing a corporate wiki is social and cultural, rather than technical, provoked quite a few replies. For the most part they agreed with the point, and several people asked how best, then, to counter this.

I certainly have no silver bullets, but I can offer one observation that has been true for every wiki that I’ve seen succeed: At least one person needs to use it both as their primary source of information, and their primary tool for sharing information to others.

Every time someone asks them a question, they respond “It’s on the wiki” (and if they’re more polite than I usually am, tell them where). Every time someone starts launch into a protracted explanation of something, they interrupt with “Can I read about this on the wiki?” Gradually they train everyone that information flow, at least as far as they’re concerned, happens on the wiki.

For most people who haven’t experienced wikis before there’s a bootstrapping problem. They regularly go to the wiki looking for information and find that it’s not there. If this happens enough times, anyone will get frustrated. Eventually they give up, and revert back to their old ways of working, bemoaning the uselessness of this latest technology. The threshold differs from person to person, but any early stage wiki is usually going to hit a majority of people’s limits fairly quickly unless it’s pre-seeded with a lot of useful information. Most people tend not to really start seeing the value of a wiki until the information they require is there more often than not.

A true wiki champion, the type needed by every successful wiki, somehow doesn’t despair at the absence of information. Rather they see it as another opportunity to make the wiki even more useful. They’ll either make sure that when they find the information, they add it to the wiki, or, if they’re a true master, ensure that the person who does have the information adds it to the wiki for them.

In my experience, this pattern of driving everyone else to the wiki every time they need information, or want to share it, builds a momentum fairly quickly. The death of a wiki is not related to the absence of information; it’s related to the absence of use. One regular user can push those in his immediate work circle to become regular users almost by stealth. And once they cross to acting likewise, the battle is usually complete.

I’d love to hear others’ stories (whether of success or failure), of the issues involved introducing wikis to companies. There’s bound to be lots of useful examples out there to learn from. Post your story somewhere and link to this post, so I’ll notice it. Or just drop me an email, and I’ll collate and summarise.

UPDATE: By request I’ve also enabled comments on this post. In general I have them turned off, for a variety of reasons, but I’ll see how it goes here …

UPDATE 2: Well, it didn’t take long for the comment spam to turn up, so I guess it’s back to email.

After my recent ramblings about wikis I got an email a few days ago from one of the developers of a new hosted wiki service inviting me to check out their service.

The service was quite interesting, and had a few nice tweaks on some of the standard wiki features, but something unidentifiable was missing, and it got me thinking about where wikis are in general, and where they could or should be.

Although wikis have been around for a long time, they’re still mostly geek technology that hasn’t “crossed the chasm” yet. I believe they certainly have the potential to do so, and be hugely disruptive en route, particularly in the whole sphere of corporate knowledge management, in all of its various guises and disguises.

There are a few companies which seem to be actively targeting this corporate space. Jotspot, in particular, seem to have some of the features that might enable them to get a few big ticket sales. (Of course, the primary function of tools like Salesforce.com integration is presumably to assure credibility when talking to Fortune 500 companies – whether customers might actually use such functionality, and my guess is that most won’t, it certainly positions Jotspot as a serious enterprise player). But yet Joe Kraus is clearly also excited by the possibility of the Long Tail, and is looking for ways to tap into that. Their offer of a free hosted wiki for open-source projects is certainly a nice attempt to get the early adopters on board.

I certainly wish them good luck with this, but I’m not sure that, at this stage, the same company is going to be able to sell into the large corporates, and also pick up the long tail. The market is too early, and the technology is too disruptive. As Hugh at gapingvoid points out again and again, the problem with technology implementations is usually a social one, rather than a technical one, and this is rarely more true than with so called “social software”.

We’ve used corporate wikis in several organisations now, and for several purposes, and the biggest issues are most certainly to do with getting staff to embrace the entire concept of wiki, rather than wrestling with technology. We’ve been reasonably successful at this, but it’s been in small companies with little corporate politics. I’ve certainly been in several organisations where the concept of a completely open knowledge base that anyone can not just view, but add to, and edit, however they want, would fill a significant proportion of the staff with horror.

I think the easiest way to get around this will be smuggle wikis in the back door. I imagine a Jotspot implementation that appears to be little more than a new front end into a pre-existing customer database, but which a few select employees start to realise can be extended in interesting ways. Gradually the power of being able to add new semi-structured information on top of the database begins to sink in, and before anyone ever realises that they actually have a “wiki”, it’s become the definitive source of useful information about a whole variety of things that were previously outside the scope of the CRM system.

But simultaneously I think there is a huge opportunity for the smaller, built from the ground up, single purpose wiki. More and more conferences have a conference wiki where attendees can collaboratively create and refine a guide to the conference. When we hosted YAPC::Europe last year, we took this a stage further, and invited prospective speakers to propose their talks via the wiki to seek attendee comments well in advance. This wasn’t as successful as we liked, but several of the speakers told us that the feedback they had received from this helped them prepare better talks.

At the minute, however, it’s still too hard to set up a wiki. Everyone is working on making it much easier, and I’m sure I’ll get email from lots of people telling me that their product has finally solved this, and you can have your wiki up and running in 30 seconds. As most of the people setting up wikis at the minute are techies, the problem isn’t that noticeable. But we have lots of small business clients who have little to no technical expertise, who would really benefit from a wiki. But at the minute they would either have to pay someone like us to set one up, or try one of the hosted wiki services.

But, of course, the hosted services instantly hit the credibility and security issues involved in hosting someone’s private data. These sorts of companies really want their wiki inside their firewall. They’re certainly not going to trust a company they’ve never really heard of, and have no business relationship with, to host all their critical business data.

So we’re still in a dead end. Lots of small organisations would benefit from a wiki, but they don’t know it yet, and aren’t prepared for the social changes that would be needed in their organisation. And for those that do know this, and are ready for it, the process is far too complex. And if they do get up and running, but later realise they’ve picked the wrong implementation, then they’re in even more trouble, because almost every wiki out there uses slightly different formatting and mark-up rules, and even if you can export your data, nothing will actually import it correctly.

This, of course, all adds up to a gigantic opportunity for people willing to solve all these issues. Few of them are simple. The combination of them all certainly isn’t. It’s not going to be an easy ride for the companies that try. But if were, then everyone would be doing it.

Six months ago, after a six month hiatus, I started blogging again for a couple of months. Then the events of life caught up with me again, and I’ve been silent again since then. This doesn’t mean I haven’t been blogging. In fact I’ve probably been blogging more than ever. It’s just that these days almost all of it happens behind the firewall, rather than to the wider world. And even then, a lot of it, isn’t strictly speaking “blogging” any more, but “wikiing”, (or whatever people are going to call that) as I’m finding it a better and better way to structure information.

There’s a certain irony in that, as it was a discussion with Dave Winer about wikis back in 2002 that started me blogging. Wiki software has come a long way since then. A lot of the wiki extensions are certainly interesting, and probably generally useful, although a lot of them seem to misunderstand the “wiki way”. But two features have completely changed the way I use wikis. Firstly, RSS feeds. I always knew that this would be useful on a wiki, but I hugely underestimated just how useful.

Once of the first things we did upon taking over the running of UNITE was to introduce a ticketing system for support enquiries, blogs for staff to narrate their work, and a wiki as a repository of general company information. These are, in many ways, now crucial to the functioning of the business. However, with three disparate systems, there’s a certain amount of information overlap. Not everyone looks at every support ticket, so wider information out of that has to usually either be blogged, or put on the wiki. Any information that may need to be known at some stage in the future in a specific context gets put on the wiki, hopefully in a sensible place. But any information that people might find more generally useful now gets put on a blog (everyone has to read the ‘Announce’ blog, but most people tend to at least skim everyone else’s personal blog as well). Until we switched to a wiki with RSS, some information would end up having to be copied around all three systems.

Now that’s much less of an issue. People don’t need to duplicate information onto the blog if they think it might be of random interest. The people who might be interested can now pick it up from the wiki. Where before you couldn’t ever rely on someone stumbling across it, now people can subscribe to the RSS and notice it passing by. Being able to see every change scrolling past also means that the wiki gets ‘refactored’ much more readily and regularly, which leads, of course, to a much more useful knowledge base.

The second wiki addition that has qualitatively changed how I work is the “discussion” page. I don’t know if Wikipedia was the first wiki to add this, but it’s certainly the most prominent example of it. There it allows articles to maintain their feel as bring part of a real encyclopaedia, even when the content is under heavy discussion. On Ward’s original c2.com wiki the discussions all took place in situ and on emerging topics it was often difficult to follow what was going on. Behind our firewall, this feature has found had a different application. On our wiki we store lots of correspondence. The ‘content’ / ‘discussion’ split allows us to have a page for a specific letter or email, with comments on it neatly separated out.

Switching one of our internal wikis to MediaWiki rather than Kwiki has also convinced me of the disproportionate difference it makes to be able to have “real names” for pages, rather than “WikiNames”. My software development background means I can live quite happily with CamelCase, as can most of the staff of in an ISP. But the switch to MediaWiki style naming has been hugely freeing, and I now find it really frustrating every time I have to find an unnatural way to make a WikiName on our older wiki.

As to future developments, I’ve been watching Jotspot quite closely since I saw the demo at Web 2.0 last October. (Jon Udell’s screencast gives a great overview). The ease of adding structured information to the unstructured (or at best semi-structured) wiki data has huge potential. It’s probably still a little too cumbersome for widespread adoption at this point, but it’s certainly something to watch.

I have some big ideas in this whole area, so I expect I’ll be talking a bit more about wikis soon…

The most distinctive characteristic of a wiki is that anyone in the group can edit, modify or even delete material on the pages. Such a free-form collaborative process can be messy and chaotic, and it requires a commitment to the group that may not sit well with some egos. But over time, wiki advocates say, a group voice or consensus emerges into what some enthusiasts call “emergent intelligence.”

The creative anarchy of the wiki is the philosophical inverse of conventional corporate groupware software. Groupware’s highly structured rules and processes do not always reflect the way people really work. Employees often ignore costly corporate-sanctioned software and revert to informal social networks – whether simply e-mail or impromptu water-cooler discussions.

— New York Times [via Chris Winters]

One of the really neat features about Wikis is the back linking – click on the title of any page, and see what other pages link to it. I was talking to Ken MacLeod about this a few days ago, and how we could apply it to weblog entries (after the conversation flow problems I talked about a few days ago).

Now I see that Joel has added automatic linkbacks, beating me to it! Nice.