Today Open Knowledge and the Open Definition Advisory Council are pleased to announce the release of version 2.0 of the Open Definition. The Definition “sets out principles that define openness in relation to data and content” and plays a key role in supporting the growing open data ecosystem.
Adapted from an impromptu rant I gave to some people interested in funding government transparency projects.
I’ve spent the past year and change working on a site, watchdog.net, that publishes government information online. In doing that, I’ve learned a lot: I’ve looked at everything from pollution records to voter registration databases and I’ve figured out a number of bureacratic tricks to get information out of the government. But I’ve also become increasingly skeptical of the transparency project in general, at least as it’s carried out in the US.
The way a typical US transparency project works is pretty simple. You find a government database, work hard to get or parse a copy, and then put it online with some nice visualizations.
The problem is that reality doesn’t live in the databases. Instead, the databases that are made available, even if grudgingly, form a kind of official cover story, a veil of lies over the real workings of government. If you visit a site like GovTrack, which publishes information on what Congresspeople are up to, you find that all of Congress’s votes are on inane items like declaring holidays and naming post offices. The real action is buried in obscure subchapters of innocuous-sounding bills and voted on under emergency provisions that let everything happen without public disclosure.
So government transparency sites end up having three possible effects. The vast majority of them simply promote these official cover stories, misleading the public about what’s really going on. The unusually cutting ones simply make plain the mindnumbing universality of waste and corruption, and thus promote apathy. And on very rare occasions you have a “success”: an extreme case is located through your work, brought to justice, and then everyone goes home thinking the problem has been solved, as the real corruption continues on as before.
In short, the generous impulses behind transparency sites end up doing more harm than good.
But this is nothing new. The whole history of the “good government” movement in the US is of “reformers” who, intentionally or otherwise, weakened the cause of democracy. They too were primarily supported by large foundations, mostly Ford and Rockefeller. They replaced democratically-elected mayors with professional city managers, which required a supermajority to overrule. They insisted on nonpartisan elections, making it difficult to organize people into political blocs. Arguing it would reduce corruption, they insisted city politicians serve without paying, ensuring the jobs were only open to the wealthy.
I worry that transparency groups may be making the same “mistake”.
These are some dark thoughts, so I want to add a helpful alternative: journalism. Investigative journalism lives up to the promise that transparency sites make. Let me give three examples: Silverstein, Taibbi, Caro.
Ken Silverstein regularly writes brilliant pieces about the influence of money in politics. And he uses these sorts of databases to do so. But the databases are always a small part of a larger picture, supplemented with interviews, documents, and even undercover investigation — he recently did a piece where he posted as a representative of the government of Turkmenistan and described how he was wined and dined by lobbyists eager to build support for that noxious regime. The story, and much more, is told in his book Turkmeniscam. (His book Washington Babylon is similarly indispensible.)
Matt Taibbi, in his book The Great Derangement, describes how Congress really works. He goes to the capitol and lays out the whole scene: the Congressmen naming post offices on the House floor, the journalists typing in the press releases they’re handed, the key actions going on behind the scenes and out of the public eye, the continual use of emergency procedures to evade disclosure laws.
And Robert Caro, in his incredible book The Power Broker (one of the very best books ever published, I’m convinced) takes on this fundamental political question of “Who’s actually responsible for what my government is doing?” For forty years, everyone in New York thought they knew the answer: power was held by the city council, the mayor, the state legislature, and the governor. After all, they run the government, right?
And for forty years, they were all wrong. Power was held — held, for the most part, absolutely, without any checks or outside influence — by one man: Parks Commissioner Robert Moses. All that time, everyone (especially the press) treated Robert Moses as merely the Parks Commissioner, a mere public servant serving his elected officials. In reality, he pulled the strings of all those elected officials.
These journalists tackled all the major questions supposedly addressed by US transparency sites — who’s buying influence? what is Congress doing? who’s in power in my neighborhood? — and not only tell a richer, more informative story, but come to strikingly different answers to the questions. In this era where investigative reporting budgets have been cut to the bone and newspapers are folding left and right, it’s fallen to nonprofits like ProPublica and the Center for Independent Media and, from a previous era, the Center for Public Integrity, to pick up the slack. They’ve been using the Internet in innovative ways to supplement good old-fashioned narrative journalism, where transparency sites are a supplement, rather than an end-in-themselves.
For too long we’ve been funding transparency projects on the model of if-we-build-it-they-will-come: that we don’t know what transparency will be useful for, but once it’s done it will lead to all sorts of exciting possibilities. Well, we’ve built it. And they haven’t come. The only success story its proponents can point to is that transparency projects have bred even more transparency projects. I’m done working on watchdog.net; I’m done hurting America. It’s time to give old-fashioned narrative journalism a try.
Previously: Disinfecting the Sunlight Foundation [November 2006]
Alex Howard has written an excellent article over at the O'Reilly Radar listing eight business models for government open data, a handy list for those in government agencies attempting to justify to senior management or Ministers why releasing government data is important and valuable.