Creating Technology for Social Change

Clay Shirky: Planning Shan’t Trump Reality

Clay Shirky‘s on stage with Ginny Hunt at Harvard’s Institute of Politics discussing the lessons we can take away from the boondoggle (#netrevolution).

Clay’s first point is that of all the criticism of and the Affordable Care Act, no one has argued that it’s a bad idea to rely on the web as the central component of citizen interaction with a government program. All of the other communications options, from phone to fax, have been considered second-rate fallback options.

This change has happened almost imperceptibly, but it is nevertheless a marker of where we are.

There’s a lesson to be learned from the website’s poor performance, especially given Obama for America’s campaign success with technology.

Internet technology and politics have hooked up every 2-4 years since 1992, when Clinton hosted an internal campaign listserv on MIT servers. Now, the internet and politics have gotten married.

Clay’s amazed that President Obama was allowed by his staff to get up and compare to on October 1st, when the site was already beginning to fail. Worse, he drove more people to a site already buckling under its traffic load. That’s not a technology problem, that’s a problem with political culture, and an inability to tell the boss the truth. Clay expects change within the government’s technology leadership.

Reading internal communications, it was clear that the Republican opposition’s implacable complaining weighed on the platform’s creators. But even if we assume the basest, most political motives of avoiding this criticism,’s strategy of failing big, all at once, failed hard. The history of federal government IT projects is a graveyard of hugely expensive failed projects. With, even the political rationale of escaping blame failed.


Clay Shirky and Ginny Hunt at Harvard

The core question we should look to is: When is reality allowed to trump planning?

In Washington, the standard answer to this question is ‘never’. Clay blames the Waterfall model of software development, which, he says, requires participants to promise that they will learn nothing during the actual building of the technology. It asks people to be at their wisest and most experienced at the beginning of the project, before they’ve actually attempted to do anything.

This model is broken, Clay says.

Civic technologists need to figure out how to introduce more reality into the planning of government.

We can look to software’s agile management model, which allows us to fail iteratively and push changes frequently. If we don’t do this, Clay says, none of the procurement or hiring changes also being called for will make much difference. If you can’t tell your boss’s boss that the project’s going to bomb, it’s going to bomb.

Clay compares the timing of likely failure to jumping 39 stories all at once, or jumping one story 39 times, and getting pretty good at it. By saving up all the failure for launch day, they failed as big as possible.

Government can do technology right. Look at NASA, the Pentagon, and the NSA’s ability to hire technology contractors to competently build challenging technology. The difference is that someone on the inside has significant technical authority. They know the tech, and they can override the sense that politics has to trump reality.

Clay recalls the assembly lines of Chrysler and Toyota in the 1980s, when Toyota was eating Chrysler’s lunch. At Chrysler, assembly line workers encountering a dysfunctional part would take the part off the line and keep assembling. At Toyota, every person working on the line had the authority to pause everything. Although initially inefficient, this behavior creates incentives to fix the problem that brought the dysfunctional part to that worker in the first place.

In management theory, companies like Toyota are called double loop organizations. Single loop organizations fix problems. Double loop organizations fix problems as well as their causes.

We won’t get vital civic technology without people who understand the technology being able to pause the assembly line.

Clay regularly asks his students to write post-mortems mid-project. They can often say, before the project is finished, what’s going to go wrong. People knew what was going to go wrong with well before it did. The trick is to change government so that those people are heard early enough.

failure is an option

Failure is always an option

Engineers understand this. The screenwriters of Apollo 13 wrote the “Failure is not an option!” line as a fantastical interpretation of how managers and engineers should interact. If you ever yell that at an engineer, Clay suggests you wash your mouth out with soap. And if you’re an engineer and anyone ever yells that at you, you might want to look into another job.