The cost of surveillance
A friend - a cultured friend with a fine understanding of the Internet - commented recently that maybe mass surveillance wasn't such a bad idea if it meant people not getting blown up. It seems worthwhile to document here what I should have said in case I need it again.
Everyone supports the goal that people should not get blown up. But.
For now, we will make some stipulations. Let's ignore that being under constant observation would likely mean greater social conformity; he granted that. In arguing that this might not be a bad thing, he cited the "honest experiment". This was conducted in 2006 at the University of Newcastle in 2006, where an honor system was in place: you took your coffee and dropped the suggested amount in an honesty box. For ten weeks, psychologists Melissa Bateson, Daniel Nettle, and Gabriel Roberts added a picture over the box that alternated weekly between a flower and a pair of eyes. The result: the eyes picture correlated with 2.76 times the money collected (PDF).
My suspicion is that ten weeks isn't long, and that it's possible that after a longer time of habituation the effect would fade, like clicking "OK". Call it a quibble, and let's say for the purposes of this discussion that we don't care about chilling effects, despite their effect on things like investigative journalism; useful blogs like Groklaw; Internet use, free speech, and religious practices in targeted populations; or inhibits public debate .
We didn't discuss the cost of the surveillance infrastructure. Given that any society has limited resources, tradeoffs must be made. The billions spent on spying are billions unavailable for health care, education, or closing the poverty gap that helps create angry, alienated radicals, Granted, costs are dropping on a micro level, but not on a macro level. As John Mueller and Mark G. Stewart point out in a January 2014 paper studying the 215 program(PDF), even if you minimize concerns about privacy and civil liberties concerns, the program that collects and analyzes all that metadata would likely fail a cost-benefit test. Besides the direct cost, which the NSA does not disclose, the huge indirect costs include opportunity costs as the FBI chases down the millions of useless leads churned out of the data. Working from the publicly available information on the 53 terrorist plots that have come to light in the US since 9/11, involving under 100 suspected terrorists, the paper notes, "Overall, where the plots have been disrupted, the task was accomplished by ordinary policing methods. The NSA programs scarcely come up at all." Using generous assumptions, the authors estimate that the program would be cost-effective only if its full cost is less than $33.3 million a year, out of the NSA's annual budget of $10 billion.
But let's say that either we don't care or that it's worth it. Let's also ignore the statistic Bruce Schneier likes to cite about the 500 extra car deaths a year when people avoid flying because of post-9/11 airport security.
And, just for grins, let's assume that the spy agency itself is trustworthy and would never abuse the data in its control.
What, then, is wrong with surveillance?
A platform built for spying can be coopted by others for their own purposes. When you create the mechanisms that could underpin a police state you are creating a system that will be dangerous if one ever arises.
More immediately, having spent today listening to some quite terrifying real scenarios about cyberattacks mounted by criminals and nation-states, I think we must take seriously the danger that such a platform can be penetrated and subverted. The old threat model was blackmail: your secrets could be used against you and anyone you worked for. Today, our vulnerabilities are our children and loved ones, whose details on social networks can be linked, matched, and studied to craft utterly convincing spear-phishing messages that a specialist working for a security company may open - and with it awormhole into products entering into critical infrastructure. A spy platform is a spy platform. If you can't break in from the outside, insiders can be bribed, blackmailed, or coerced - or grow your own specialists to deploy and serve as moles.
Spy systems tend to expand: function creep, one of those irregular verbs again. I collect data to catch terrorists; you use it to find criminals; he uses it to find litterbugs.
In her 2008 paper, Technological Due Process, Danielle Citron studied what happens when algorithms make decisions about people's lives. One of the underpinnings of democracy - law - loses out because of the difficulty of translating human-written law into binary code (as Ellen Ullman discussed in Close to the Machine.
Finally, the IF in "if it meant people not getting blown up" is significant. Everything I've seen suggests that even if such a system may work someday it does not work now, as noted above. Millions of us; tiny number of terrorists who become expert at evading surveillance. Much easier to watch us than catch them.
Now toss back in all the stuff we left out: the cost to democracy, the chilling effect on free association, freedom of expression, even search terms. Still worth it?
Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.