When software eats the world
One part of our brains knows that software can be fragile. Another part of our brains, when faced with the choice of trusting the human or trusting the machine...trusts the machine. It may have been easier to pry trust away from the machine twenty years ago, when systems crashed more often, sometimes ruining months of work and the mantra, "Have you tried turning it off and back on again?" didn't yet work as a reliable way of restoring function. Perhaps more important, we didn't *have* to trust software because we had canonical hard copies. Then, as predicted, the copies became "backups". Now, often, they don't exist at all, with the result that much of what we think we know is becoming less well-attested. How many of us even print out our bank statements any more? Three recent stories highlight this.
First is the biggest UK computer-related scandal for many years, the outrageous Post Office prosecution of hundreds of subpostmasters of theft and accounting fraud, all while insisting that their protests of innocence must all be lies because its software, sourced from Fujitsu, could not possibly be wrong. Eventually, the Court of Appeal quashed 39 convictions and excoriated both the Post Office and Fujitsu for denying the existence of two known bugs that led to accounting discrepancies. They should never have been able to get away with their claim of infallibility - first, because generations of software engineers could have told the court that all software has bugs, and second, because Ross Anderson's work proving that software vulnerabilities were the cause of phantom ATM withdrawals, overriding the UK banking industry's insistence that its software, too, was infallible.
At Lawfare, Susan Landau, discussing work she did in collaboration with Steve Bellovin, Matt Blaze, and Brian Owsley. uses the Post Office fiasco as a jumping-off point to discuss the increasing problem of bugs in software used to produce evidence presented in court. Much of what we think of as "truth" - Breathalyzer readings, forensic tools, Hawkeye line calls in tennis matches - are not direct measurements but software-derived interpretations of measurements. Hawkeye at least publishes its margin for error even though tennis has decided to pretend it doesn't exist. Manufacturers of evidence-producing software, however, claim commercial protection, leaving defendants unable to challenge the claims being made about them. Landau and her co-authors conclude that courts must recognize that they can't assume the reliability of evidence produced bysoftware and that defendants must be able to conduct "adversarial audits".
Second story. At The Atlantic, Jonathan Zittrain complains that the Internet is "rotting". Link rot - broken links when pages get deleted or reorganized - and content drift, which sees the contents of a linked page change over time, are familiar problems for anyone who posts anything online. Gabriel Weinberg, the founder of search engine DuckDuckGo, has has talked about API rot, which breaks dependent functionality. Zittrain's particular concern is legal judgments, which increasingly may incorporate disappeared or changed online references like TikTok videos and ebooks. Ebooks in particular can be altered on the fly, leaving no trace of that thing you distinctly remember seeing.
Zittrain's response has been to help create sites to track these alterations and provide permanent links. It probably doesn't matter much that the net.wars archive has (probably) thousands of broken links. As long as the Internet Archive's Wayback Machine continues to exist as a source for vaped web pages, most of the ends of those links can be recovered. The Archive is inevitably incomplete, and only covers the open web. But it *does* matter if the basis for a nation's legal reasoning and precedents - what Zittrain calls "long-term writing" - can't be established with any certainty. Hence the enormous effort put in by the UK's National Archives to convert millions of pages of EU legislation so all could understand the legitimacy of post-Brexit UK law.
Third story. It turns out the same is true for the brick-by-brick enterprise we call science. In the 2020 study Open is not forever, authors Mikael Laakso, Lisa Matthias, and Najko Jahn find journal rot. Print publications are carefully curated and preserved by librarians and archivists, as well as the (admittedly well-funded) companies that publish them. Open access journals, however, have had a patchy record of success, and the study finds that between 2000 and 2019 174 open access journals from all major research disciplines and from all geographical regions vanished from the web. In science, as in law, it's not enough to retain the end result; you must be able to show your work and replicate your reasoning.
It's more than 20 years since I heard experts begin to fret about the uncertain durability of digital media; the Foundation for Information Research included the need for reliable archives in its 1998 founding statement. The authors of the journal study note that the journals themselves are responsible for maintaining their archives and preserving their portion of the scholarly record; they conclude that solving this problem will require the participation of the entire scholarly community.
What isn't clear, at least to me, is how we assure the durability of the solutions. It seemed a lot easier when it was all on paper in a reassuringly solid building.
Illustrations: The UK National Archives, in Kew (photo by Erian Evans via Wikimedia)..
Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.