At his blog, the science fiction writer Charles Stross has a posting in which he calls modern corporations 'old, slow AIs'. Partly, Stross was responding to the philosopher Nick Bostrom, whose book Superintelligence (2014) warned of 'paperclip maximisers' — that is, AIs programmed to optimise for one thing, no matter the wider costs. Such an AI could use up all the planet's resources and kill all the humans in the service of maximising paperclip output. Substitute profits for paperclips, and you have corporations, even if the owners write 'Don't be evil' in the IPO prospectus.
That is what Harvard professor Shoshana Zuboff is talking about when she says in The Age of Surveillance Capitalism that she is writing about logic, not technology. Today's largest companies (Google and Facebook are her primary, but not sole, focus) see all human behaviour as free raw material for them to exploit as they see fit, and the technology they build follows that logic. You could call it cultural appropriation on a global scale, or you could use Zuboff's more economist-style words, 'the commodification of behaviour'.
Or, you could use the title's 'surveillance capitalism', which seems such a natural term for the relentless way ad brokers and would-be exploiters track us through online (and, increasingly, physical) space that it's a surprise it has a creator. Its invention is widely attributed to Zuboff, who first used it in a 2014 essay.
Most discussions of how we got here focus on surveillance and its expansion outward from government — primarily the intelligence agencies and the military. By contrast, Zuboff grounds her argument in the shift from industrialisation to 'informatisation', and examines how capitalism has been distorted by the new captains of industry. Her discussion is broad enough to encompass Émile Durkheim's study of the division of labour, Thomas Piketty's study of capital, and the community claims of Mark Zuckerberg. The book concludes with a call to action: we should harness our indignation and reclaim our digital future. Privacy advocates and scholars are embracing this book as a major contribution, and the 2019 Computers, Privacy, and Data Protection (CPDP) conference seemed wowed by Zuboff's summary presentation.
And yet. Even though it could stand some streamlining, the book has some unexpected gaps. Zuboff has picked August 9, 2011 as an inflection point. On that day, she writes, Apple (we now know temporarily) became the world's largest company by market capitalisation, a police shooting sparked riots in London, and the New York Times noted the beginning of the court cases establishing the EU's right to be forgotten.
The conflation is somewhat contrived. The right to be forgotten had been discussed for years and was explored at book length in 2009 in Viktor Mayer-Schönberger's Delete, which Zuboff appears to have missed. Also missing are antecedents such as IBM's provision of tabulation technologies to the Third Reich and previously proposed methods of achieving freedom and control, such as 'semantic discontinuity', which Georgetown Law professor Julie E. Cohen outlined in 2012 (PDF). At CPDP, Cohen noted the many other trends in economics, capitalism, law and labour relations that have developed over decades. Some of the worst current corporate scandals — think VW — don't involve data or personal surveillance at all.
The Age of Surveillance Capitalism fits into an important shift in how we think about privacy. We have moved on from talking about whether we as individuals have something to hide. Our conversations about privacy now concern the survival of democracy: individual autonomy and human dignity are only part of it.