mystique, closeted homosexuals, church-going atheists, and back-alley
abortions. Eventually, though, they even produced people like you and me.
-
[...]
The free-market creed originated in Europe as a sweeping defense against
competition for scarce resources. The disciplines of competitive markets
promised to quiet unruly individuals and even transform them back into subjects
too preoccupied with survival to complain.
+
+
+ [...]
+
+ In Capital in the Twenty-First Century, the French economist Thomas
+ Piketty integrated years of income data to derive a general law of
+ accumulation: the rate of return on capital tends to exceed the rate of
+ economic growth. This tendency, summarized as r > g, is a dynamic that
+ produces ever-more-extreme income divergence and with it a range of
+ antidemocratic social consequences long predicted as harbingers of an
+ eventual crisis of capitalism.
+
+ [...]
+
+ Many scholars have taken to describing these new conditions as
+ neofeudalism, marked by the consolidation of elite wealth and power far
+ beyond the control of ordinary people and the mechanisms of democratic
+ consent.55 Piketty calls it a return to “patrimonial capitalism,” a
+ reversion to a premodern society in which one’s life chances depend
+ upon inherited wealth rather than meritocratic achievement.56
+
+ [...]
+
+ We now have the tools to grasp the collision in all of its destructive
+ complexity: what is unbearable is that economic and social inequalities
+ have reverted to the preindustrial “feudal” pattern but that we, the
+ people, have not. We are not illiterate peasants, serfs, or slaves.
+ Whether “middle class” or “marginalized,” we share
+
+ [...]
+
+ Nevertheless, Occupy revealed a similar conflict between inequality’s
+ facts and inequality’s feelings, expressed in a creatively
+ individualized political culture that insisted on “direct democracy”
+ and “horizontal leadership.”60 Some analysts concluded that it was this
+ conflict that ultimately crippled the movement, with its “inner core”
+ of leaders unwilling to compromise their highly individualized approach
+ in favor of the strategies and tactics required for a durable mass
+ movement.61 However
+
+ [...]
+
+ This is the existential contradiction of the second modernity that
+ defines our conditions of existence: we want to exercise control over
+ our own lives, but everywhere that control is thwarted.
+ Individualization has sent each one of us on the prowl for the
+ resources we need to ensure effective life, but at each turn we are
+ forced to do battle with an economics and politics from whose vantage
+ point we are but ciphers. We live in the knowledge that our lives have
+ unique value, but we are treated as invisible
+
+ [...]
+
+ The deepest contradiction of our time, the social philosopher Zygmunt
+ Bauman wrote, is “the yawning gap between the right of self-assertion
+ and the capacity to control the social settings which render such
+ self-assertion feasible. It is from that abysmal gap that the most
+ poisonous effluvia contaminating the lives of contemporary individuals
+ emanate.”
+
+ [...]
+
+ When it comes to genuine economic mutation, there is always a tension
+ between the new features of the form and its mother ship. A combination
+ of old and new is reconfigured in an unprecedented pattern.
+ Occasionally, the elements of a mutation find the right environment in
+ which to be “selected” for propagation. This is when the new form
+ stands a chance of becoming fully institutionalized and establishes its
+ unique migratory path toward the future. But it’s even more likely that
+ potential mutations meet their fate in “transition failure,” drawn back
+ by the gravitational pull of established practices.63
+
+ [...]
+
+ Among the many violations of advocacy expectations, ubiquitous
+ “terms-of-service agreements” were among the most pernicious.67 Legal
+ experts call these “contracts of adhesion” because they impose
+ take-it-or-leave-it conditions on users that stick to them whether they
+ like it or not.
+
+ [...]
+
+ These “contracts” impose an unwinnable infinite regress upon the user
+ that law professor Nancy Kim describes as “sadistic.”
+
+ [...]
+
+ The digital milieu has been essential to these degradations. Kim points
+ out that paper documents once imposed natural restraints on contracting
+ behavior simply by virtue of their cost to produce, distribute, and
+ archive. Paper contracts require a physical signature, limiting the
+ burden a firm is likely to impose on a customer by requiring her to
+ read multiple pages of fine print. Digital terms, in contrast, are
+ “weightless.
+
+ [...]
+
+ Radin calls this “private eminent domain,” a unilateral seizure of
+ rights without consent. She
+
+ [...]
+
+ Once firms understood that the courts were disposed to validate their
+ click-wrap and browse-wrap agreements, there was nothing to stop them
+ from expanding the reach of these degraded contracts “to extract from
+ consumers additional benefits unrelated to the transaction.”73 This
+ coincided with the discovery of behavioral surplus that we examine in
+ Chapter 3, as
+
+ [...]
+
+ 2008 two Carnegie Mellon professors calculated that a reasonable
+ reading of all the privacy policies that one encounters in a year would
+ require 76 full workdays at a national opportunity cost of $781
+ billion.75 The numbers are much higher today. Still, most
+
+ [...]
+
+ These developments reflect the simple truth that genuine economic
+ reformation takes time and that the internet world, its investors and
+ shareholders, were and are in a hurry. The credo of digital innovation
+ quickly turned to the language of disruption and an obsession with
+ speed, its campaigns conducted under the flag of “creative
+ destruction.” That famous, fateful phrase coined by evolutionary
+ economist Joseph Schumpeter was seized upon as a way to legitimate what
+ Silicon Valley euphemistically calls “permissionless innovation.”77
+ Destruction rhetoric promoted what I think of as a “boys and their
+ toys” theory of history, as if the winning hand in capitalism is about
+ blowing things up with new technologies. Schumpeter’s analysis was, in
+ fact, far more nuanced and complex than modern destruction rhetoric
+ suggests.
+
+ [...]
+
+ was as if a shark had been silently circling the depths all along, just
+ below the surface of the action, only to occasionally leap glistening
+ from the water in pursuit of a fresh bite of flesh
+
+ [...]
+
+ Over time, the shark revealed itself as a rapidly multiplying,
+ systemic, internally consistent new variant of information capitalism
+ that had set its sights on domination. An unprecedented formulation of
+ capitalism was elbowing its way into history: surveillance capitalism.
+
+ [...]
+
+ As we shall explore in detail throughout the coming chapters, thanks to
+ surveillance capitalism the resources for effective life that we seek
+ in the digital realm now come encumbered with a new breed of menace.
+ Under this new regime, the precise moment at which our needs are met is
+ also the precise moment at which our lives are plundered for behavioral
+ data, and all for the sake of others’ gain. The result is a perverse
+ amalgam of empowerment inextricably layered with diminishment. In the
+ absence of a decisive societal response that constrains or outlaws this
+ logic of accumulation, surveillance capitalism appears poised to become
+ the dominant form of capitalism in our time.
+
+ [...]
+
+ This left us wholly unprepared to defend ourselves from new companies
+ with imaginative names run by young geniuses that seemed able to
+ provide us with exactly what we yearn for at little or no cost. This
+ new regime’s most poignant harms, now and later, have been difficult to
+ grasp or theorize, blurred by extreme velocity and camouflaged by
+ expensive and illegible machine operations, secretive corporate
+ practices, masterful rhetorical misdirection, and purposeful cultural
+ misappropriation. On this road, terms whose meanings we take to be
+ positive or at least banal—“the open internet,” “interoperability,” and
+ “connectivity”—have been quietly harnessed to a market process in which
+ individuals are definitively cast as the means to others’ market ends.
+
+ [...]
+
+ The new harms we face entail challenges to the sanctity of the
+ individual, and chief among these challenges I count the elemental
+ rights that bear on individual sovereignty, including the right to the
+ future tense and the right to sanctuary. Each of these rights invokes
+ claims to individual agency and personal autonomy as essential
+ prerequisites to freedom of will and to the very concept of democratic
+ order.
+
+ [...]
+
+ The Spanish Data Protection Agency recognized that not all information
+ is worthy of immortality
+
+ [...]
+
+ As for the Spanish people, their Data Protection Agency, and the
+ European Court of Justice, the passage of time is likely to reveal
+ their achievements as a stirring early chapter in the longer story of
+ our fight for a third modern that is first and foremost a human future,
+ rooted in an inclusive democracy and committed to the individual’s
+ right to effective life. Their message is carefully inscribed for our
+ children to ponder: technological inevitability is as light as
+ democracy is heavy, as temporary as the scent of rose petals and the
+ taste of honey are enduring.
+
+ [...]
+
+ The point for us is that every successful vaccine begins with a close
+ understanding of the enemy disease. The mental models, vocabularies,
+ and tools distilled from past catastrophes obstruct progress. We smell
+ smoke and rush to close doors to rooms that are already fated to
+ vanish. The result is like hurling snowballs at a smooth marble wall
+ only to watch them slide down its facade, leaving nothing but a wet
+ smear: a fine paid here, an operational detour there, a new encryption
+ package there.
+
+ [...]
+
+ It is the habitat for progress “at the speed of dreams,” as one Google
+ engineer vividly describes it.100 My aim here is to slow down the
+ action in order to enlarge the space for such debate and unmask the
+ tendencies of these new creations as they amplify inequality, intensify
+ social hierarchy, exacerbate exclusion
+
+ [...]
+
+ Peter Drucker’s field studies for his seminal Concept of the
+ Corporation, the 1946 book that codified the practices of the twentieth
+
+ [...]
+
+ The closest thing we have to a Buck Weaver or James Couzens codifying
+ Google’s practices and objectives is the company’s longtime chief
+ economist, Hal Varian, who aids the cause of understanding with
+ scholarly articles that explore important themes. Varian has been
+ described as “the Adam Smith of the discipline of Googlenomics” and the
+ “godfather” of its advertising model.6 It is in
+
+ [...]
+
+ Nowadays there is a computer in the middle of virtually every
+ transaction… now that they are available these computers have several
+ other uses.”8 He then identifies four such new uses: “data extraction
+ and analysis,” “new contractual forms due to better monitoring,”
+ “personalization and customization,” and “continuous
+ experiments.”Varian’s discussions of
+
+ [...]
+
+ Data extraction and analysis,” Varian writes, “is what everyone is
+ talking about when they talk about big data.” “Data” are the raw
+ material necessary for surveillance capitalism’s novel manufacturing
+ processes. “Extraction” describes the social relations and material
+ infrastructure with which the firm asserts authority over those raw
+ materials to achieve economies of scale in its raw-material supply
+ operations. “Analysis” refers to the complex of highly specialized
+ computational systems that I will generally refer to in these chapters
+ as “machine intelligence.” I like this umbrella phrase because it
+ trains us on the forest rather than the trees, helping us decenter from
+ technology to its objectives. But in choosing this phrase I also follow
+ Google’s lead. The company describes itself “at the forefront of
+ innovation in machine intelligence,” a term in which it includes
+ machine learning as well as “classical” algorithmic production, along
+ with many computational operations that
+
+ [...]
+
+ Amit Patel, a young Stanford graduate student with a special interest
+ in “data mining,” is frequently credited with the groundbreaking
+ insight into the significance of Google’s accidental data caches. His
+
+ [...]
+
+ Google’s engineers soon grasped that the continuous flows of collateral
+ behavioral data could turn the search engine into a recursive learning
+ system that constantly improved search results and spurred product
+ innovations such as spell check,
+
+ [...]
+
+ was Google that recognized the gold dust in the detritus of its
+ interactions with its users and took the trouble to collect it up.…
+ Google exploits information that is a by-product of user interactions,
+ or data exhaust, which is automatically recycled
+
+ [...]
+
+ that early stage of Google’s development, the feedback loops involved
+ in improving its Search functions produced a balance of power: Search
+ needed people to learn from, and people needed Search
+
+ [...]
+
+ Hal Varian, who joined Google as its chief economist in 2002, would
+ note, “Every action a user performs is considered a signal to be
+ analyzed and fed back into the system.”16 The Page Rank algorithm,
+ named after its founder
+
+ [...]
+
+ The key point for us rests on a critical distinction. During this early
+ period, behavioral data were put to work entirely on the user’s behalf.
+ User data provided value at no cost, and that value was reinvested in
+ the user experience in the form of improved services: enhancements that
+ were also offered at no cost to users. Users provided the raw material
+ in the form of behavioral data, and those data were harvested to
+ improve speed, accuracy, and relevance and to help build ancillary
+ products such as translation. I call this the behavioral value
+ reinvestment cycle, in which all behavioral data are reinvested in the
+ improvement of the product or service (see Figure 1).
+
+ [...]
+
+ cycle was similarly oriented toward the individual as its subject, but
+ without a physical product to sell, it floated outside the marketplace,
+ an interaction with “users” rather than a market transaction with
+ customers.
+
+ [...]
+
+ Users are not paid for their labor, nor do they operate the means of
+ production, as we’ll discuss in more depth later in this chapter.
+ Finally, people often say that the user is the “product.” This is also
+ misleading, and it is a point that we will revisit more than once. For
+ now let’s say that users are not products, but rather we are the
+ sources of raw-material supply. As we shall see, surveillance
+ capitalism
+
+ [...]
+
+ impatient money
+
+ [...]
+
+ These behavioral data available for uses beyond service improvement
+ constituted a surplus, and it was on the strength of this behavioral
+ surplus that the young company would find its way to the “sustained and
+ exponential profits” that would be necessary for survival. Thanks to a
+ perceived
+
+ [...]
+
+ the New York Times reported, “The precision of the Carol Brady data was
+ eye-opening for some.” Even Brin was stunned by the clarity of Search’s
+ predictive power, revealing events and trends before they “hit the
+ radar” of traditional media. As he told the Times, “It was like trying
+ an electron microscope for the first time. It was like a
+ moment-by-moment barometer
+
+ [...]
+
+ Google maximizes the revenue it gets from that precious real estate by
+ giving its best position to the advertiser who is likely to pay Google
+ the most in total, based on the price per click multiplied by Google’s
+ estimate of the likelihood that someone will actually click on the
+ ad.”42 That pivotal multiplier was the result of Google’s advanced
+ computational capabilities trained on its most significant and secret
+ discovery: behavioral surplus
+
+ [...]
+
+ competitors, in which ads were targeted to keywords or content, were
+ unable to identify relevant ads “for a particular user.” Now the
+ inventors offered a scientific solution that exceeded the
+ most-ambitious dreams of any advertising executive
+
+ [...]
+
+ This new Google assures its actual customers that it will do whatever
+ it takes to transform the natural obscurity of human desire into
+ scientific fact. This Google is the superpower that establishes its own
+ values and pursues its own purposes above and beyond the social
+ contracts to which others are bound.
+
+ [...]
+
+ Google’s unique auction methods and capabilities earned a great deal of
+ attention, which distracted observers from reflecting on exactly what
+ was being auctioned: derivatives of behavioral surplus. Click-through
+ metrics institutionalized “customer” demand for these prediction
+ products and thus established the central importance of economies of
+ scale in surplus supply operations. Surplus capture would have to
+ become automatic and ubiquitous if the new logic was to succeed, as
+ measured by the successful trading of behavioral futures.
+
+ [...]
+
+ kind of commerce that depended upon online surveillance at scale.
+ Insiders referred to Google’s new science of behavioral
+
+ [...]
+
+ here was an unprecedented and lucrative brew: behavioral surplus, data
+ science, material infrastructure, computational power, algorithmic
+ systems, and automated platforms. This convergence produced
+ unprecedented “relevance” and billions of auctions. Click-through rates
+ skyrocketed. Work on AdWords and AdSense became just as important as
+ work on Search
+
+ [...]
+
+ their community effectively declared a “state of exception” in which it
+ was judged necessary to suspend the values and principles that had
+ guided Google’s founding and early practices.
+
+ [...]
+
+ Google’s inventions, their origins in emergency, and the 180-degree
+ turn from serving users to surveilling them. Most of all, he credited
+ the discovery of behavioral surplus as the game-changing asset that
+ turned Google into a fortune-telling giant, pinpointing Google’s
+ breakthrough transformation of the Overture model, when the young
+ company first applied its analytics of behavioral surplus to predict
+ the likelihood of a click:
+
+ [...]
+
+ Google loosed a new incarnation of capitalism upon the world, a
+ Pandora’s box whose contents we are only beginning
+
+ [...]
+
+ On the strength of Google’s inventions, discoveries, and strategies, it
+ became the mother ship and ideal type of a new economic logic based on
+ fortune-telling and selling—an ancient and eternally lucrative craft
+ that has fed on humanity’s confrontation with uncertainty from the
+ beginning of the human story.
+
+ [...]
+
+ The scientific and material complexity that supported the capture and
+ analysis of behavioral surplus also enabled the hiding strategy, an
+ invisibility cloak over the whole operation. “Managing search at our
+ scale is a very serious barrier to entry,” Schmidt warned would-be
+ competitors.79 To be sure, there are always sound business
+
+ [...]
+
+ public were told that Google’s magic derived from its exclusive
+ capabilities in unilateral surveillance of online behavior and its
+ methods specifically designed to override individual decision rights?
+ Google policies had to enforce secrecy in order to protect operations
+ that were designed to be undetectable because they took things from
+ users without asking and employed those unilaterally claimed resources
+ to work in the service of others’ purposes.
+
+ [...]
+
+ George Orwell once observed that euphemisms are used in politics, war,
+ and business as instruments that “make lies sound truthful and murder
+ respectable.”81 Google has been careful to camouflage the significance
+ of
+
+ [...]
+
+ Google discovered this necessary element of the new logic of
+ accumulation: it must assert the rights to take the information upon
+ which its success depends.
+
+ [...]
+
+ signing on with Facebook, the talented Sandberg became the “Typhoid
+ Mary” of surveillance capitalism as she led Facebook’s transformation
+ from a social networking site to an advertising behemoth. Sandberg
+ understood that Facebook’s social graph represented an awe-inspiring
+ source of behavioral surplus: the extractor’s equivalent of a
+ nineteenth-century prospector stumbling into a valley that sheltered
+ the largest diamond mine and the deepest gold mine ever to be
+ discovered. “We have better information than anyone else. We know
+ gender, age, location, and it’s real data as opposed to the stuff other
+ people infer,” Sandberg
+
+ [...]
+
+ Sandberg understood that through the artful manipulation of Facebook’s
+ culture of intimacy and sharing, it would be possible to use behavioral
+ surplus not only to satisfy demand but also to create demand. For
+ starters, that meant inserting advertisers into the fabric of
+ Facebook’s online culture, where they could
+
+ [...]
+
+ This new market form declares that serving the genuine needs of people
+ is less lucrative, and therefore less important, than selling
+ predictions of their behavior. Google discovered that we are less
+ valuable than others’ bets on our future behavior. This changed
+ everything.
+
+ [...]
+
+ VIII. Summarizing the Logic and Operations of Surveillance Capitalism
+
+ [...]
+
+ is obscene to suppose that this harm can be reduced to the obvious fact
+ that users receive no fee for the raw material they supply. That
+ critique is a feat of misdirection that would use a
+
+ [...]
+
+ remarkable questions here concern the facts that our lives are rendered
+ as behavioral data in the first place; that ignorance is a condition of
+ this ubiquitous rendition; that decision rights vanish before one even
+ knows that there is a decision to make; that there are consequences to
+ this diminishment of rights that we can neither see nor foretell; that
+ there is no exit, no voice, and no loyalty, only helplessness,
+ resignation, and psychic numbing; and that encryption is the only
+ positive action left to discuss when we sit around the dinner table and
+ casually ponder how to hide from the forces that hide from us.
+
+ [...]
+
+ Social theorist David Harvey builds on Arendt’s insight with his notion
+ of “accumulation by dispossession”: “What accumulation by dispossession
+ does is to release a set of assets… at very low (and in some instances
+ zero) cost. Overaccumulated capital can seize hold of such assets and
+ immediately turn them to profitable use.” He adds that entrepreneurs
+ who are determined to “join the system” and enjoy “the benefits of
+ capital accumulation” are often the ones who drive this
+
+ [...]
+
+ Even when knowledge derived from our behavior is fed back to us as a
+ quid pro quo for participation, as in the case of so-called
+ “personalization,” parallel secret operations pursue the conversion of
+ surplus into sales that point far beyond our interests. We have no
+ formal control because we are not essential to this market action. In
+ this future we are exiles from our own behavior, denied access to or
+ control over knowledge derived from its dispossession by others for
+ others. Knowledge
+
+ [...]
+
+ When asked about government regulation, Schmidt said that technology
+ moves so fast that governments really shouldn’t try to regulate it
+ because it will change too fast, and any problem will be solved by
+ technology. ‘We’ll move much faster than any government.’”26 Both Brin
+ and Page are even more candid in their contempt
+
+ [...]
+
+ Economic historians describe the dedication to lawlessness among the
+ Gilded Age “robber barons” for whom Herbert Spencer’s social Darwinism
+ played the same role that Hayek, Jensen, and even Ayn Rand play for
+ today’s digital barons. In the same way that surveillance capitalists
+ excuse their corporations’ unprecedented
+
+ [...]
+
+ There was no need for law, they argued, when one had the “law of
+ evolution,” the “laws of capital,” and the “laws of industrial
+ society.” John Rockefeller insisted that his outsized oil fortune was
+ the result of “the natural law of trade development.” Jay Gould, when
+ questioned by Congress on the need for federal regulation of railroad
+ rates, replied that rates were already regulated by “the laws of supply
+ and demand, production and consumption.”31 The millionaires mobilized
+ in 1896 to defeat the populist Democrat William Jennings Bryan, who had
+ vowed to tether economic policy to the political realm, including
+ regulating the railroads and protecting the people from “robbery
+
+ [...]
+
+ Surveillance After September 11, surveillance scholar David Lyon
+
+ [...]
+
+ After several decades in which data-protection officials, privacy
+ watchdogs, civil rights groups, and others have tried to mitigate
+ negative social effects of surveillance, we are witnessing a sharp tilt
+ toward more exclusionary and intrusive surveillance practices.”56 This
+ abrupt refocusing of governmental power and policy after the 9/11
+ attacks in New York City and Washington, DC
+
+ [...]
+
+ With the attacks of September 11, 2001, everything changed. The new
+ focus was overwhelmingly on security rather than privacy.”61 The
+ privacy provisions debated just months earlier vanished from the
+ conversation more or less overnight. In both the US Congress and across
+ the EU, legislation was
+
+ [...]
+
+ including Germany (a country that had been highly sensitized to
+ surveillance under the hammer of both Nazi and Stalinist
+ totalitarianism), the UK, and France.62 In the US the failure to
+ “connect the dots” on the terrorist attack was a source of shame and
+ dismay that overwhelmed other concerns. Policy guidelines shifted from
+ “need to know” to “need to share” as agencies were urged to tear down
+ walls and blend databases for comprehensive information and analysis.63
+ In a parallel development, privacy scholar Chris Jay
+
+ [...]
+
+ The elective affinity between public and private missions was evident
+ as early as 2002, when former NSA Chief Admiral John Poindexter
+ proposed his Total Information Awareness (TIA) program with a vision
+ that reads like an early guide to the foundational mechanisms of
+ behavioral surplus capture and analysis:
+
+ [...]
+
+ secret public-private intelligence collaborations that tend to be
+ “orchestrated around handshakes rather than legal formalities, such as
+ search warrants, and may be arranged this way to evade oversight and,
+ at times, to defy the law.”88 He observed that intelligence agencies
+ are irresistibly drawn to “and in some respects dependent upon” firms’
+ privately held data resources.89
+
+ [...]
+
+ former NSA Director Mike McConnell offered another glimpse into the
+ elective affinities between Google and the intelligence community.
+ Writing in the Washington Post, McConnell made clear that Google’s
+ surveillance-based operations in data capture, extraction, and analysis
+ were both taken for granted and coveted. Here the boundaries of private
+ and public melt in the intense heat of new threats and their
+ high-velocity demands that must be met in “milliseconds.” In
+ McConnell’s future there is one “seamless” surveillance empire in which
+ the requirements of self-preservation leave no opportunity for the
+ amenities of
+
+ [...]
+
+ Once again, history offers us no control groups
+
+ [...]
+
+ 1) the demonstration of Google’s unique capabilities as a source of
+ competitive advantage in electoral politics; (2) a deliberate blurring
+ of public and private interests through relationships and aggressive
+ lobbying activities; (3) a revolving door of personnel who migrated
+ between Google and the Obama administration, united by elective
+ affinities during Google’s crucial growth years of 2009–2016; and (4)
+ Google’s intentional campaign of influence over academic work and the
+ larger cultural conversation so vital to policy
+
+ [...]
+
+ Obama used his proximity to Schmidt to cement his own identity as the
+ innovation candidate poised to disrupt business as usual in
+ Washington.98 Once elected, Schmidt joined the Transition Economic
+ Advisory Board and appeared next to Obama at
+
+ [...]
+
+ Political correspondent Jim Rutenberg’s New York Times account of the
+ data scientists’ seminal role in the 2012 Obama victory offers a vivid
+ picture of the capture and analysis of behavioral surplus as a
+ political methodology. The campaign knew “every single wavering voter
+ in the country that it needed to persuade to vote for Obama, by name,
+ address, race, sex, and income,” and it had figured out how to target
+ its television ads to these individuals. One breakthrough was the
+ “persuasion score” that identified
+
+ [...]
+
+ According to the Center for Media and Democracy’s investigatory
+ research report, “The Googlization of the Far Right,” the corporation’s
+ 2012 list of grantees featured a new
+
+ [...]
+
+ Meanwhile, a list of Google Policy Fellows for 2014 included
+ individuals from a range of nonprofit organizations whom one would
+ expect to be leading the fight against that corporation’s
+ concentrations of information and power, including the Center for
+ Democracy and Technology, the Electronic Frontier Foundation, the
+ Future of Privacy Forum, the National Consumers League, the Citizen
+ Lab, and the Asociación por los Derechos Civiles.116 In July 2017 the
+ Wall Street Journal reported that
+
+ [...]
+
+ That summer, one of the New America Foundation’s most highly regarded
+ scholars and a specialist in digital monopolies, Barry Lynn, posted a
+ statement praising the EU’s historic decision to levy a $2.7 billion
+ fine on Google as the result of a multiyear antitrust investigation.
+ According to the New York Times and Lynn’s own account, New America’s
+ director bent to pressure from Schmidt, firing Lynn and his Open
+ Markets team of ten researchers. “Google is very aggressive in throwing
+ its money around Washington and Brussels, and then pulling strings,”
+ Lynn told the New York Times. “People are so afraid of Google now.” The
+ reporters cite Google
+
+ [...]
+
+ Google in the lead, surveillance capitalism vastly expanded the market
+ dynamic as it learned to expropriate human experience and translate it
+ into coveted behavioral predictions. Google and this larger
+ surveillance project have been birthed, sheltered, and nurtured to
+ success by the historical conditions of their era—second-modernity
+ needs, the neoliberal inheritance, and the realpolitik of surveillance
+ exceptionalism—as well as by their own purpose-built fortifications
+ designed to protect supply chain operations from scrutiny through
+ political and cultural capture.
+
+ [...]
+
+ the capture of behavioral surplus and the acquisition of decision
+ rights. Like a river running to the sea, if one route is blocked
+
+ [...]
+
+ increasingly ruthless cycle of kidnapping human experience, cornering
+ surplus supplies, and competing in new behavioral futures markets
+
+ [...]
+
+ The extraction imperative demands that everything be possessed. In this
+ new context, goods and services are merely surveillance-bound supply
+ routes. It’s not the car; it’s the behavioral data from driving the
+ car. It’s not the map; it’s the behavioral data from interacting with
+ the map. The ideal here is continuously expanding borders that
+ eventually describe the world and everything in it, all the time.
+
+ [...]
+
+ Traditionally, monopolies on goods and services disfigure markets by
+ unfairly eliminating competition in order to raise prices at will.
+ Under surveillance capitalism, however, many of the practices defined
+ as monopolistic actually function as means of cornering user-derived
+ raw-material supplies. There is no monetary price for the user to pay,
+ only an opportunity
+
+ [...]
+
+ The corporation unfairly impedes competitors in Search in order to
+ protect the dominance of its most important supply route, not primarily
+ to fix prices. These cornering operations are not abstractions,
+
+ [...]
+
+ products such as Android are valued more for supply than for sales.
+ Disconnect, Inc., founded in 2011 by two former Google engineers and a
+ privacy-rights attorney, developed
+
+ [...]
+
+ Google executive, noting that if other manufacturers switched to
+ Skyhook, it “would be awful for Google, because it will cut off our
+ ability to continue collecting data” for the company’s Wi-Fi location
+ database. Court documents from Skyhook’s eventual lawsuit against
+ Motorola (and Samsung) include an e-mail from Google’s senior vice
+ president of Mobile
+
+ [...]
+
+ Finally, extraordinary research from the French nonprofit Exodus
+ Privacy and the Yale Privacy Lab in 2017 documented the exponential
+ proliferation of tracking software. Exodus identified 44 trackers in
+ more than 300 apps for Google’s Android platform, some
+
+ [...]
+
+ For example, the ad tracker FidZup developed “communication between a
+ sonic emitter and a mobile phone.…” It can detect the presence of
+ mobile phones and therefore their owners by diffusing a tone, inaudible
+ to the human ear, inside a building: “Users installing
+
+ [...]
+
+ a pattern foreshadowed by the Google patent that we examined in Chapter
+ 3 and that we shall see repeatedly in the coming chapters, the research
+ findings emphasize that the always-on tracking is impervious to the
+ Android “permissions system,” despite its promises of user control.17
+
+ [...]
+
+ Disconnect software was banned from Google Play’s vast catalog of
+ mobile apps, leading to Disconnect’s lawsuit against Google in 2015.
+ The startup’s complaint explains that “advertising companies including
+ Google use these invisible
+
+ [...]
+
+ dispossession operations reveal a predictable sequence of stages that
+ must be crafted and orchestrated in great detail in order to achieve
+ their ultimate destination as a system of facts through which surplus
+ extraction is normalized.The four stages of the cycle are incursion,
+ habituation, adaptation, and redirection. Taken together, these stages
+ constitute a “theory of change” that describes and predicts
+ dispossession as a political and cultural
+
+ [...]
+
+ with Google’s wider practice: it’s great to empower people, but not too
+ much, lest they notice the pilfering of their decision rights and try
+ to reclaim them. The firm wants to enable people to make
+
+ [...]
+
+ Google’s ideal society is a population of distant users, not a
+ citizenry. It idealizes people who are informed, but only in the ways
+ that the corporation chooses. It means for us to be docile, harmonious,
+ and, above all, grateful.
+
+ [...]
+
+ Within days, an independent analysis by German security experts proved
+ decisively that Street View’s cars were extracting unencrypted personal
+ information from homes. Google was forced to concede that it had
+ intercepted and stored “payload data,” personal information grabbed
+ from unencrypted Wi-Fi transmissions. As its apologetic blog post
+ noted,
+
+ [...]
+
+ Google’s “Spy-Fi” scandal filled headlines around the world. Many
+ believed that the Street View revelations would inflict irreparable
+
+ [...]
+
+ April 2012 FCC report is heart wrenching in its way, a melancholic
+ depiction of democracy’s vulnerability in the face-off with a wealthy,
+ determined, and audacious surveillance capitalist opponent. In November
+ 2010 the FCC sent Google a letter of inquiry
+
+ [...]
+
+ The second point is that in retrospect, one sees that the very idea of
+ a single rogue engineer was designed and elaborated as a brilliant
+ piece of misdirection, a classic scapegoating ploy. It directed
+ attention away from the ambitious and controversial agenda of the
+ extraction imperative toward a different narrative of a single infected
+ cell excised from the flesh of an enormous but innocent organism. All
+ that was left was to excise the infected flesh and let the organism
+ declare itself cured of its privacy kleptomania. Then—a return to the
+ streets
+
+ [...]
+
+ This is to say that her job was a logical impossibility. That she may
+ have nevertheless taken it seriously is suggested by
+
+ [...]
+
+ Street View’s redirection and elaboration announced a critical shift in
+ the orientation and ambition of the surveillance program: it would no
+ longer be only about routes, but about routing
+
+ [...]
+
+ For now, suffice to say that Street View and the larger project of
+ Google Maps illustrate the new and even more ambitious goals toward
+ which this cycle of dispossession would soon point: the migration from
+ an online data source to a real-world monitor to an advisor to an
+ active shepherd—from knowledge to influence to control. Ultimately,
+ Street View’s elaborate data would become the basis for another complex
+ of spectacular Google incursions: the self-driving car and “Google
+ City,” which we learn more about in Chapter 7. Those programs aim to
+ take surplus capture to new levels while opening up substantial new
+ frontiers for the establishment of behavioral futures markets in the
+ real world of goods and services. It is important to understand that
+ each level of innovation builds on the one before and that all are
+ united in one aim, the extraction of behavioral surplus at scale.In
+ this progression, Google perceives an opportunity
+
+ [...]
+
+ Google discovered by chance or intention the source of every mapmaker’s
+ power
+
+ [...]
+
+ The first US rectangular land survey captured this language perfectly
+ in its slogan: “Order upon the Land.”72 The cartographer is the
+ instrument of power as the author of that order, reducing reality to
+ only two conditions: the map and oblivion. The cartographer’s truth
+ crystallizes the message that Google and all surveillance capitalists
+ must impress upon all humans: if you are not on our map, you do not
+ exist
+
+ [...]
+
+ Google has done incrementally and furtively what would plainly be
+ illegal
+
+ [...]
+
+ done all at once.”98
+
+ [...]
+
+ Nevertheless, the company made a canny decision not to disclose the
+ true extent of Cortana’s knowledge to its users. It wants to know
+ everything about you, but it does not want you to know how much it
+ knows or that its operations are entirely geared to continuously
+ learning more. Instead, the “bot” is programmed to ask for permission
+ and confirmation. The idea is to avoid spooking the public by
+ presenting Cortana’s intelligence as “progressive” rather than
+ “autonomous,” according to the project’s group program manager, who
+ noted that people do not want to be surprised by how much their phones
+ are starting to take over: “We made an explicit decision to be a little
+ less ‘magical’ and little
+
+ [...]
+
+ The Siren Song of Surveillance Revenues
+
+ [...]
+
+ PrecisionID
+
+ [...]
+
+ ID is then broadcast to every “unencrypted website a Verizon customer
+ visits from a mobile device. It allows third-party advertisers and
+ websites to assemble a deep, permanent profile of visitors’ web
+ browsing habits without their consent.”126 Alarmed by the threat of
+ fresh competition, Google, posing as a privacy advocate, launched a
+ campaign for a new internet protocol that would prevent “header
+ injections” such as Verizon’s PrecisionID.127 Privacy expert and
+ journalist Julia Angwin and
+
+ [...]
+
+ UIDH [unique identifier header], and expect that to be available soon.”
+ The New
+
+ [...]
+
+ capitalism gene therapy. As Verizon’s president of Operations told
+ investors, “For
+
+ [...]
+
+ The companies understood, and they persuaded Republican senators, that
+ the principle of consent would strike a serious blow to the
+ foundational mechanisms of the new capitalism: the legitimacy of
+ unilateral surplus dispossession, ownership rights to surplus, decision
+ rights over surplus, and the right to lawless space for the prosecution
+ of these activities.146 To this end the resolution also prevented
+
+ [...]
+
+ another trend, surveillance in the interest of behavioral surplus
+ capture and sale has become a service in its own right. Such companies
+ are often referred to as “software-as-a-service” or SaaS, but they are
+ more accurately termed “surveillance as a service,” or “SVaaS.” For
+ example, a new app-based approach to lending instantly establishes
+ creditworthiness based on detailed
+
+ [...]
+
+ You’re able to get in and really understand the daily life of these
+ customers,” explained the CEO of one lending company that analyzes
+ 10,000 signals per customer.151 Such methods were originally
+
+ [...]
+
+ Surveillance capitalism was born digital, but as we shall see in
+ following chapters, it is no longer confined to born-digital companies.
+ This logic for translating investment into revenue is highly adaptive
+ and exceptionally lucrative as long as raw-material supplies are free
+ and law is kept at bay. The rapid migration to surveillance revenues
+ that is now underway recalls the late-twentieth-century shift from
+ revenues derived from goods and services to revenues derived from
+ mastering the speculative and shareholder-value-maximizing
+
+ [...]
+
+ Who knows? Who decides? Who decides who decides
+
+ [...]
+
+ According to the philosopher of language John Searle, a declaration is
+ a particular way of speaking and acting that establishes facts out of
+ thin air, creating a new reality where there was nothing. Here is how
+ it works: sometimes we speak to simply describe the world—“you have
+ brown eyes”—or to change it—“Shut the door.” A declaration combines
+ both, asserting a new reality by describing the world as if a desired
+ change were already true: “All humans are created equal.” “They are
+ yours to command.” As Searle writes, “We
+
+ [...]
+
+ Searle concludes, “All of institutional reality, and therefore… all of
+ human civilization is created by… declarations
+
+ [...]
+
+ Instead, Durkheim trained his sights on the social transformation
+ already gathering around him, observing that “specialization” was
+ gaining “influence” in politics, administration, the judiciary,
+ science, and the arts. He concluded that the division of labor was no
+ longer quarantined in the industrial workplace. Instead, it had burst
+ through those factory walls to becoming the critical organizing
+ principle of industrial society. This is also an example of Edison’s
+ insight: that the principles of capitalism initially aimed at
+ production eventually shape the wider social and moral milieu.
+ “Whatever opinion one has about the division of labor,” Durkheim wrote,
+ “everyone knows that it exists, and is more and more becoming one of
+ the fundamental bases of the social order.”17 Economic imperatives
+ predictably mandated the division of labor in production, but what was
+ the purpose of the division of labor in society? This was the question
+ that motivated Durkheim’s analysis, and his century
+
+ [...]
+
+ What would hold society together in the absence of the rules and
+ rituals of clan and kin? Durkheim’s answer was the division of labor.
+ People’s needs for a coherent new source of meaning and structure were
+ the cause, and the effect was an ordering
+
+ [...]
+
+ conclusions are still relevant for us now. He argued that the division
+ of labor accounts for the interdependencies and reciprocities that link
+ the many diverse members of a modern industrial society in a larger
+ prospect of solidarity. Reciprocities breed mutual need, engagement,
+ and respect, all of which imbue this new ordering principle with moral
+ force
+
+ [...]
+
+ Britain, university administrators are already talking about a “missing
+ generation” of data scientists. The huge salaries of the tech firms
+ have lured so many professionals that there is no one left to teach the
+ next generation of students. As one scholar described it, “The real
+ problem is these people are not dispersed through society. The
+ intellect and expertise is concentrated in a small number of
+ companies.”32
+
+ [...]
+
+ Under the regime of surveillance capitalism, the corporation’s
+ scientists are not recruited to solve world hunger or eliminate
+ carbon-based fuels. Instead, their genius is meant to storm the gates
+ of human experience, transforming it into data and translating it into
+ a new market colossus that creates wealth by predicting, influencing
+
+ [...]
+
+ under scrutiny, those long-awaited delivery trucks look more like
+ automated vehicles of invasion and conquest: more Mad Max than Red
+ Cross, more Black Sails than Carnival Cruise. The wizards behind their
+ steering wheels careen across every hill and hollow, learning how to
+ scrape and stockpile our behavior
+
+ [...]
+
+ Schmidt was, in fact, merely paraphrasing computer scientist Mark
+ Weiser’s seminal 1991 article, “The Computer for the 21st Century,”
+ which has framed Silicon Valley’s technology objectives for nearly
+ three decades. Weiser introduced what he called “ubiquitous computing”
+ with two legendary sentences: “The most profound technologies are those
+ that disappear. They weave themselves into the fabric of everyday life
+ until they are indistinguishable from it.” He described a new way of
+ thinking “that allows the computers themselves to vanish into the
+ background.… Machines that fit the human environment instead of forcing
+ humans to enter theirs will make using a computer as refreshing as
+ taking a walk in the woods.”2
+
+ [...]
+
+ new phase, supply operations were enlarged and intensified to
+ accommodate economies of scope and economies of action. What does this
+ entail? The shift toward economies of scope defines a new set of aims:
+ behavioral surplus must be vast, but it must also be varied. These
+ variations are developed along two dimensions. The first is the
+ extension of extraction operations from the virtual world into the
+ “real
+
+ [...]
+
+ call economies of action. In order to achieve these economies, machine
+ processes are configured to intervene in the state of play in the real
+ world among real people and things. These interventions are designed to
+ enhance certainty by doing things: they nudge, tune, herd, manipulate,
+ and modify behavior in specific directions by executing actions as
+ subtle as inserting a specific phrase into your Facebook news feed,
+ timing the appearance of a BUY button on your phone, or shutting down
+ your car engine when an insurance payment is late.
+
+ [...]
+
+ means of behavioral modification.” The aim of this undertaking is not
+ to impose behavioral norms, such as conformity or obedience, but rather
+ to produce behavior that reliably, definitively, and certainly leads to
+ desired commercial results. The research director of Gartner, the
+ well-respected business advisory and research firm, makes the point
+ unambiguously when he observes
+
+ [...]
+
+ shadow text.4As the prediction imperative gathers force, it gradually
+ becomes clear that extraction was the first phase of a
+ far-more-ambitious project. Economies of action mean that real-world
+ machine architectures must be able to know as well as to do. Extraction
+ is not enough; now it must be twinned with execution. The extraction
+ architecture is combined with a new execution architecture, through
+ which hidden economic objectives are imposed upon the vast and varied
+ field of behavior.5 Gradually, as surveillance capitalism’s imperatives
+
+ [...]
+
+ is an extraordinary statement because there can be no such guarantees
+ in the absence of the power to make it so. This wider complex that we
+ refer to as the “means of behavioral modification” is the expression of
+ this gathering power. The prospect of guaranteed outcomes alerts us to
+ the force of the prediction imperative, which demands that surveillance
+ capitalists make the future for the sake of predicting it. Under this
+ regime, ubiquitous computing is not just a knowing machine; it is an
+ actuating machine designed to produce more certainty about us and for
+ them.
+
+ [...]
+
+ Finally, I want to underscore that although it may be possible to
+ imagine something like the “internet of things” without surveillance
+ capitalism, it is impossible to imagine surveillance capitalism without
+ something like the “internet of things.” Every command arising from the
+ prediction imperative requires this pervasive real-world material
+ “knowing and doing” presence. The new apparatus is the material
+ expression of the prediction imperative, and it represents a new kind
+ of power animated by the economic compulsion toward certainty. Two
+ vectors converge in this fact: the early ideals of ubiquitous computing
+ and the economic imperatives of surveillance capitalism. This
+ convergence signals the metamorphosis of the digital infrastructure
+ from a thing that we have to a thing that has us.
+
+ [...]
+
+ was coaxed to life nearly sixty years ago under the warm equatorial sun
+ of the Galapagos Islands, when a giant tortoise stirred from her torpor
+ to swallow a succulent chunk of cactus into which a dedicated scientist
+ had wedged a small machine. It was a time when scientists reckoned with
+ the obstinacy of free-roaming animals and concluded that surveillance
+ was the necessary price of knowledge. Locking these creatures in a zoo
+ would only eliminate the very behavior that scientists wanted to study,
+ but how were they to be surveilled? The solutions once concocted by
+ scholars of elk herds, sea turtles, and geese have been refurbished by
+ surveillance capitalists and presented as an inevitable feature of
+ twenty-first-century life on Earth. All that has changed is that now we
+ are the animals
+
+ [...]
+
+ If you’re not in the system, you don’t exist
+
+ [...]
+
+ dark data
+
+ [...]
+
+ This means that surplus must be both plentiful (economies of scale) and
+ varied (economies of scope) in both range and depth.
+
+ [...]
+
+ gamification
+
+ [...]
+
+ all else fails, insurers are advised to induce a sense of inevitability
+ and helplessness in their customers. Deloitte counsels companies to
+ emphasize “the multitude of other technologies already in play to
+ monitor driving” and that “enhanced surveillance and/or geo-location
+ capabilities are part of the world we live in now, for better or
+ worse.”46
+
+ [...]
+
+ Behavioral data drawn from their experience are processed, and the
+ results flow in two directions. First, they return to the drivers,
+ executing procedures to interrupt and shape behavior in order to
+ enhance the certainty, and therefore profitability, of predictions
+ (economies of action). Second, prediction products that rank and sort
+ driver behavior flow into newly convened behavioral futures markets in
+ which third parties lay bets on what drivers will do now, soon, and
+ later: Will
+
+ [...]
+
+ uncontract
+
+ [...]
+
+ This is not the automation of society, as some might think, but rather
+ the replacement of society with machine action dictated by economic
+ imperatives. The uncontract is not
+
+ [...]
+
+ Despite its pervasiveness both in Silicon Valley and in the wider
+ culture of data scientists and technology developers, inevitabilism is
+ rarely discussed or critically evaluated. Paradiso
+
+ [...]
+
+ Paradiso imagines a society in which it falls to each individual to
+ protect herself from the omniscient ubiquitous sensate computational
+ systems of the new apparatus. Rather than paradise, it seems a recipe
+ for a new breed of madness. Yet this is precisely the world that is now
+ under construction around us, and this madness appears to be a happy
+ feature of the plan.
+
+ [...]
+
+ Cisco Kinetic gets the right data to the right applications at the
+ right time… while executing policies to enforce data ownership,
+ privacy, security and even data sovereignty laws.”73 But, as is so
+ often the case, the most audacious effort to transform the urban
+ commons into the surveillance capitalist’s equivalent of Paradiso’s
+ 250-acre marsh comes from Google, which has introduced and legitimated
+ the concept of the “for-profit city.” Just as MacKay had counseled and
+ Weiser proselytized, the computer would be operational everywhere and
+ detectable nowhere, always beyond the edge of individual awareness. In
+ 2015, shortly after Google reorganized
+
+ [...]
+
+ We fund it all… through a very novel advertising model.… We can
+ actually then target ads to people in proximity, and then obviously
+ over time track them through things like beacons and location services
+ as well as their browsing activity
+
+ [...]
+
+ As we are shorn of alternatives, we are forced to purchase products
+ that we can never own while our payments fund our own surveillance and
+ coercion. Adding insult to injury, data rendered by this wave of things
+
+ [...]
+
+ life pattern marketing” based on techniques derived from military
+ intelligence known as “patterns of life analysis
+
+ [...]
+
+ It allows you to tap into people’s compulsive nature by encouraging
+ impulse buys with the notifications you send out.… It also allows you
+ to gain insight on your current customers by reading what they’re
+ saying on Yelp and Facebook.…”24 Another mobile marketing firm
+ recommends
+
+ [...]
+
+ November 2017 Quartz investigative reporters discovered that since
+ early 2017, Android phones had been collecting location information by
+ triangulating the nearest cell towers, even when location services were
+ disabled, no apps were running, and no carrier SIM card was installed
+ in the phone. The information was used to manage Google’s “push”
+ notifications and messages sent to users on their Android phones,
+ enabling the company to track “whether an individual with
+
+ [...]
+
+ databases of ruin.”34
+
+ [...]
+
+ The company built an “employment index” for the national economy as
+ well as a “consumption index.” It also touted its ability to generate
+ quite-specific predictions such
+
+ [...]
+
+ The agencies’ well-meaning guidelines overlook the inconvenient truth
+ that transparency and privacy represent friction for surveillance
+ capitalists in much the same way that improving working conditions,
+ rejecting child labor, or shortening the working day represented
+ friction for the early industrial capitalists. It took targeted laws to
+ change working conditions back
+
+ [...]
+
+ The only real protection is when an app randomly but regularly
+ generates a new MAC address for your phone, but of the nine trackers,
+ only Apple’s performed this operation. The report also identifies a
+ general pattern of careless
+
+ [...]
+
+ other words, privacy policies are more aptly referred to as
+ surveillance policies, and that is what I suggest we call them. There
+ are many new territories of body rendition
+
+ [...]
+
+ Nobody reckoned with the fact that the prediction imperative makes
+ individual ignorance the preferred condition for rendition operations,
+ just as Arendt had observed and Mackay had prescribed for animals in
+ the wild. Original sin prefers the dark. The talks continued without
+ the advocates, and
+
+ [...]
+
+ 1767 the political economist Nathaniel Forster worried that
+ “fashionable luxury” was spreading “like a contagion,” and he
+ complained of the “perpetual restless ambition in each of the inferior
+ ranks to raise themselves to the level of those immediately above
+ them.”4 Adam Smith wrote insightfully on this social process, noting
+ that upper-class luxuries can in time be recast as “necessaries.” This
+ occurs as “the established rules of decency” change to reflect new
+ customs introduced by elites, triggering lower-cost production methods
+ that transform what was once unattainable into newly affordable goods
+ and services.5 Ford’s Model T
+
+ [...]
+
+ Conversation” stands alone in its promise to dominate raw-material
+ supply, and the rewards to the One Voice would be astronomical. Casual
+ talk helps to blur the boundaries between “it”—the apparatus saturated
+ with commercial agents—and us. In conversation we imagine friendship.
+ The more we fancy the apparatus as our confidante, nanny, governess,
+ and support system—a disembodied, pervasive “Mrs. Doubtfire” for each
+ person—the more experience we allow it to render, and the richer its
+ supply operations grow. Communication is the first human joy, and a
+ conversational interface is prized for the frictionless ease in which a
+ mere utterance can trigger action
+
+ [...]
+
+ 2018, Amazon had inked deals with home builders, installing its Dot
+ speakers directly into ceilings throughout the house as well as Echo
+ devices and
+
+ [...]
+
+ Mark Zuckerberg’s unilateral upending of established privacy norms in
+ 2010, when he famously announced that Facebook users no longer have an
+ expectation of privacy. Zuckerberg had described the corporation’s
+ decision to unilaterally release users’ personal information,
+ declaring, “We decided that these would be the social norms now, and we
+ just went for it.”55 Despite
+
+ [...]
+
+ The personalization project descends deeper toward the ocean floor with
+ these new tools, where they lay claim to yet a new frontier of
+ rendition trained not only on your personality but also on your
+ emotional life. If this project of surplus from the depths is to
+ succeed, then your unconscious—where feelings form before there are
+ words to express them—must be recast as simply one more source of
+ raw-material supply for machine rendition and analysis, all of it for
+ the sake of more-perfect prediction. As a market research report on
+ affective computing explains, “Knowing the real-time emotional state
+ can help businesses to sell their product and thereby increase
+ revenue.”88 Emotion analytics products such as SEWA use
+
+ [...]
+
+ Conditioning” is a well-known approach to inducing behavior change,
+ primarily associated with the famous Harvard behaviorist B. F. Skinner.
+ He argued that behavior modification should mimic the evolutionary
+ process, in which naturally occurring behaviors are “selected” for
+ success by environmental conditions. Instead of the earlier, more
+ simplistic model of stimulus/response, associated with behaviorists
+ such as Watson and Pavlov, Skinner interpolated a third variable:
+ “reinforcement.” In his laboratory work with mice and pigeons, Skinner
+ learned how to observe a range of naturally occurring behaviors in the
+ experimental animal and then reinforce the specific action, or
+ “operant,” that he wanted the animal to reproduce. Ultimately, he
+ mastered intricate designs or “schedules” of reinforcement that could
+ reliably shape precise behavioral routines. Skinner called the
+ application
+
+ [...]
+
+ Conditioning at scale is essential to the new science of massively
+ engineered human behavior.” He believes that smartphones, wearable
+ devices, and the larger
+
+ [...]
+
+ Varian endorsed and celebrated this self-authorizing experimental role,
+ warning that all the data in the world “can only measure correlation,
+ not causality.”3 Data tell what happened but not why it happened
+
+ [...]
+
+ element in the construction of high-quality prediction products—i.e.,
+ those that approximate guaranteed outcomes—depends upon causal
+ knowledge. As Varian says, “If you really want to understand causality,
+ you have to run experiments. And if you run experiments continuously,
+ you can continuously improve your system.”4
+
+ [...]
+
+ Psychologists have found that the more a person can project himself or
+ herself into the feelings of another and take the other’s perspective,
+ the more likely he or she is to be influenced by subliminal cues,
+ including hypnosis. Empathy orients people toward other people. It
+ allows one to get absorbed in emotional experience and to resonate with
+ others’ experiences
+
+ [...]
+
+ Facebook’s persistence warns us again of the dispossession cycle’s
+ stubborn march. Facebook had publicly acknowledged and apologized for
+ its overt experimental incursions into behavior modification and
+ emotional manipulation, and it promised adaptations to curb or mitigate
+ these practices. Meanwhile, a new threshold of intimate life had been
+ breached. Facebook’s potential mastery of emotional manipulation became
+ discussable and even taken for granted as habituation set in. From
+ Princeton’s Fiske to critic Grimmelmann and supporter Meyer, the
+ experts believed that if Facebook’s activities were to be forced into a
+ new regulatory regime, the corporation would merely continue in secret
+
+ [...]
+
+ Individual awareness is the
+
+ [...]
+
+ The evasion of individual and group awareness was critical to
+ Facebook’s behavior-modification success, just as MacKay had
+ stipulated. The first paragraph of the research article on emotional
+ contagion celebrates this evasion: “Emotional states can be transferred
+ to others via emotional contagion, leading people to experience the
+ same emotions without their awareness.” Nor do the young adults of
+ Australia’s great cities suspect that the precise measure of their
+ fears and fantasies is exploited for commercial result at the hour and
+ moment of their greatest vulnerability.
+
+ [...]
+
+ enemy of telestimulation because it is the necessary condition for the
+ mobilization of cognitive and existential resources. There is no
+ autonomous judgment without awareness. Agreement and disagreement,
+ participation and withdrawal, resistance or collaboration: none of
+ these self-regulating choices can exist without awareness.
+
+ [...]
+
+ Indeed, some theorists have suggested that the primary purpose of self
+ awareness is to enable self-regulation.” Every threat to human autonomy
+ begins with an assault on awareness, “tearing down our capacity to
+ regulate our thoughts, emotions, and desires.”22
+
+ [...]
+
+ salience of self-awareness as a bulwark against self-regulatory failure
+ is also underscored in the work of two Cambridge University researchers
+ who developed a scale to measure a person’s “susceptibility to
+ persuasion.” They found that the single most important determinant of
+ one’s ability to resist persuasion is what they call “the ability to
+ premeditate.”23 This means that people who harness self-awareness to
+ think through the consequences of their actions are more disposed to
+ chart their own course and are significantly less vulnerable to
+ persuasion techniques. Self-awareness also figures in the
+ second-highest-ranking factor on their scale: commitment. People who
+ are consciously committed to a course of action or set of principles
+
+ [...]
+
+ We have seen already that democracy threatens surveillance revenues.
+ Facebook’s practices suggest an equally disturbing conclusion: human
+ consciousness itself is a threat to surveillance revenues, as awareness
+ endangers the larger project of behavior modification. Philosophers
+ recognize “self-regulation,” “self-determination,” and “autonomy” as
+ “freedom of will.” The word autonomy derives from the Greek and
+ literally means “regulation by the self.” It stands in contrast to
+ heteronomy, which means “regulation by others.” The competitive
+ necessity of economies of action means that surveillance capitalists
+ must use all means available to supplant autonomous action with
+ heteronomous action.
+
+ [...]
+
+ However, it would be dangerous to nurse the notion that today’s
+ surveillance capitalists simply represent more of the same. This
+ structural requirement of economies of action turns the means of
+ behavioral modification into an engine of growth. At no other time in
+ history have private corporations of unprecedented wealth and power
+ enjoyed the free exercise of economies of action supported by a
+ pervasive global architecture of ubiquitous computational knowledge and
+ control constructed and maintained by all the advanced scientific
+ know-how that money can buy.
+
+ [...]
+
+ Most research on games concludes that these structures can be effective
+ at motivating action, and researchers generally predict that games will
+ increasingly be used as the methodology of choice to change individual
+ behavior.34 In practice, this has meant that the power of games to
+ change behavior is shamelessly instrumentalized as gamification spreads
+ to thousands of situations in which a company merely wants to tune,
+ herd, and condition the behavior of its customers or employees toward
+ its own objectives
+
+ [...]
+
+ One analyst compiled a survey of more than ninety such “gamification
+ cases,” complete with return-on-investment statistics.35 Ian Bogost, a
+ professor of interactive computing at Georgia Tech and a digital
+ culture observer, insists that these systems should be called
+ “exploitationware” rather than games because their sole aim is behavior
+ manipulation and modification.36
+
+ [...]
+
+ The zeal for Pokémon Go gradually diminished, but the impact of Hanke’s
+ accomplishments is indelible. “We’ve only just scratched the surface,”
+ Hanke told a crowd of fans.46 The game had demonstrated that it was
+ possible to achieve economies of action on a global scale while
+ simultaneously directing specific individual actions toward precise
+ local market opportunities where high bidders enjoy an ever-closer
+ approximation of guaranteed outcomes. Niantic’s distinctive
+ accomplishment
+
+ [...]
+
+ TechCrunch noted the game’s “precise location tracking” and “ability to
+ perform audio fingerprinting” through its access to your camera and
+ microphone, concluding, “So it’s prudent to expect some of your
+ location data to end up in Google’s hands.”48 The Electronic Privacy
+ Information Center noted in a letter of complaint to the Federal
+
+ [...]
+
+ However, it does not acknowledge that its services operate on two
+ levels: game services for players and prediction services for Niantic’s
+ customers. The company concedes that it uses third-party services,
+ including Google’s, to “collect and interpret data,” but it is careful
+ to sidestep the aims of those analyses.51 The seven-page letter
+ mentions “sponsored
+
+ [...]
+
+ The genius of Pokémon Go was to transform the game you see into a
+ higher-order game of surveillance capitalism, a game about a game
+
+ [...]
+
+ the end we recognize that the probe was designed to explore the next
+ frontier: the means of behavioral modification. The game about the game
+ is, in fact, an experimental facsimile of surveillance capitalism’s
+ design for our future
+
+ [...]
+
+ Thus began a morbidly fascinating and often bizarre chapter in the
+ history of American spy craft.55 Much of the new work was conducted in
+ the context of the CIA’s highly classified MKUltra project, which was
+ tasked with “research and development of chemical, biological, and
+ radiological materials capable for employment in clandestine operations
+ to control human behavior
+
+ [...]
+
+ Another factor was the 1971 publication of B. F. Skinner’s incendiary
+ social meditation Beyond Freedom & Dignity. Skinner prescribed a future
+ based on behavioral control, rejecting the very idea of freedom (as
+ well as every tenet of a liberal society) and cast the notion of human
+ dignity as an accident of self-serving narcissism
+
+ [...]
+
+ First Amendment, the subcommittee argued, “must equally protect the
+ individual’s right to generate ideas,” and the right to privacy should
+ protect citizens from intrusions into their thoughts, behavior,
+ personality, and identity lest these concepts “become meaningless.” It
+ was in this context that Skinnerian behavioral engineering was singled
+ out for critical examination
+
+ [...]
+
+ Where is the hammer of democracy now, when the threat comes from your
+ phone, your digital assistant, your Facebook login? Who will stand for
+ freedom now, when Facebook threatens to retreat into the shadows if we
+ dare to be the friction that disrupts economies of action that have
+ been carefully, elaborately, and expensively constructed to exploit our
+ natural empathy, elude our awareness, and circumvent our prospects for
+ self-determination? If we fail to take notice now, how long before we
+ are numb to this incursion and to all the incursions? How long until we
+ notice nothing at all? How long before we forget who we were before
+ they owned us, bent over the old texts of self-determination in the dim
+
+ [...]
+
+ Now we know that surveillance capitalists’ ability to evade our
+ awareness is an essential condition for knowledge production. We are
+ excluded because we are friction that impedes
+
+ [...]
+
+ The commodification of behavior under the conditions of surveillance
+ capitalism pivots us toward a societal future in which an exclusive
+ division of learning is protected by secrecy, indecipherability, and
+ expertise. Even when knowledge derived from your behavior is fed back
+ to you in the first text as a quid pro quo for participation, the
+ parallel secret operations of the shadow text capture surplus for
+ crafting into prediction products destined for other marketplaces that
+ are about you rather than for you. These markets do not depend upon you
+ except first as a source of raw material from which surplus is derived,
+ and then as a target for guaranteed outcomes
+
+ [...]
+
+ this future we are exiles from our own behavior, denied access to or
+ control over knowledge derived from our experience. Knowledge,
+ authority, and power rest with surveillance capital, for which we are
+ merely “human natural resources
+
+ [...]
+
+ Centuries of debate have been levied on the notion of free will, but
+ too often their effect has been to silence our own declarations of
+ will, as if we are embarrassed to assert this most fundamental human
+ fact. I recognize my direct experience of freedom as an inviolate truth
+ that cannot be reduced to the behaviorists’ formulations of life as
+ necessarily accidental and random, shaped by external stimuli beyond my
+ knowledge or
+
+ [...]
+
+ influence and haunted by irrational and untrustworthy mental processes
+ that I can neither discern nor avoid
+
+ [...]
+
+ American philosopher John Searle, whose work on the “declaration” we
+ discussed in Chapter 6, comes to a similar conclusion in his
+ examination of “free will.” He points to the “causal gap” between the
+ reasons for our actions and their enactment. We may have good reasons
+ to do something, he observes, but that does not necessarily mean it
+ will be done. “The traditional name of this gap in philosophy is ‘the
+ freedom of the will.’” In response to the “sordid history” of this
+ concept, he reasons, “even if the gap is an illusion it is one we
+ cannot shake off.… The notion of making and keeping promises
+ presupposes the gap.… [It] requires consciousness and a sense of
+ freedom on the part of the promise-making and promise-keeping agent
+
+ [...]
+
+ Our freedom flourishes only as we steadily will ourselves to close the
+ gap between making promises and keeping them. Implicit in this action
+ is an assertion that through my will I can influence the future. It
+ does not imply total authority over the future, of course, only over my
+ piece
+
+ [...]
+
+ should an experience as elemental as this claim on the future tense be
+ cast as a human right? The short answer is that it is only necessary
+ now because it is imperiled. Searle argues that such elemental
+ “features of human life” rights are crystallized as formal human rights
+ only at that moment in history when they come under systematic threat.
+ So, for example, the ability to speak is elemental. The concept of
+ “freedom of speech” as a formal right emerged only when society evolved
+ to a degree of political complexity that the freedom to speak came
+ under threat. The philosopher observes that speech is not more
+ elemental to human life than breathing or being able to move one’s
+ body. No one has declared a “right to breathe” or a “right to bodily
+ movement” because these elemental rights have not come under attack and
+ therefore do not require formal protection. What counts as a basic
+ right, Searle argues, is both “historically contingent” and “pragmatic
+
+ [...]
+
+ Most simply put, there is no freedom without uncertainty; it is the
+ medium in which human will is expressed in promises. Of course, we do
+ not only make promises to ourselves; we also make promises to one
+ another. When we join our wills and our promises, we create the
+ possibility of collective action toward a shared future, linked in
+ determination to make our vision real in the world. This is the origin
+ of the institution we call “contract,” beginning with the ancient
+ Romans.6 Contracts originated as shared “islands of predictability”
+ intended to mitigate uncertainty for the human community, and they
+ still retain this meaning. “The simplest way
+
+ [...]
+
+ The uncontract aims instead for a condition that the economist Oliver
+ Williamson describes as “contract utopia”: a state of perfect
+ information known to perfectly rational people who always perform
+ exactly as promised.12 The problem is, as Williamson writes, “All
+ complex contracts are unavoidably incomplete
+
+ [...]
+
+ you have ever seen a house built according to architectural plans, then
+ you have a good idea of what Williamson means. There is no blueprint
+ that sufficiently details everything needed to convert drawings and
+ specifications into an actual house. No plan anticipates every problem
+ that might arise, and most do not come close. The builders’ skills are
+ a function of how they collaborate to invent the actions that fulfill
+ the intention of the drawings as they solve the unexpected but
+ inevitable complications that arise along the way. They work together
+ to construct a reality from the uncertainty of the plan
+
+ [...]
+
+ Were “contract utopia” to exist, Williamson says, it would best be
+ described as a “plan” that, like other “utopian modes,” requires “deep
+ commitment to collective purposes” and “personal subordination.”
+ Subordination to what? To the plan. Contract in this context of perfect
+ rationality is what Williamson describes as “a world of planning.” Such
+ planning was the basic institution of socialist economics, where the
+ “new man” was idealized as possessing “a high level of cognitive
+ competence” and therefore, it was espoused, could design highly
+ effective plans.14 Varian deftly swaps out socialism’s “new man” and
+ installs instead a market defined by surveillance capitalism’s economic
+ imperatives, expressed through a ubiquitous computational architecture,
+ the machine intelligence capabilities to which data are continuously
+ supplied, the analytics that discern patterns, and the algorithms that
+ convert them into rules. This is the essence of the uncontract, which
+
+ [...]
+
+ Uncertainty is not
+
+ [...]
+
+ chaos but rather the necessary habitat of the present tense. We choose
+ the fallibility of shared promises and problem solving over the certain
+ tyranny imposed by a dominant power or plan because this is the price
+ we pay for the freedom to will, which founds our right to the future
+ tense. In the absence of this freedom, the future collapses into an
+ infinite present of mere behavior, in which there can be no subjects
+ and no projects: only objects. In the future
+
+ [...]
+
+ Life inclines us to take action and to make commitments even when the
+ future is unknown. Anyone who has brought a child into the world or
+
+ [...]
+
+ the real world of human endeavor, there is no perfect information and
+ no perfect rationality
+
+ [...]
+
+ improve their approximation to guaranteed outcomes. Just as industrial
+ capitalism was driven to the continuous intensification of the means of
+ production, so surveillance capitalists are now locked in a cycle of
+ continuous intensification of the means of behavioral modification
+
+ [...]
+
+ Surveillance capitalists’ interests have shifted from using automated
+ machine processes to know about your behavior to using machine
+ processes to shape your behavior according to their interests. In other
+ words, this decade-and-a-half trajectory has taken us from automating
+ information flows about you to automating you. Given the conditions of
+ increasing ubiquity, it has become difficult if not impossible to
+ escape this audacious, implacable web
+
+ [...]
+
+ In order to reestablish our bearings, I have asked for a rebirth of
+ astonishment and outrage. Most of all, I have asked that we reject the
+ Faustian pact of participation for dispossession that requires our
+ submission to the means of behavioral modification built on the
+ foundation of the Google declarations. I am also mindful, though, that
+ when we ask How did they get away with it? there are many compelling
+ reasons to consider, no one of which stands alone
+
+ [...]
+
+ need laws that reject the fundamental legitimacy of surveillance
+ capitalism’s declarations and interrupt its most basic operations,
+ including the illegitimate rendition of human experience as behavioral
+ data; the use of behavioral surplus as free raw material; extreme
+ concentrations of the new means of production
+
+ [...]
+
+ shock and awe
+
+ [...]
+
+ withdrawal of agreement takes two broad forms, a distinction that will
+ be useful as we move into Part III. The first is what I call the
+ counter-declaration. These are defensive measures such as encryption
+ and other privacy tools, or arguments for “data ownership.” Such
+ measures may be effective in discrete situations
+
+ [...]
+
+ turn to the history of the Berlin Wall as an illustration of these two
+ forms of disagreement
+
+ [...]
+
+ industrial capitalism dangerously disrupted nature, what havoc might
+ surveillance capitalism wreak on human nature? The answer to this
+ question requires a return to imperatives
+
+ [...]
+
+ Industrial capitalism brought us to the brink of epic peril, but not as
+ a consequence of an evil lust for destruction or runaway technology.
+ Rather, this result was ineluctably driven by its own inner logic of
+ accumulation, with its imperatives of profit maximization, competition,
+ the relentless drive for labor productivity through the technological
+ elaboration of production, and growth funded by the continuous
+ reinvestment of
+
+ [...]
+
+ Similarly, the meaning of Polanyi’s prophecy for us now can be grasped
+ only through the lens of surveillance capitalism’s economic imperatives
+ as they frame its claim to human experience. If we are to rediscover
+ our sense of astonishment, then let it be here: if industrial
+ civilization flourished at the expense of nature and now threatens to
+ cost us the Earth, an information civilization shaped by surveillance
+ capitalism will thrive at the expense of human nature and threatens to
+ cost us our humanity
+
+ [...]
+
+ The idea from the start was that naming and taming are inextricable,
+ that fresh and careful naming can better equip us to intercept these
+ mechanisms of dispossession, reverse their action, produce urgently
+ needed friction, challenge the pathological division of learning, and
+ ultimately synthesize new forms of information capitalism that
+ genuinely meet our needs for effective life
+
+ [...]
+
+ the heart of Gentile’s political philosophy is the concept of the
+ “total.”3 The state was to be understood as an inclusive organic unity
+ that transcends individual
+
+ [...]
+
+ secret plans executed by secret police, the silent complicities and
+ hidden atrocities, the ceaseless transformation of who or what was up
+ or down, the intentional torsion of facts into anti-facts accompanied
+ by a perpetual deluge of propaganda, misinformation, euphemism, and
+ mendacity. The authoritative leader, or “egocrat,” to use the French
+ philosopher Claude Lefort’s term, displaces the rule of law and
+ “common” sense to become the quixotic judge of what is just or unjust,
+ truth or lie, at each moment.9
+
+ [...]
+
+ Great Terror
+
+ [...]
+
+ murders of whole sectors of the Soviet population, from poets to
+ diplomats, generals to political loyalists. According to Soviet
+ historian Robert Conquest, that two-year period saw seven million
+ arrests, one million executions, two million deaths in labor camps, one
+ million people imprisoned, and another seven million people still in
+ camps by the end of 1938.11 Despite the immediacy of catastrophic
+
+ [...]
+
+ Until the rise of surveillance capitalism, the prospect of
+ instrumentarian power was relegated to a gauzy world of dream and
+ delusion. This new species of power follows the logic of Planck, Meyer,
+ and Skinner in the forfeit of freedom for knowledge, but those
+ scientists each failed to anticipate the actual terms of this
+ surrender. The knowledge that now displaces our freedom is proprietary.
+ The knowledge is theirs, but the lost freedom belongs solely to us.
+ With this origin story in
+
+ [...]
+
+ Instrumentarian power cultivates an unusual “way of knowing” that
+ combines the “formal indifference” of the neoliberal worldview with the
+ observational perspective of radical behaviorism
+
+ [...]
+
+ Forget the cliché that if it’s free, “You are the product.” You are not
+ the product; you are the abandoned carcass. The “product” derives from
+ the surplus that is ripped from your life.Big Other finally enables the
+ universal technology of behavior that, as Skinner, Stuart MacKay, Mark
+ Weiser, and Joe Paradiso each insisted, accomplishes its aims quietly
+ and persistently, using methods that intentionally bypass our
+ awareness, disappearing into the background of all things. Recall that
+ Alphabet/Google’s Eric Schmidt provoked uproar in 2015 when in response
+ to a question on the future of the web, he said, “The internet will
+ disappear.” What he really meant was that “The internet will disappear
+ into Big Other.”
+
+ [...]
+
+ We may confuse Big Other with the behaviorist god of the vortex, but
+ only because it effectively conceals the machinations of surveillance
+ capital that are the wizard behind the digital curtain
+
+ [...]
+
+ Under the regime of instrumentarian power, the mental agency and
+ self-possession of the right to the future tense are gradually
+ submerged beneath a new kind of automaticity: a lived experience of
+ stimulus-response-reinforcement aggregated as the comings and goings of
+ mere organisms. Our conformity is irrelevant to instrumentarianism’s
+ success. There is no need for mass submission to social norms, no loss
+ of self to the collective induced by terror and compulsion, no offers
+ of acceptance
+
+ [...]
+
+ Take one wrong step, one deviation from the path of seamless
+ frictionless predictability, and that same voice turns acid in an
+ instant as it instructs “the vehicular monitoring system not to allow
+ the car to be started.”
+
+ [...]
+
+ belonging as a reward for bending to the group. All of that is
+ superseded by a digital order that thrives within things and bodies,
+ transforming volition into reinforcement and action into conditioned
+ response. In this way instrumentarian power produces endlessly accruing
+ knowledge for surveillance capitalists and endlessly diminishing
+ freedom for us as it continuously renews surveillance capitalism’s
+ domination of the division of learning in society. False consciousness
+ is no longer produced by the hidden facts of class and their relation
+ to production but rather by the hidden facts of instrumentarian power’s
+ command over the division of learning in society as it usurps the
+ rights to answer the essential questions: Who knows? Who decides? Who
+ decides who decides? Power was once identified with the ownership of
+ the means of production, but it is now identified with ownership of the
+ means of behavioral modification that is Big Other.
+
+ [...]
+
+ The last stage of the laboring society, the society of jobholders,
+ demands of its members a sheer automatic functioning, as though
+ individual life had actually been submerged in the over-all life
+ process of the species and the only active decision still required of
+ the individual were to let go, so to speak, to abandon his
+ individuality, the still individually sensed pain and trouble of
+ living, and acquiesce in a dazed
+
+ [...]
+
+ tranquilized,” functional type of behavior. The trouble with modern
+ theories of behaviorism is not that they are wrong but that they could
+ become true, that they actually are the best possible conceptualization
+ of certain obvious trends in modern society. It is quite conceivable
+ that the modern age—which began with such an unprecedented and
+ promising outburst of human activity—may end in the deadliest, most
+ sterile passivity history has ever known.5 Is this to be our home
+
+ [...]
+
+ Now imagine, decades hence, another thinker meditating on the
+ “disturbing relevance” of instrumentarian power, observing that “the
+ true problems of our time cannot be understood, let alone solved,
+ without acknowledgement that instrumentarianism became this century’s
+ curse only because it so terrifyingly took care of its problems.” What
+ problems? I have
+
+ [...]
+
+ In the age of surveillance capitalism it is instrumentarian power that
+ fills the void, substituting machines for social relations, which
+ amounts to the substitution of certainty for society. In this imagined
+ collective life, freedom is forfeit to others’ knowledge, an
+ achievement that is only possible with the resources of the shadow text
+
+ [...]
+
+ private institutions of capital led the way in this ambitious
+ reformation of collective life and individual experience, but they
+ found necessary support from public institutions, especially as the
+ declaration of a “war on terror” legitimated every inclination to
+ enshrine machine-produced certainty as the ultimate solution to
+ societal uncertainty. These mutual affinities assured that
+ instrumentarian power would not be a stepchild but rather an equal
+ partner or even, with increasing regularity, the lord and master upon
+ whom the state depends in its quest for “total awareness.” That
+ instrumentarian power is regarded as the certain
+
+ [...]
+
+ White House briefing memo encouraged the companies to develop a
+ “radicalism algorithm” that would digest social media and other sources
+ of surplus to produce something comparable to a credit score, but aimed
+ at evaluating the “radicalness” of online content.14 The turn to
+ instrumentarian
+
+ [...]
+
+ Global Internet Forum to Counter Terrorism. The objective was to
+ tighten the net of instrumentarian power through
+
+ [...]
+
+ One startup, Geofeedia, specializes in detailed location tracking of
+ activists and protesters, such as Greenpeace members or union
+ organizers, and the computation of individualized
+
+ [...]
+
+ ACLU attorney countered that the government is using tech companies “to
+ build massive dossiers on people” based on nothing more than their
+ constitutionally protected speech.26 Another, more prominent
+ surveillance-as-a-service company, Palantir, once touted by Bloomberg
+ Businessweek as “the war on terror’s secret weapon,” was found to be in
+ a secret collaboration with the New Orleans Police Department to test
+ its “predictive policing” technology. Palantir’s software not only
+ identified gang members but also “traced people’s ties to other gang
+ members, outlined
+
+ [...]
+
+ thrust their scores into an inexorable downward spiral: “First your
+ score drops. Then your friends hear you are on the blacklist and,
+ fearful that their scores might be affected, quietly drop you as a
+ contact. The algorithm notices, and your
+
+ [...]
+
+ places less value on privacy than does Western culture and that most
+ Chinese have accommodated to the certain knowledge of online government
+ surveillance and censorship. The most common word for privacy, yinsi,
+ didn’t even appear in popular Chinese dictionaries until the
+ mid-1990s.42 Chinese citizens have accepted national ID cards with
+ biometric chips, “birth permits,” and now social credit rankings
+ because their society has been saturated with surveillance and
+ profiling for decades. For example, the “dang’an” is a wide-ranging
+ personal dossier compiled on hundreds of millions of urban residents
+ from childhood and maintained throughout life. This “Mao-era system for
+ recording the most intimate details of life” is updated by teachers,
+ Communist Party officials, and employers. Citizens have no rights to
+ see its contents, let alone contest them. The dossier is only one
+ feature of long-institutionalized
+
+ [...]
+
+ government urges the tech companies to train their algorithms for a
+ “radicalism” score. Indeed, the work of the shadow text is to evaluate,
+ categorize, and predict our behavior in millions of ways that we can
+ neither know nor combat—these are our digital dossiers. When it comes
+ to credit scoring, US and UK banks and
+
+ [...]
+
+ the Chinese context, the state will run the show and own it, not as a
+ market project but as a political one, a machine solution that shapes a
+ new society of automated behavior for guaranteed political and social
+ outcomes: certainty without terror. All the pipes from all the supply
+ chains will carry behavioral surplus to this new, complex means of
+ behavioral modification. The state will assume the role of the
+ behaviorist god, owning the shadow text and determining the schedule of
+ reinforcements and the behavioral routines that it will shape. Freedom
+ will be forfeit to knowledge, but it will be the state’s knowledge that
+ it exercises, not for the sake of revenue but for the sake of its own
+ perpetuation.
+
+ [...]
+
+ Joe” Stalin
+
+ [...]
+
+ The road from Shenzhen to an American or European airport also leads to
+ the Roomba vacuum cleaner mapping your living room and your breakfast
+ with Alexa
+
+ [...]
+
+ one direction lies the possibility of a synthetic declaration for a
+ third modernity based on the strengthening of democratic institutions
+ and the creative construction of a double movement for our time. On
+ this road we harness the digital to forms of information capitalism
+ that reunite supply and demand in ways that are both genuinely
+ productive of effective life and compatible with a flourishing
+ democratic social order. The first step down this road begins with
+ naming, establishing our bearings, reawakening our astonishment, and
+ sharing a sense of righteous indignity.
+
+ [...]
+
+ They aim to fashion a new society that emulates machine learning in
+ much the same way that industrial society was patterned on the
+ disciplines and methods of factory production. In their vision,
+ instrumentarian power replaces social trust, Big Other substitutes
+ certainty for social relations, and society as we know it shades into
+ obsolescence
+
+ [...]
+
+ Citing Abraham Lincoln, Facebook’s founder located his company’s
+ mission in the evolutionary time line of civilization, during which
+ humanity organized itself first in tribes, then cities, then nations.
+ The next phase of social evolution would be “global community,” and
+ Facebook was to lead the way, constructing the means and overseeing the
+ ends.14 Speaking at Facebook’s 2017 developers’ conference, Zuckerberg
+ linked his assertion of the company’s historic role in establishing a
+ “global community” to the standard myth of the modern utopia, assuring
+ his followers, “In the future, technology is going to… free us up to
+ spend more time on the things we all care about, like enjoying and
+ interacting with each other and expressing ourselves in new ways.… A
+ lot more of us are gonna do what today is considered the arts, and
+ that’s gonna form the basis of a lot of our communities
+
+ [...]
+
+ The “societal goal” articulated by the leading surveillance capitalists
+ fits snugly into the notion of limitless technological progress that
+ dominated utopian thought from the late eighteenth century through the
+ late nineteenth century, culminating with Marx. Indeed, surveillance
+ capitalists such as Nadella, Page, and Zuckerberg conform to five of
+ the six elements with which the great scholars of utopian thought,
+ Frank and Fritzie Manuel, define the classic profile of the most
+ ambitious modern utopianists: (1) a tendency toward highly focused
+ tunnel vision that simplifies the utopian challenge, (2) an earlier and
+ more trenchant grasp of a “new state of being” than other
+ contemporaries, (3) the obsessive pursuit and defense of an idée fixe,
+ (4) an unshakable belief in the inevitability of one’s ideas coming to
+ fruition, and (5) the drive for total reformation at the level of the
+ species and the entire world system
+
+ [...]
+
+ Often a utopian foresees the later evolution and consequences of
+ technological development already present in an embryonic state; he may
+ have antennae sensitive to the future. His gadgets, however, rarely go
+ beyond the mechanical potentialities of his age. Try as he may to
+ invent something wholly new, he cannot make a world out of nothing.”18
+ In our time, however, surveillance capitalists can and do make such a
+ world—a genuinely historic deviation from the norm. Individually and
+ collectively, the
+
+ [...]
+
+ The only way to grasp the theory advanced in their applied utopistics
+ is to reverse engineer their operations and scrutinize their meaning,
+ as we have done throughout these chapters.
+
+ [...]
+
+ Microsoft’s instrumentarian society, the factories and workplaces are
+ like Skinner’s labs, and the machines replace his pigeons and rats.
+ These are the settings where the architecture and velocities of
+ instrumentarian power are readied for translation to society in a
+ digital-age iteration of Walden Two in which machine relations are the
+ model for social relations. Nadella’s construction site exemplifies the
+ grand confluence in which machines and humans are united as objects in
+ the cloud, all instrumented and orchestrated in accordance with the
+ “policies.” The magnificence of “policies” lies precisely in the fact
+
+ [...]
+
+ result is that “policies” are functionally equivalent to plans, as Big
+ Other directs human and machine action. It ensures that doors will be
+ locked or unlocked, car engines will shut down or come to life, the
+ jackhammer will scream “no” in suicidal self-sacrifice, the worker will
+ adhere to norms, the group will swarm to defeat anomalies. We will all
+ be safe as each organism hums in harmony with every other organism,
+ less a society than a population that ebbs and flows in perfect
+ frictionless confluence, shaped by the means of behavioral modification
+ that elude our awareness and thus can neither be mourned nor resisted.
+
+ [...]
+
+ the twentieth century the critical success factors of industrial
+ capitalism—efficiency, productivity, standardization,
+ interchangeability, the minute division of labor, discipline,
+ attention, scheduling, conformity, hierarchical administration, the
+ separation of knowing and doing, and so forth—were discovered and
+ crafted in the workplace and then transposed to society, where they
+ were institutionalized in schools, hospitals, family life, and
+ personality. As generations of scholars have documented, society became
+ more factory-like so that we might train and socialize the youngest
+ among us to fit the new requirements of a mass production order.
+
+ [...]
+
+ With conspicuously thin theory complemented by thick practice, the
+ patented device is designed to monitor user behavior in order to
+ preemptively detect “any deviation from normal or acceptable behavior
+ that is likely to affect the
+
+ [...]
+
+ Alternatively, the behavior could be assessed in relation to a “feature
+ distribution representing normal and/or acceptable behavior for an
+ average member of a population
+
+ [...]
+
+ user’s mental state
+
+ [...]
+
+ the circle widens as the patent specifications unfold. The scientists
+ note the utility of alerts for health care providers, insurance
+ companies, and law-enforcement personnel. Here is a new
+ surveillance-as-a-service opportunity geared to preempt whatever
+ behavior clients choose. Microsoft’s patent returns us to Planck,
+
+ [...]
+
+ In each case, corporate objectives define the “policies” toward which
+ confluent behavior harmoniously streams.
+
+ [...]
+
+ The machine hive—the confluent mind created by machine learning—is the
+ material means to the final elimination of the chaotic elements that
+ interfere with guaranteed outcomes
+
+ [...]
+
+ Instead of the typical assurances that machines can be designed to be
+ more like human beings and therefore less threatening, Schmidt and
+ Thrun argue just the opposite: it is necessary for people to become
+ more machine-like.
+
+ [...]
+
+ In this world the “correct” outcomes are known in advance and
+ guaranteed in action. The same ubiquitous instrumentation and
+ transparency that define the machine system must also define the social
+ system, which in the end is simply another way of describing the ground
+ truth of instrumentarian society.
+
+ [...]
+
+ this human hive, individual freedom is forfeit to collective knowledge
+ and action. Nonharmonious elements are preemptively targeted with high
+ doses of tuning, herding, and conditioning, including the full
+ seductive force of social persuasion and influence. We march in
+ certainty, like the smart machines. We learn to sacrifice our freedom
+ to collective knowledge imposed by others and for the sake of their
+ guaranteed outcomes. This is the signature of the third modernity
+ offered up by surveillance capital as its answer to our quest for
+ effective life together
+
+ [...]
+
+ Pentland is often referred to as the “godfather of wearables,”
+ especially Google Glass. In 1998 he predicted that wearables “can
+ extend one’s senses, improve memory, aid the wearer’s social life and
+ even help him or her stay calm and collected
+
+ [...]
+
+ Most noteworthy is that Pentland “completes” Skinner, fulfilling his
+ social vision with big data, ubiquitous digital instrumentation,
+ advanced mathematics, sweeping theory, numerous esteemed coauthors,
+ institutional legitimacy, lavish funding, and corporate friends in high
+ places without having attracted the worldwide backlash, moral
+ revulsion, and naked vitriol once heaped on Harvard’s outspoken
+ behaviorist. This fact alone suggests the depth of psychic numbing to
+ which we have succumbed and the loss of our collective bearings.
+
+ [...]
+
+ ’s like watching beavers from outer space, like Jane Goodall watching
+ gorillas. You observe from a distance.”7 (This is a slur on Goodall, of
+ course, whose seminal genius was her ability to understand the gorillas
+ she studied not as “other ones” but rather as “one of us.”)
+
+ [...]
+
+ The team saw that it would be possible to exploit the increasingly
+ “ubiquitous infrastructure” of mobile phones and combine those data
+ with new streams of information from their wearable behavioral
+ monitors. The result was a radical new solution that Pentland and Eagle
+ called “reality mining
+
+ [...]
+
+ Pentland argued that information gathered by his
+ sociometers—“unobtrusive wearable sensors” measuring communication,
+ voice tones, and body language—“could help managers understand who is
+ working with whom and infer the relationships between colleagues” and
+ “would be an efficient way to find people who might work well
+ together.”20
+
+ [...]
+
+ people analytics
+
+ [...]
+
+ Pentland appeared in 2016 at a conference organized by Singularity
+ University, a Silicon Valley hub of instrumentarian ideology funded in
+ part by Larry Page. An interviewer tasked to write about Pentland
+ explains, “Though people are one of the most valuable assets in an
+ organization, many companies are still approaching management with a
+ 20th century mentality.… Pentland saw the factor that was always
+ messing things up was—the people.”29 Like Nadella, Pentland described
+ his aims as developing the social systems that would work along the
+ same lines as the machine systems, using behavioral data flows to judge
+ the “correctness” of action patterns and to intervene when it is
+ necessary to change “bad” action to “correct” action. “If people aren’t
+ interacting correctly and information isn’t spreading correctly,”
+ Pentland warns, “people
+
+ [...]
+
+ Pentland articulated his ambitions for the capabilities and objectives
+ of this new milieu in a series of papers, published primarily between
+ 2011 and 2014, but one remarkable 2011 essay of which he is the sole
+ author stands out: “Society’s Nervous System: Building Effective
+ Government, Energy, and Public Health Systems.”31
+
+ [...]
+
+ The initial premise is reasonable enough: industrial-age technology
+ once revolutionized the world with reliable systems for water, food,
+ waste, energy, transportation, police, health care, education, and so
+ forth, but these systems are now hopelessly “old,” “centralized,”
+ “obsolete,” and “unsustainable.” New digital systems are required that
+ must be “integrated,” “holistic,” “responsive,” “dynamic,” and
+ “self-regulating”: “We need a radical rethinking of societies’ systems.
+ We must create a nervous system for humanity that maintains the
+ stability of our societies’ systems
+
+ [...]
+
+ What is missing… are the dynamic models of demand and reaction,” along
+ with an architecture that guarantees “safety, stability, and
+ efficiency.… The models required must describe human
+
+ [...]
+
+ Regarding incentives, Pentland outlines a principle of “social
+ efficiency,” which means that participation must provide value to the
+ individual but also to the system as a whole.37 For the sake of this
+ wholeness, it is believed, each of us will surrender to a totally
+ measured life of instrumentarian order
+
+ [...]
+
+ Skinner advocated, via Frazier, that the virtue of a “planned society”
+ is “to keep intelligence on the right track, for the good of society
+ rather than of the intelligent individual.… It does this by making sure
+ that the individual will not forget his
+
+ [...]
+
+ Pentland says that “continuous streams of data about human behavior”
+ mean that everything from traffic, to energy use, to disease, to street
+ crime will be accurately forecast, enabling a “world without war or
+ financial crashes, in which infectious disease is quickly detected and
+ stopped, in which energy, water, and other resources are no longer
+ wasted, and in which governments are part of the solution rather than
+ part of the problem.”48 This new “collective intelligence” operates to
+ serve the greater good as we learn to act “in a coordinated manner”
+ based on “social universals.” “Great leaps in health care,
+ transportation
+
+ [...]
+
+ The main barriers are privacy concerns and the fact that we don’t yet
+ have any consensus around the trade-offs between personal and social
+ values.” Like Skinner, he is emphatic that these attachments to a
+ bygone era of imperfect knowledge threaten to undermine the prospect of
+ a perfectly engineered future society: “We cannot ignore the public
+ goods that such a nervous system could provide.…”49 Pentland avoids the
+ question “Whose greater good?” How is the greater good determined when
+ surveillance capitalism owns the machines and the means of behavioral
+ modification? “Goodness” arrives already oriented toward the interests
+ of the owners of the means of behavioral modification and the clients
+ whose guaranteed outcomes they seek to achieve. The greater good is
+ someone’s, but it may not be ours
+
+ [...]
+
+ Capitalism and socialism are equally tainted by their shared emphasis
+ on economic growth, which breeds overconsumption and pollution. Skinner
+ is intrigued by the Chinese system but rejects it on the grounds of the
+ bloody revolution that any effort to convert Westerners would entail.
+ “Fortunately,” Skinner concludes in the preface to Walden Two, “there
+ is another possibility.” This option is Skinner’s version of a
+ behaviorist society that provides a way in which “political action is
+ to be avoided.” In Walden Two a “plan” replaces politics, overseen by a
+ “noncompetitive” group of “Planners” who eschew power in favor of the
+ dispassionate administration of the schedules of reinforcement aimed at
+ the greater good.52 Planners exercise unique control over society but
+ “only because that control is necessary for the proper functioning of
+ the community
+
+ [...]
+
+ Pentland worries that our political-economic constructs such as
+ “market” and “class” hail from an old, slow world of the eighteenth and
+ nineteenth centuries. The new, “light-speed hyperconnected world”
+ leaves no time for the kind of rational deliberation and face-to-face
+ negotiation and compromise that characterized the social milieu in
+ which such political concepts originated
+
+ [...]
+
+ There is no room for politics in this instrumentarian society because
+ politics means establishing and asserting our bearings. Individual
+ moral and political bearings are a source of friction that wastes
+ precious time and diverts behavior from confluence
+
+ [...]
+
+ Computation thus replaces the political life of the community as the
+ basis for governance. The depth and breadth of instrumentation make it
+ possible, Pentland says, to calculate idea flow, social network
+ structure, the degree of social influence between people, and even
+ “individual susceptibilities to new ideas.” Most important,
+ instrumentation makes it possible for those with the God view to modify
+ others’ behavior. The data provide a “reliable prediction of how
+ changing any of these variables will change the performance of all the
+ people
+
+ [...]
+
+ Frazier acknowledges that you cannot coerce people into doing the right
+ thing. The solution is far more subtle and sophisticated, based upon
+ scientifically calibrated schedules of reinforcement: “Instead you have
+ to set up certain behavioral processes which will lead the individual
+ to design his own ‘good’ conduct.… We call that sort of thing
+ ‘self-control.’ But don’t be misled, the control always rests in the
+ last analysis in the hands of society
+
+ [...]
+
+ Pentland’s idea is comparable: “The social physics approach to getting
+ everyone to cooperate” is “social network incentives,” his version of
+ “reinforcement.” With such incentives, he explains, “we focus on
+ changing the connections between people rather than focusing on getting
+ people individually to change their behavior.… We can leverage those
+ exchanges to generate social pressure for change.”60 Social media is
+ critical to establishing these tuning capabilities, Pentland believes,
+ because this is the environment in which social pressure can best be
+ controlled, directed, manipulated, and scaled
+
+ [...]
+
+ Pentland ignores the role of empathy in emulation because empathy is a
+ felt experience that is not subject to the observable metrics required
+ for computational governance. Instead, Pentland subscribes to the label
+ Homo imitans to convey that it is mimicry, not empathy, and certainly
+ not politics, which defines human existence
+
+ [...]
+
+ stream of ideas as a swarm or collective intelligence, flowing through
+ time, with all the humans in it learning from each other’s experiences
+ in order to jointly discover the patterns of preferences and habits of
+ action that best
+
+ [...]
+
+ What is being abolished is autonomous man—the inner man, the
+ homunculus, the possessing demon, the man defended by the literatures
+ of freedom and dignity
+
+ [...]
+
+ One important study of Bitcoin, the cryptocurrency that relies on
+ blockchain, suggests that such machine solutions both express and
+ contribute to the general erosion of the social fabric in ways that are
+ both consistent with instrumentarianism and further pave the way for
+ its success. Information scholars Primavera De Filippi and Benjamin
+ Loveluck conclude that contrary to popular belief, “Bitcoin is neither
+ anonymous nor privacy-friendly
+
+ [...]
+
+ We can begin by asking our children. Without knowing it, we sent the
+ least formed and most vulnerable among us to scout the hive and settle
+ its wilderness. Now their messages are filtering in from the frontier
+
+ [...]
+
+ Indeed, Facebook’s early advantage in this work arose in no small
+ measure from the simple fact that its founders and original designers
+ were themselves adolescents and emerging adults. They designed
+ practices for an imagined universe of adolescent users and college
+
+ [...]
+
+ contrary to Pentland’s belief that “class” divisions would disappear,
+ life in the hive produces new cleavages and forms of stratification:
+ not only tune or be tuned but also pressure or be pressured
+
+ [...]
+
+ Schüll learned that addictive players seek neither entertainment nor
+ the mythical jackpot of cash. Instead, they chase what Harvard Medical
+ School addiction researcher Howard Shaffer calls “the capacity of the
+ drug or gamble to shift subjective experience,” pursuing an
+ experiential state that Schüll calls the “machine zone,” a state of
+ self-forgetting in which one is carried along by an irresistible
+ momentum that feels like one is “played by the machine.”12 The machine
+ zone achieves a sense of complete immersion
+
+ [...]
+
+ Addiction by Design
+
+ [...]
+
+ Shaffer, the addiction researcher, has identified five elements that
+ characterize this state of compulsion: frequency of use, duration of
+ action, potency, route of administration, and player attributes
+
+ [...]
+
+ Perhaps the most difficult quality to capture is that in this period
+ that precedes the hard bargaining, an “inner” sense of “self” simply
+ does not yet exist. It is a time when “I” am whatever the “others”
+ think of me, and how “I” feel is a function of how the “others” treat
+ me. Instead of a stable sense of identity, there is only a chameleon
+ that reinvents itself depending upon the social mirror into which it is
+ drawn. In this condition, the “others” are not individuals but the
+ audience for whom I perform. Who “I” am depends upon the audience. This
+ state
+
+ [...]
+
+ Research shows that these big leaps in self-construction are stimulated
+ by experiences such as structured reflection, conflict, dissonance,
+ crisis, and failure. The people who help trigger this new inward
+ connection refuse to act as our mirrors. They reject fusion in favor of
+ genuine reciprocity
+
+ [...]
+
+ What are the consequences of the failure to win a healthy balance
+ between inner and outer, self and relationship? Clinical studies
+ identify specific patterns associated with this developmental
+ stagnation. Not surprisingly, these include an inability to tolerate
+ solitude, the feeling of being merged with others, an unstable sense of
+ self, and even an excessive need to control others as a way of keeping
+ the mirror close. Loss of the mirror is the felt equivalent of
+ extinction
+
+ [...]
+
+ The cultivation of inner resources is thus critical to the capacity for
+ intimacy and relationship, challenges that have become more
+ time-consuming with each new phase of the modern era. And while young
+ people are bound as ever to the enduring existential task of
+ self-making, our story suggests three critical ways in which this task
+ now converges with history and the unique conditions of existence in
+ our time
+
+ [...]
+
+ For example, Evil by Design author Chris Nodder, a user-experience
+ consultant, explains that evil design aims to exploit human weakness by
+ creating interfaces that “make users emotionally involved in doing
+ something that benefits the designer more than them.” He coaches his
+ readers in psychic numbing, urging them to accept the fact that such
+ practices have become the standard suggesting that consumers and
+ designers find ways to “turn them to your advantage
+
+ [...]
+
+ Facebook’s precocious mastery of “social proof”: “Much of our behavior
+ is determined by our impressions of what is the correct thing to do…
+ based on what we observe others doing.… This influence is known as
+ social proof
+
+ [...]
+
+ Most critical is that the more the need for the “others” is fed, the
+ less able one is to engage the work of self-construction. So
+ devastating is the failure to attain that positive equilibrium between
+ inner and outer life that Lapsley and Woodbury say it is “at the heart”
+ of most adult personality disorders
+
+ [...]
+
+ more that a user “liked,” the more that she informed Facebook about the
+ precise shape and composition of her “hand,” thus allowing the company
+ to continuously tighten the glove and increase the predictive value of
+ her signals.
+
+ [...]
+
+ On the demand side, Facebook’s “likes” were quickly coveted and craved,
+ morphing into a universal reward system or what one young app designer
+ called “our generation’s crack cocaine.” “Likes” became those variably
+ timed dopamine shots, driving users to double down on their bets “every
+ time they
+
+ [...]
+
+ News Feed is also the fulcrum of the social mirror. In the years
+ between revulsion and reverence, News Feed became Facebook’s most
+ intensely scrutinized object of data science and the subject of
+ extensive organizational innovation, all of it undertaken at a level of
+ sophistication and capital intensity that one might more naturally
+ associate with the drive to solve world hunger, cure cancer, or avert
+ climate destruction
+
+ [...]
+
+ laugh, cry, smile, click, like, share, or comment.”40 The glove
+ tightens around the hand with closed feedback loops enabled by the God
+ view, which favors posts from people with whom you have already
+ interacted, posts that have drawn high levels of engagement from
+ others, and posts that are like the ones with which you have already
+ engaged
+
+ [...]
+
+ According to the 302 most significant quantitative research studies on
+ the relationships between social media use and mental health (most of
+ them produced since 2013), the psychological process that most defines
+ the Facebook experience is what psychologists call “social
+ comparison.”45 It is usually considered a natural and virtually
+
+ [...]
+
+ One study found an increase in criminal larceny as television diffused
+ across society, awakening an awareness of and desire for consumer
+ goods. A related issue was that increased exposure to television
+ programs depicting affluence led to “the overestimation of others’
+ wealth and more dissatisfaction with one’s own life
+
+ [...]
+
+ Both television and social media deprive us of real-life encounters, in
+ which we sense the other’s inwardness and share something of our own,
+ thus establishing some threads of communality. Unlike
+
+ [...]
+
+ consequence of the new density of social comparison triggers and their
+ negative feedback loops is a psychological condition known as FOMO
+ (“fear of missing out”). It
+
+ [...]
+
+ Profile inflation triggers more negative self-evaluation among
+ individuals as people compare themselves to others, which then leads to
+ more profile inflation, especially among larger networks that include
+ more “distant friends.” As one study concluded, “Expanding one’s social
+ network by adding a number of distant friends through Facebook may be
+ detrimental by stimulating negative emotions for users
+
+ [...]
+
+ This compulsive behavior is intended to produce relief in the form of
+ social reassurance, but it predictably breeds more anxiety and more
+ searching.52 Social comparison
+
+ [...]
+
+ When considered from the vantage point of the self-other balance,
+ positive social comparisons are just as pernicious as negative
+ comparisons. Both are substitutes for the “hard bargain” of carving out
+ a self that is capable of reciprocity rather than fusion
+
+ [...]
+
+ Facebook use does not promote well-being.… Individual social media
+ users might do well to curtail their use of social media and focus
+ instead on real-world relationships
+
+ [...]
+
+ This is the world of Pentland’s “social learning,” his theory of
+ “tuning” little more than the systematic manipulation of the rewards
+ and punishments of inclusion and exclusion. It succeeds through the
+ natural human inclination to avoid psychological pain
+
+ [...]
+
+ confluence,” in which harmonies are achieved at the expense of the
+ psychological integrity of participants
+
+ [...]
+
+ This synthetic hive is a devilish pact for a young person. In terms of
+ sheer everyday effectiveness—contact, logistics, transactions,
+ communications—turn away, and you are lost. And if you simply crave the
+ fusion juice that is proof of life at a certain age and stage—turn
+ away, and you are extinguished
+
+ [...]
+
+ Just as Pentland stipulated, these closed loops are imposed outside the
+ realm of politics and individual volition. They move in stealth,
+ crafting their effects at the level of automatic psychological
+ responses and tipping the self-other balance toward the
+ pseudo-harmonies of the hive mind. In this process, the inwardness that
+ is the necessary source of autonomous action and moral judgment suffers
+ and suffocates. These are the preparatory steps toward the death of
+ individuality that Pentland advocates. In fact, this
+
+ [...]
+
+ the eighteenth century’s political ideal of the individual as the
+ repository of inalienable dignity, rights, and obligations; (2) the
+ early twentieth century’s individualized
+
+ [...]
+
+ human being called into existence by history, embarking on Machado’s
+ road because she must, destined to create “a life of one’s own” in a
+ world of ever-intensifying social complexity and receding traditions;
+ and (3) the late twentieth century’s psychologically autonomous
+ individual whose inner resources and capacity for moral judgment rise
+ to the challenges of self-authorship that history demands and act as a
+ bulwark against the predations of power. The self-authorship toward
+ which young people strive
+
+ [...]
+
+ post-political societal processes that bind the hive rely on social
+ comparison and social pressure for their durability and predictive
+ certainty, eliminating the need for trust
+
+ [...]
+
+ the closing lines of Jean-Paul Sartre’s existential drama No Exit, the
+ character Garcin arrives at his famous realization, “Hell is other
+ people.” This was not intended as a statement of misanthropy but rather
+ a recognition that the self-other balance can never be adequately
+ struck as long as the “others” are constantly “watching.” Another
+ mid-century social psychologist, Erving Goffman, took up these themes
+ in his immortal The Presentation of Self in Everyday Life. Goffman
+ developed the idea of the “backstage” as the region in which the self
+ retreats from the performative demands of social life.
+
+ [...]
+
+ work as in life, “control of the backstage” allows individuals “to
+ buffer themselves from the deterministic demands that surround them.”
+ Backstage, the language is one of reciprocity, familiarity, intimacy,
+ humor. It offers the seclusion in which one can surrender to the
+ “uncomposed” face in sleep, defecation, sex, “whistling, chewing,
+ nibbling, belching, and flatulence.” Perhaps most of all, it is an
+ opportunity for “regression,” in which we don’t have to be “nice”: “The
+ surest sign of backstage solidarity is to feel that it is safe to lapse
+ into an asociable mood of sullen, silent irritability.” In the absence
+ of such respite where a “real” self can incubate and grow, Sartre’s
+ idea of hell begins to make sense.62
+
+ [...]
+
+ Milgram identified three key themes in the subway experiment as he and
+ his students debriefed their experiences. The first was a new sense of
+ gravitas toward “the enormous inhibitory anxiety that ordinarily
+ prevents us from breaching social norms.” Second was that the reactions
+ of the “breacher” are not an expression of individual personality but
+ rather are “a compelled playing out of the logic of social relations
+
+ [...]
+
+ Embarrassment and the fear of violating apparently trivial norms often
+ lock us into intolerable predicaments.… These are not minor regulatory
+ forces in social life, but basic ones.” Finally, Milgram understood
+ that any confrontation of social norms crucially depends upon the
+ ability to escape. It was not an adolescent who boarded the subway that
+ day. Milgram was an erudite adult and an expert on human behavior,
+ especially the mechanisms entailed in obedience to authority, social
+ influence, and conformity. The subway was just an ordinary slice of
+ life, not a capital-intensive architecture of surveillance and behavior
+ modification, not a “personalized reward device.” Still, Milgram could
+ not fight off the anxiety of the situation. The only thing that made it
+ tolerable was the possibility of an exit. Unlike Milgram, we face an
+ intolerable situation
+
+ [...]
+
+ are meant to fuse with the system and play to extinction: not the
+ extinction of our funds but rather the extinction of our selves.
+ Extinction is a design feature formalized in the conditions of no exit.
+ The aim of the tuners is to contain us within “the power of immediate
+ circumstances” as we are compelled by the “logic of social relations”
+ in the hive to bow to social pressure exerted in calculated patterns
+ that exploit our natural empathy. Continuously tightening feedback
+ loops cut off the means of exit, creating impossible levels of anxiety
+ that further drive the loops toward confluence. What is to be killed
+ here is the inner impulse toward autonomy and the arduous, exciting
+ elaboration of the autonomous self as a source of moral judgment and
+ authority capable of asking for a subway seat or standing against rogue
+ power.
+
+ [...]
+
+ To exit means to enter the place where a self can be birthed and
+ nurtured. History has a name for that kind of place: sanctuary
+
+ [...]
+
+ We know that nothing guarantees safety and certainty in this world, but
+ we are comforted by the serenity of this home and its layered silences.
+ The days unfurl now
+
+ [...]
+
+ In the march of institutional interests intent on implementing Big
+ Other, the very first citadel to fall is the most ancient: the
+ principle of sanctuary. The sanctuary privilege has stood as an
+ antidote to power since the beginning of the human story. Even in
+ ancient societies where tyranny prevailed, the right of sanctuary stood
+ as a fail-safe. There was an exit from totalizing power, and that exit
+ was the entrance to a sanctuary in the form of a city, a community, or
+ a temple.4 By the time of the Greeks, sanctuaries were sacred sites
+ built across the ancient Greek world and consecrated to the purposes of
+ asylum and religious sacrifice. The Greek word asylon means
+ “unplunderable” and founds the notion
+
+ [...]
+
+ sanctuary as an inviolable space.5 The right of asylum survived into
+ the eighteenth century in many parts of Europe, attached to holy sites,
+ churches, and monasteries. The demise of the sanctuary privilege was
+ not a repudiation but rather a reflection of social evolution and the
+ firm establishment of the rule of law. One historian summarized this
+ transformation: “justice as sanctuary.”6 In the modern
+
+ [...]
+
+ empirical study makes the point. In “Psychological Functions of
+ Privacy,” Darhl Pedersen defines privacy as a “boundary control
+ process” that invokes the decision rights associated with “restricting
+ and seeking interaction
+
+ [...]
+
+ The same themes appear from the perspective of psychology. Those who
+ would eviscerate sanctuary are keen to take the offensive, putting us
+ off guard with the guilt-inducing question “What have you got to hide?”
+ But as we have seen, the crucial developmental challenges of the
+ self-other balance cannot be negotiated adequately without the sanctity
+ of “disconnected” time and space for the ripening of inward awareness
+ and the possibility of reflexivity: reflection on and by oneself. The
+ real psychological truth is this: If you’ve got nothing to hide, you
+ are nothing.
+
+ [...]
+
+ six categories of privacy behaviors: solitude, isolation
+
+ [...]
+
+ proper realm of inaccessibility or secrecy with respect to the world at
+ large as well as a recognition of the important social dimension of
+ such protected inner space.…”7
+
+ [...]
+
+ contemplation, autonomy, rejuvenation, confiding, freedom, creativity,
+ recovery, catharsis, and concealment
+
+ [...]
+
+ anonymity, reserve, intimacy with friends, and intimacy with family
+
+ [...]
+
+ billions of sensors filled with personal data fall outside of Fourth
+ Amendment protections, a large-scale surveillance network will exist
+ without constitutional limits
+
+ [...]
+
+ This theme is illustrated in the odyssey of Belgian mathematician and
+ data protection activist Paul-Olivier Dehaye, who in December 2016
+ initiated a request for his personal data collected through Facebook’s
+ Custom Audiences and tracking Pixel tools, which would reveal the web
+ pages where Facebook had tracked him. Dehaye probably knew more about
+ the rogue data operations of Cambridge Analytica than anyone in the
+ world, outside of its own staff and masterminds
+
+ [...]
+
+ of the right to contest “automatic decision making.” If the algorithms
+ are to be contestable in any meaningful way, it will require new
+ countervailing authority and power, including machine resources and
+ expertise to reach into the core disciplines of machine intelligence
+ and construct new approaches that are available for inspection, debate,
+ and combat. Indeed, one expert has already proposed the creation of a
+ government agency—an “FDA for algorithms
+
+ [...]
+
+ is already possible to see a new awakening to empowering collective
+ action, at least in the privacy domain. One example is None of Your
+ Business (NOYB), a nonprofit organization led by privacy activist Max
+ Schrems. After many years of legal contest, Schrems made history in
+ 2015 when his challenge to Facebook’s data-collection and
+ data-retention practices—which he asserted were in violation of EU
+ privacy law—led the Court of Justice of the European Union to
+ invalidate the Safe Harbor agreement that governed data transfers
+
+ [...]
+
+ the absence of synthetic declarations that secure the road to a human
+ future, the intolerability of glass life turns us toward a societal
+ arms race of counter-declarations in which we search for and embrace
+ increasingly complex ways to hide in our own lives, seeking respite
+ from lawless machines and their masters. We do this to satisfy our
+ enduring need for sanctuary and as an act of resistance with which to
+ reject the instrumentarian disciplines of the hive, its “extended
+ chilling effects,” and Big Other’s relentless greed. In the context of
+ government surveillance, the practices of “hiding” have been called
+ “privacy protests” and are well-known for drawing the suspicion of
+ law-enforcement agencies.33 Now, hiding is also invoked by Big Other
+ and its market masters, whose reach is far and deep as they install
+ themselves in our walls, our bodies, and on our streets, claiming our
+ faces, our feelings, and our fears of exclusion. I have suggested
+
+ [...]
+
+ Equally more poignant is the way in which a new generation of
+ activists, artists, and inventors feels itself called to create the art
+ and science of hiding.34 The intolerable conditions of glass life
+ compel these
+
+ [...]
+
+ Chicago artist Leo Selvaggio produces 3-D–printed resin prosthetic
+ masks to confound facial recognition. He calls his effort “an organized
+ artistic intervention.”35 Perhaps most poignant is the Backslash Tool
+ Kit: “a series of functional devices designed for protests and riots of
+ the future,
+
+ [...]
+
+ New Museum for Contemporary Art in Manhattan, and you pass a display of
+ its bestseller: table-top mirrors whose reflecting surface is covered
+ with the bright-orange message “Today’s Selfie Is Tomorrow’s Biometric
+ Profile.” This “Think Privacy Selfie Mirror” is a project of the young
+ Berlin-based artist Adam Harvey, whose work is aimed at the problem of
+ surveillance and
+
+ [...]
+
+ Trevor Paglen’s richly orchestrated performance art combines music,
+ photography, satellite imagery, and artificial intelligence to reveal
+ Big Other’s omnipresent knowing and doing. “It’s trying to look inside
+ the software that is running an AI… to look into the architectures of
+ different computer vision
+
+ [...]
+
+ greatest danger is that we come to feel at home in glass life or in the
+ prospect of hiding from it. Both alternatives rob us of the
+ life-sustaining inwardness
+
+ [...]
+
+ Glass life is intolerable, but so is fitting our faces with masks and
+ draping our bodies in digitally resistant fabrics to thwart the
+ ubiquitous lawless machines. Like every counter-declaration, hiding
+ risks becomes an adaptation when it should be a rallying point for
+ outrage. These conditions are unacceptable. Tunnels under this wall are
+ not enough. This wall must come down.
+
+ [...]
+
+ Surveillance capitalists are no different from other capitalists in
+ demanding freedom from any sort of constraint. They insist upon the
+ “freedom to” launch every novel practice while aggressively asserting
+ the necessity of their “freedom from” law and regulation. This classic
+ pattern reflects two bedrock assumptions about capitalism made by its
+ own theorists: The first is that markets are intrinsically unknowable.
+ The second is that the ignorance produced by this lack of knowledge
+ requires wide-ranging freedom of action for market actors
+
+ [...]
+
+ Adam Smith’s famous metaphor of the “invisible hand” drew on these
+ enduring realities of human life. Each individual, Smith reasoned,
+ employs his capital locally in pursuit of immediate comforts and
+ necessities. Each one attends to “his own security… his own gain… led
+ by an invisible hand to promote an end which was no part of his
+ intention.” That end is the efficient employ of capital in the broader
+ market: the wealth of nations. The individual actions that produce
+ efficient markets add up to a staggeringly complex pattern, a mystery
+ that no one person or entity could hope to know or understand, let
+ alone to direct
+
+ [...]
+
+ Adam Smith,” Hayek wrote, “was the first to perceive that we have
+ stumbled upon methods of ordering human economic cooperation that
+ exceed the limits of our knowledge and perception. His ‘invisible hand’
+ had perhaps better have been described as an invisible or unsurveyable
+ pattern
+
+ [...]
+
+ As with Planck, Meyer, and Skinner, both Hayek and Smith unequivocally
+ link freedom and ignorance. In Hayek’s framing, the mystery of the
+ market is that a great many people can behave effectively while
+ remaining ignorant of the whole. Individuals not only can choose
+ freely, but they must freely choose their own pursuits because there is
+ no alternative, no source of total knowledge or conscious control to
+ guide them. “Human design” is impossible, Hayek says, because the
+ relevant information flows are “beyond the span of the control of any
+ one mind
+
+ [...]
+
+ However, Big Other and the steady application of instrumentarian power
+ challenge the classic quid pro quo of freedom for ignorance. When it
+ comes to surveillance capitalist operations, the “market” is no longer
+ invisible
+
+ [...]
+
+ Hayek chose the market over democracy, arguing that the market system
+ enabled not only the division of labor but also “the coordinated
+ utilization of resources based on equally divided knowledge.” This
+ system, he argued, is the only one compatible with freedom. Perhaps
+ some other kind of civilization might have been devised, he reckoned,
+ “like the ‘state’ of the termite ants,” but it would not be compatible
+ with human freedom.4
+
+ [...]
+
+ More astonishing still is that surveillance capital derives from the
+ dispossession of human experience, operationalized in its unilateral
+ and pervasive programs of rendition: our lives are scraped and sold to
+ fund their freedom and our subjugation, their knowledge and our
+ ignorance about what they know
+
+ [...]
+
+ One conclusion of our investigations is that surveillance capitalism’s
+ command and control of the division of learning in society are the
+ signature feature that breaks with the old justifications of the
+ invisible hand and its entitlements. The combination of knowledge and
+ freedom works to accelerate the asymmetry of power between surveillance
+ capitalists and the societies in which they operate. This cycle will be
+ broken only when we acknowledge as citizens, as societies, and indeed
+ as a civilization that surveillance capitalists know too much to
+ qualify for freedom
+
+ [...]
+
+ The surveillance capitalists that operate at hyperscale or outsource to
+ hyperscale operations dramatically diminish any reliance on their
+ societies as sources of employees, and the few for whom they do
+ compete, as we have seen, are drawn from the most-rarefied strata of
+ data science
+
+ [...]
+
+ The absence of organic reciprocities with people as either sources of
+ consumers or employees is a matter of exceptional importance in light
+ of the historical relationship between market capitalism and democracy
+
+ [...]
+
+ ’s dependency on the “masses” and their contribution to the prosperity
+ necessitated by the new organization of production.23 The rise of
+ volume production and its wage-earning labor force established British
+ workers’ economic power and led to a growing appreciation of their
+ political legitimacy and power. This produced a new sense of
+ interdependence between ordinary people and elites. Acemoglu and
+ Robinson conclude that the “dynamic positive feedback” between
+ “inclusive economic institutions” (i.e., industrial firms defined by
+ employment reciprocities) and political institutions was critical to
+ Britain’s substantial and nonviolent democratic reforms. Inclusive
+ economic institutions, they argue, “level the playing field,”
+ especially when it comes to the fight for power, making it more
+ difficult for elites to “crush the masses” rather than accede to their
+ demands. Reciprocities in employment produced and sustained
+ reciprocities in politics
+
+ [...]
+
+ sharp contrast to the pragmatic concessions of Britain’s early
+ industrial capitalists, surveillance capitalists’ extreme structural
+ independence from people breeds exclusion rather than inclusion and
+ lays the foundation for the unique approach that we have called
+ “radical indifference
+
+ [...]
+
+ significant result of the systematic application of radical
+ indifference is that the public-facing “first text” is vulnerable to
+ corruption with content that would normally be perceived as repugnant:
+ lies, systematic disinformation, fraud, violence, hate speech, and so
+ on. As long as content contributes to “growth tactics,” Facebook
+ “wins.” This vulnerability can be an explosive problem on
+
+ [...]
+
+ guiding principles of radical indifference are reflected in the
+ operations of Facebook’s hidden low-wage labor force charged with
+ limiting the perversion of the first text. Nowhere is surveillance
+ capitalism’s outsized influence over the division of learning in
+ society more concretely displayed than in this outcast function of
+ “content moderation,” and nowhere is the nexus of economic imperatives
+ and
+
+ [...]
+
+ The larger point of the exercise is to find the point of equilibrium
+ between the ability to pull users and their surplus into
+
+ [...]
+
+ site and the risk of repelling them. This is a calculation of radical
+ indifference that has nothing to do with assessing the truthfulness of
+ content or respecting reciprocities with users.36 This tension helps to
+ explain why disinformation is not a priority. One investigative report
+ quotes a Facebook insider: “They absolutely have the tools to shut down
+ fake news.
+
+ [...]
+
+ radical indifference is a permanent invitation to the corruption of the
+ first text
+
+ [...]
+
+ It is obvious that the rogue forces of disinformation grasp this fact
+ more crisply than do Facebook’s or Google’s genuine users and customers
+ as those forces learn to exploit the blind eye of radical indifference
+ and escalate the perversion of learning in an open society
+
+ [...]
+
+ Surveillance capitalism’s antidemocratic and antiegalitarian juggernaut
+ is best described as a market-driven coup from above. It is not a coup
+ d’état in the classic sense but rather a coup de gens: an overthrow of
+ the people concealed as the technological Trojan horse that is Big
+ Other. On the strength of its annexation of human experience, this coup
+ achieves exclusive concentrations of knowledge and power that sustain
+ privileged influence over the division of learning in society: the
+ privatization of the central principle of social ordering in the
+ twenty-first century. Like the adelantados and their silent
+ incantations of the Requirimiento, surveillance capitalism operates in
+ the declarative form and imposes the social relations of a premodern
+ absolutist authority. It is a form of tyranny that feeds on people but
+ is not of the people. In a surreal paradox, this coup is celebrated as
+ “personalization,” although it defiles, ignores, overrides, and
+ displaces everything about you and me that is personal
+
+ [...]
+
+ Tyranny” is not a word that I choose lightly. Like the instrumentarian
+ hive, tyranny is the obliteration of politics. It is founded on its own
+ strain of radical indifference in which every person, except the
+ tyrant, is understood as an organism among organisms in an equivalency
+ of Other-Ones. Hannah Arendt observed that tyranny is a perversion of
+ egalitarianism because it treats all others as equally insignificant:
+ “The tyrant rules in accordance with his own will and interest… the
+ ruler who rules one against all, and the ‘all’ he oppresses are all
+ equal, namely equally powerless.” Arendt notes that classical political
+ theory regarded the tyrant as “out of mankind altogether… a wolf in
+ human shape.…”55Surveillance capitalism rules by instrumentarian power
+ through its materialization in Big Other, which, like the ancient
+ tyrant, exists out of mankind while paradoxically assuming human shape
+
+ [...]
+
+ Polanyi’s lens, we see that surveillance capitalism annexes human
+ experience to the market dynamic so that it is reborn as behavior: the
+ fourth “fictional commodity.” Polanyi’s first three fictional
+ commodities—land, labor, and money—were subjected to law. Although
+ these laws have been imperfect, the institutions of labor law,
+ environmental law, and banking law are regulatory frameworks intended
+ to defend society (and nature, life, and exchange) from the worst
+ excesses of raw capitalism’s destructive power. Surveillance
+ capitalism’s expropriation of human experience has faced no such
+ impediments.
+
+ [...]
+
+ will be Facebook, he says, that will address problems that are
+ civilizational in scale and scope, building “the long-term
+ infrastructure to bring humanity together” and keeping people safe with
+ “artificial intelligence” that quickly understands “what is happening
+ across our community.”56 Like Pentland, Zuckerberg imagines machine
+ intelligence that can “identify risks that nobody would have flagged at
+ all, including terrorists planning attacks using private channels,
+ people bullying someone too afraid to report it themselves, and other
+ issues both local and global.”57 When asked about his responsibility to
+ shareholders, Zuckerberg told CNN, “That’s why it helps to have control
+ of the company
+
+ [...]
+
+ industrial civilization aimed to exert control over nature for the sake
+ of human betterment. Machines were our means of extending and
+ overcoming the limits of the animal body so that we could accomplish
+ this aim of domination. Only later did we begin to fathom the
+ consequences
+
+ [...]
+
+ Years later, in his moving 1966 essay “Education after Auschwitz,”
+ social theorist Theodor Adorno attributed the success of German fascism
+ to the ways in which the quest for effective life had become an
+ overwhelming burden for too many people: “One must accept that fascism
+ and the terror it caused are connected with the fact that the old
+ established authorities… decayed and were toppled, while the people
+ psychologically were not yet ready for self-determination. They proved
+ to be unequal to the freedom that fell into their laps.”67
+
+ [...]
+
+ We can now see that surveillance capitalism takes an even more
+ expansive turn toward domination than its neoliberal source code would
+ predict, claiming its right to freedom and knowledge, while setting its
+ sights on a collectivist vision that claims the totality of society.
+ Though still sounding like Hayek, and even Smith, its antidemocratic
+ collectivist ambitions reveal it as an insatiable child devouring its
+ aging fathers
+
+ [...]
+
+ The critical role of public opinion explains why even the most
+ destructive “ages” do not last forever. I echo here what Edison said a
+ century ago: that capitalism is “all wrong, out of gear.” The
+ instability of Edison’s day threatened every promise of industrial
+ civilization. It had to give way, he insisted, to a new synthesis that
+ reunited capitalism and its populations. Edison was prophetic.
+ Capitalism has survived the longue durée less because of any specific
+ capability and more because of its plasticity. It survives and thrives
+ by periodically renewing its roots in the social, finding new ways to
+ generate new wealth by meeting new needs. Its evolution has been marked
+ by a convergence of basic principles—private property, the profit
+ motive, and
+
+ [...]
+
+ It is not OK for every move, emotion, utterance, and desire to be
+ catalogued, manipulated, and then used to surreptitiously herd us
+ through the future tense for the sake of someone else’s profit. “These
+ things are brand-new,” I tell them. “They are unprecedented. You should
+ not take them for granted because they are not OK
+
+ [...]
+
+ Burnham’s cowardice is a cautionary tale. We are living in a moment
+ when surveillance capitalism and its instrumentarian power appear to be
+ invincible. Orwell’s courage demands that we refuse to cede the future
+ to illegitimate power. He asks us to break the spell of enthrallment,
+ helplessness, resignation, and numbing. We answer his call when we bend
+ ourselves toward friction, rejecting the smooth flows of coercive
+ confluence. Orwell’s courage sets us against the relentless tides of
+ dispossession that demean all human experience. Friction, courage, and
+ bearings are the resources we require to begin the shared work of
+ synthetic declarations that claim the digital future as a human place,
+ demand that digital capitalism operate as an inclusive force bound to
+ the people it must serve, and defend the division of learning in
+ society as a source of genuine democratic renewal
projects / blog.git / commitdiff