Privacy and surveillance scholars increasingly worry that data collectors can use the information they gather about our behaviors, preferences, interests, incomes, and so on to manipulate us. Yet what it means, exactly, to manipulate someone, and how we might systematically distinguish cases of manipulation from other forms of influence—such as persuasion and coercion—has not been thoroughly enough explored in light of the unprecedented capacities that information technologies and digital media enable. In this paper, we develop a definition of manipulation that (...) addresses these enhanced capacities, investigate how information technologies facilitate manipulative practices, and describe the harms—to individuals and to social institutions—that flow from such practices. -/- We use the term “online manipulation” to highlight the particular class of manipulative practices enabled by a broad range of information technologies. We argue that at its core, manipulation is hidden influence—the covert subversion of another person’s decision-making power. We argue that information technology, for a number of reasons, makes engaging in manipulative practices significantly easier, and it makes the effects of such practices potentially more deeply debilitating. And we argue that by subverting another person’s decision-making power, manipulation undermines his or her autonomy. Given that respect for individual autonomy is a bedrock principle of liberal democracy, the threat of online manipulation is a cause for grave concern. (shrink)

In this classic work, Herbert Marcuse takes as his starting point Freud's statement that civilization is based on the permanent subjugation of the human instincts, his reconstruction of the prehistory of mankind - to an interpretation of the basic trends of western civilization, stressing the philosophical and sociological implications.

One of the most noticeable trends in recent years has been the increasing reliance of public decision-making processes on algorithms, i.e. computer-programmed step-by-step instructions for taking a given set of inputs and producing an output. The question raised by this article is whether the rise of such algorithmic governance creates problems for the moral or political legitimacy of our public decision-making processes. Ignoring common concerns with data protection and privacy, it is argued that algorithmic governance does pose a significant threat (...) to the legitimacy of such processes. Modelling my argument on Estlund’s threat of epistocracy, I call this the ‘threat of algocracy’. The article clarifies the nature of this threat and addresses two possible solutions. It is argued that neither solution is likely to be successful, at least not without risking many other things we value about social decision-making. The result is a somewhat pessimistic conclusion in which we confront the possibility that we are creating decision-making processes that constrain and limit opportunities for human participation. (shrink)

A distinctive feature of modern capitalist societies is the tendency of the market to take over the production, maintenance, and distribution of goods that were previously produced, maintained, and distributed by nonmarket means. Yet, there is a wide range of disagreement regarding the proper extent of the market in providing many goods. Labor has been treated as a commodity since the advent of capitalism, but not without significant and continuing challenges to this arrangement. Other goods whose production for and distribution (...) on the market are currently the subject of dispute include sexual intercourse, human blood, and human body parts such as kidneys. How can we determine which goods are properly subjects of market transactions and which are not? The purpose of this article is to propose a theory of what makes economic goods differ from other kinds of goods, which can help to answer this question. (shrink)

Mobile health applications (‘health apps’) that promise the user to help her with some aspect of her health are very popular: for-profit apps such as MyFitnessPal, Fitbit, or Headspace have tens of millions of users each. For-profit health apps are designed and run as optimization systems. One would expect that these health apps aim to optimize the health of the user, but in reality they aim to optimize user engagement and, in effect, conversion. This is problematic, I argue, because digital (...) health environments that aim to optimize user engagement risk being manipulative. To develop this argument, I first provide a brief analysis of the underlying business models and the resulting designs of the digital environments provided by popular for-profit health apps. In a second step, I present a concept of manipulation that can help analyze digital environments such as health apps. In the last part of the article, I use my concept of manipulation to analyze the manipulative potential of for-profit health apps. Although for-profit health can certainly empower their users, the conditions for empowerment also largely overlap with the conditions for manipulation. As a result, we should be cautious when embracing the empowerment discourse surrounding health apps. An additional aim of this article is to contribute to the rapidly growing literature on digital choice architectures and the ethics of influencing behavior through such choice architectures. I take health apps to be a paradigmatic example of digital choice architectures that give rise to ethical questions, so my analysis of the manipulative potential of health apps can also inform the larger literature on digital choice architectures. (shrink)

Future Politics confronts the most important question of our time: how will digital technology change society?

In this classic work, Herbert Marcuse takes as his starting point Freud's statement that civilization is based on the permanent subjugation of the human instincts, his reconstruction of the prehistory of mankind - to an interpretation of the basic trends of western civilization, stressing the philosophical and sociological implications.

What is the role of education in a world where we no longer have a clear vision of the future and where the idea of a single, universal model of humanity seems like the residue of a bygone age? What role should educators play in a world where young people find themselves faced with deep uncertainty about their future, where the prospects of securing a stable, long-term career seem increasingly remote and where intensified population movements have created more diverse communities (...) in which different cultures find themselves living side by side, no longer bound together by the belief that the other would eventually be assimilated into ‘our' culture? Faced with the bewildering features of our liquid modern world, many young people are inclined to withdraw - in some cases into the online world of games and virtual relationships, in other cases into anorexia, depression, alcohol or even drug abuse, hoping to find shelter from a world perceived as more and more dangerous. Others launch into more violent forms of behaviour, like street gangs and the looting carried out by young people who have been excluded from the temples of consumption but are eager to participate in the ceremony. And all this happens while our politicians look on, uncomprehending and indifferent. In this short book Zygmunt Bauman - the leading social theorist of our liquid modern world, here in conversation with Riccardo Mazzeo - reflects on the predicament of young people today and on the role of education and the educator in a world where the certainties of our predecessors can no longer be taken for granted. (shrink)

One way to read Kierkegaard’s Works of Love is as an all out assault on the Enlightenment ideal of moral autonomy from a religious point of view. Kant is the locus classicus of this ideal, just as Descartes and Locke are, respectively, for the correlative ideals of epistemic and political autonomy. Since these three components belong to the central core of what we have come to think of as the modern understanding of the subject, Kierkegaard’s critique has a distinctively postmodern (...) flavour. But, as we shall see, it is postmodern precisely by the way it is biblically premodern.The first thing Kant says about the relation of religion to morality in his ‘Fourth Critique’ is that “morality does not need religion at all.” Because morality presupposes “man as a free agent who, just because he is free, binds himself through his reason to unconditioned laws, it stands in need neither of the idea of another Being over him, for him to apprehend his duty, nor of an incentive other than the law itself, for him to do his duty.”Kant has a more positive way of relating morality and religion. “Religion is the recognition of all duties as divine commands.” But this moral role of religion is qualified in three important ways. First, it presupposes only faith and not theoretical knowledge of God. Second, as we have already seen, the God relation cannot play any essential role either in determining the content of my duty or in motivating me to do it. The autonomy of the moral agent precludes both possibilities. Third, it involves no “special duties” to God “over and above the ethico-civil duties of humanity ...”Kierkegaard will agree to the first proviso . But his ethics is theonomous, and thus heteronomous, and he will vigorously reject the second and third constraints on the ethics of divine command he develops in Works of Love. (shrink)

The debate between Marcuse and Habermas over technology marked a significant turning point in the history of the Frankfurt School. After the 1960s Habermas's influence grew as Marcuse's declined and Critical Theory adopted a far less Utopian stance. Recently there has been a revival of quite radical technology criticism in the environmental movement and under the influence of Foucault and constructivism. This article takes a new look at the earlier debate from the standpoint of these recent developments. While much of (...) Habermas's argument remains persuasive, his defense of modernity now seems to concede far too much to the claims of autonomous technology. His essentialist picture of technology as an application of a purely instrumental form of nonsocial rationality is less plausible after a decade of historicizing research in technology studies. The article argues that Marcuse was right after all to claim that technology is socially determined even if he was unable to develop his insight fruitfully. The article derives a new approach to technology criticism from both constructivism and Habermas's communication theory. The essence of technology is shown to be historical and reflexive, like the essence of other social institutions. As such an institution, its rationality is always implemented in value?biased forms subject to political critique. (shrink)

Every day, new warnings emerge about artificial intelligence rebelling against us. All the while, a more immediate dilemma flies under the radar. Have forces been unleashed that are thrusting humanity down an ill-advised path, one that's increasingly making us behave like simple machines? In this wide-reaching, interdisciplinary book, Brett Frischmann and Evan Selinger examine what's happening to our lives as society embraces big data, predictive analytics, and smart environments. They explain how the goal of designing programmable worlds goes hand in (...) hand with engineering predictable and programmable people. Detailing new frameworks, provocative case studies, and mind-blowing thought experiments, Frischmann and Selinger reveal hidden connections between fitness trackers, electronic contracts, social media platforms, robotic companions, fake news, autonomous cars, and more. This powerful analysis should be read by anyone interested in understanding exactly how technology threatens the future of our society, and what we can do now to build something better. (shrink)