It is both an ideal and an assumption of traditional conceptions of justice for liberal democracies that citizens are autonomous, self-governing persons. Yet standard accounts of the self and of self-government at work in such theories are hotly disputed and often roundly criticized in most of their guises. John Christman offers a sustained critical analysis of both the idea of the 'self' and of autonomy as these ideas function in political theory, offering interpretations of these ideas which avoid such disputes (...) and withstand such criticisms. Christman's model of individual autonomy takes into account the socially constructed nature of persons and their complex cultural and social identities, and he shows how this model can provide a foundation for principles of justice for complex democracies marked by radical difference among citizens. His book will interest a wide range of readers in philosophy, politics, and the social sciences. (shrink)

Interactions between an intelligent software agent and a human user are ubiquitous in everyday situations such as access to information, entertainment, and purchases. In such interactions, the ISA mediates the user’s access to the content, or controls some other aspect of the user experience, and is not designed to be neutral about outcomes of user choices. Like human users, ISAs are driven by goals, make autonomous decisions, and can learn from experience. Using ideas from bounded rationality, we frame these interactions (...) as instances of an ISA whose reward depends on actions performed by the user. Such agents benefit by steering the user’s behaviour towards outcomes that maximise the ISA’s utility, which may or may not be aligned with that of the user. Video games, news recommendation aggregation engines, and fitness trackers can all be instances of this general case. Our analysis facilitates distinguishing various subcases of interaction, as well as second-order effects that might include the possibility for adaptive interfaces to induce behavioural addiction, and/or change in user belief. We present these types of interaction within a conceptual framework, and review current examples of persuasive technologies and the issues that arise from their use. We argue that the nature of the feedback commonly used by learning agents to update their models and subsequent decisions could steer the behaviour of human users away from what benefits them, and in a direction that can undermine autonomy and cause further disparity between actions and goals as exemplified by addictive and compulsive behaviour. We discuss some of the ethical, social and legal implications of this technology and argue that it can sometimes exploit and reinforce weaknesses in human beings. (shrink)

The digital age brings with it novel forms of data flow. As a result, individuals are constantly being monitored while consuming products, services and content. These abilities have given rise to a variety of concerns, which are most often framed using “privacy” and “data protection”-related paradigms. An important, oft-noted yet undertheorized concern is that these dynamics might facilitate the manipulation of subjects; a process in which firms strive to motivate and influence individuals to take specific steps and make particular decisions (...) in a manner considered to be socially unacceptable. That it is important and imperative to battle manipulation carries with it a strong intuitive appeal. Intuition, however, does not always indicate the existence of a sound justification or policy option. For that, a deeper analytic and academic discussion is called for. This Article begins by emphasizing the importance of addressing the manipulation-based argument, which derives from several crucial problems and flaws in the legal and policy setting currently striving to meet the challenges of the digital age. Next, the Article examines whether the manipulation-based concerns are sustainable, or are merely a visceral response to changing technologies which cannot be provided with substantial analytical backing. Here the Article details the reasons for striving to block manipulative conduct and, on the other hand, reasons why legal intervention should be, in the best case, limited. The Article concludes with some general implications of this discussion for the broader themes and future directions of privacy law, while trying to ascertain whether the rise of the manipulation-based discourse will lead to information privacy’s expansion or perhaps its demise. (shrink)

So‐called nudge policies utilize insights from behavioral science to achieve policy outcomes. Nudge policies try to improve people's decisions by changing the ways options are presented to them, rather than changing the options themselves or incentivizing or coercing people. Nudging has been met with great enthusiasm but also fierce criticism. This paper provides an overview of the debate on the ethics of nudging to date. After outlining arguments in favor of nudging, we first discuss different objections that all revolve around (...) the worry that nudging vitiates personal autonomy. We split up this worry into different dimensions of autonomy, such as freedom of choice, volitional autonomy, rational agency, and freedom as nondomination. We next discuss worries that nudging is manipulative, violates human dignity, and prevents more important structural reform. Throughout, we will present responses that proponents of nudging can muster. On the whole, we conclude that the objections fail to establish that the nudge program as a whole should be rejected. At the same time, they give us important guidance when moving towards an ethical assessment of nudges on a case‐by‐case basis. Towards the end, we provide some possible ways forward in debates around the ethics of nudging. (shrink)

Cass Sunstein and Richard Thaler recommend helping people make better decisions by employing ‘nudges’, which they define as noncoercive methods of influencing choice for the better. Not surprisingly, healthcare practitioners and public policy professionals have become interested in whether nudges might be a promising method of improving health-related behaviors without resorting to heavy-handed methods such as coercion, deception, or government regulation. Many nudges seem unobjectionable as they merely improve the quality and quantity available for the decision-maker. However, other nudges influence (...) decision-making in ways that do not involve providing more and better information. Nudges of this sort raise concerns about manipulation. This paper will focus on noninformational nudges that operate by changing the salience of various options. It will survey two approaches to understanding manipulation, one which sees manipulation as a kind of pressure, and one that sees it as a kind of trickery. On the pressure view, salience nudges do not appear to be manipulative. However, on the trickery view, salience nudges will be manipulative if they increase the salience so that it is disproportionate to that fact's true relevance and importance for the decision at hand. By contrast, salience nudges will not be manipulative if they merely highlight some fact that is true and important for the decision at hand. The paper concludes by providing examples of both manipulative and nonmanipulative salience nudges. (shrink)

In recent years, 'nudge units' or 'behavioral insights teams' have been created in the United States, the United Kingdom, Germany, and other nations. All over the world, public officials are using the behavioral sciences to protect the environment, promote employment and economic growth, reduce poverty, and increase national security. In this book, Cass R. Sunstein, the eminent legal scholar and best-selling co-author of Nudge, breaks new ground with a deep yet highly readable investigation into the ethical issues surrounding nudges, choice (...) architecture, and mandates, addressing such issues as welfare, autonomy, self-government, dignity, manipulation, and the constraints and responsibilities of an ethical state. Complementing the ethical discussion, The Ethics of Influence: Government in the Age of Behavioral Science contains a wealth of new data on people's attitudes towards a broad range of nudges, choice architecture, and mandates. (shrink)

For several years, scholars have (for good reason) been largely preoccupied with worries about the use of artificial intelligence and machine learning (AI/ML) tools to make decisions about us. Only recently has significant attention turned to a potentially more alarming problem: the use of AI/ML to influence our decision-making. The contexts in which we make decisions—what behavioral economists call our choice architectures—are increasingly technologically-laden. Which is to say: algorithms increasingly determine, in a wide variety of contexts, both the sets of (...) options we choose from and the way those options are framed. Moreover, artificial intelligence and machine learning (AI/ML) makes it possible for those options and their framings—the choice architectures—to be tailored to the individual chooser. They are constructed based on information collected about our individual preferences, interests, aspirations, and vulnerabilities, with the goal of influencing our decisions. At the same time, because we are habituated to these technologies we pay them little notice. They are, as philosophers of technology put it, transparent to us—effectively invisible. I argue that this invisible layer of technological mediation, which structures and influences our decision-making, renders us deeply susceptible to manipulation. Absent a guarantee that these technologies are not being used to manipulate and exploit, individuals will have little reason to trust them. (shrink)

Privacy and surveillance scholars increasingly worry that data collectors can use the information they gather about our behaviors, preferences, interests, incomes, and so on to manipulate us. Yet what it means, exactly, to manipulate someone, and how we might systematically distinguish cases of manipulation from other forms of influence—such as persuasion and coercion—has not been thoroughly enough explored in light of the unprecedented capacities that information technologies and digital media enable. In this paper, we develop a definition of manipulation that (...) addresses these enhanced capacities, investigate how information technologies facilitate manipulative practices, and describe the harms—to individuals and to social institutions—that flow from such practices. -/- We use the term “online manipulation” to highlight the particular class of manipulative practices enabled by a broad range of information technologies. We argue that at its core, manipulation is hidden influence—the covert subversion of another person’s decision-making power. We argue that information technology, for a number of reasons, makes engaging in manipulative practices significantly easier, and it makes the effects of such practices potentially more deeply debilitating. And we argue that by subverting another person’s decision-making power, manipulation undermines his or her autonomy. Given that respect for individual autonomy is a bedrock principle of liberal democracy, the threat of online manipulation is a cause for grave concern. (shrink)