Generative AI enables automated, effective manipulation at scale. Despite the growing general ethical discussion around generative AI, the specific manipulation risks remain inadequately investigated. This article outlines essential inquiries encompassing conceptual, empirical, and design dimensions of manipulation, pivotal for comprehending and curbing manipulation risks. By highlighting these questions, the article underscores the necessity of an appropriate conceptualisation of manipulation to ensure the responsible development of Generative AI technologies.

The standard way to test alternative descriptive theories of moral judgment is by asking subjects to evaluate (amongst others) sacrificial dilemmas, where acting classifies as a utilitarian moral judgment and not acting classifies as a deontological moral judgment. Previous research uncovered many situational factors that alter subject’s moral judgments without affecting which type of action utilitarianism or deontology would recommend. This literature review provides a systematic analysis of the experimental literature on the influence of situational factors on moral judgments in (...) sacrificial dilemmas. It analyses 53 articles in detail and reports mean effect sizes, as well as operationalizations, for 36 situational factors that significantly influence moral judgment. Moreover, the review discusses how the impact of situational factors relates to a dual process theory of moral judgment. It supports the view that utilitarian judgments are driven by controlled cognitive processes and shows that the drivers of deontological judgments depend on valence. (shrink)

A series of recent papers raises worries about the manipulative potential of algorithmic transparency (to wit, making visible the factors that influence an algorithm’s output). But while the concern is apt and relevant, it is based on a fraught understanding of manipulation. Therefore, this paper draws attention to the ‘indifference view’ of manipulation, which explains better than the ‘vulnerability view’ why algorithmic transparency has manipulative potential. The paper also raises pertinent research questions for future studies of manipulation in the context (...) of algorithmic transparency. (shrink)

This chapter defends the view that manipulated behaviour is explained by an injustice. Injustices that explain manipulated behaviour need not involve agential features such as intentionality. Therefore, technology can manipulate us, even if technological artefacts like robots, intelligent software agents, or other ‘mere tools’ lack agential features such as intentionality. The chapter thus sketches a comprehensive account of manipulated behaviour related to but distinct from existing accounts of manipulative behaviour. It then builds on that account to defend the possibility that (...) we are being manipulated by technology. (shrink)

In this open access book, Timothy Aylsworth and Clinton Castro draw on the deep well of Kantian ethics to argue that we have moral duties, both to ourselves and to others, to protect our autonomy from the threat posed by the problematic use of technology. The problematic use of technologies like smartphones threatens our autonomy in a variety of ways, and critics have only begun to appreciate the vast scope of this problem. In the last decade, we have seen a (...) flurry of books making “self-help” arguments about how we could live happier, more fulfilling lives if we were less addicted to our phones. But none of these authors see this issue as one involving a moral duty to protect our autonomy. (shrink)

This paper proposes a conceptual framework for evaluating how social networking platforms fare as epistemic environments for human users. I begin by proposing a situated concept of epistemic agency as fundamental for evaluating epistemic environments. Next, I show that algorithmic personalisation of information makes social networking platforms problematic for users’ epistemic agency because these platforms do not allow users to adapt their behaviour sufficiently. Using the tracing principle inspired by the ethics of self-driving cars, I operationalise it here and identify (...) three requirements that automated epistemic environments need to fulfil: (a) the users need to be afforded a range of skilled actions; (b) users need to be sensitive to the possibility to use their skills; (c) the habits built when adapting to the platform should not undermine the user’s pre-existing skills. I then argue that these requirements are almost impossible to fulfil all at the same time on current SN platforms; yet nevertheless, we need to pay attention to these whenever we evaluate an epistemic environment with automatic features. Finally, as an illustration, I show how Twitter, a popular social networking platform, will fare regarding these requirements. (shrink)

In this article, we argue that rational persuasion can be a _pro tanto_ wrong and that online platforms possess features that are especially conducive to this wrong. We begin by setting out an account of rational persuasion. This consists of four jointly sufficient conditions for rational persuasion and is intended to capture the core, uncontroversial cases of such persuasion. We then discuss a series of wrong-making features which are present in methods of influence commonly thought of as _pro tanto_ wrong, (...) such as manipulation and paternalism. It is next shown that these wrong-making features are also present in a range of cases that are, by the jointly sufficient conditions already established, rational persuasion, and so some forms of rational persuasion are _pro tanto_ wrong, including in some ways that have not previously been remarked upon. Finally, we demonstrate that online settings possess a number of features that are especially conducive to wrongful rational persuasion. (shrink)

Are we being manipulated online? If so, is being manipulated by online technologies and algorithmic systems notably different from human forms of manipulation? And what is under threat exactly when people are manipulated online? This volume provides philosophical and conceptual depth to debates in digital ethics about online manipulation. The contributions explore the ramifications of our increasingly consequential interactions with online technologies such as online recommender systems, social media, user-friendly design, micro-targeting, default-settings, gamification, and real-time profiling. The authors in this (...) volume address four broad and interconnected themes: • What is the conceptual nature of online manipulation? And how, methodologically, should the concept be defined? • Does online manipulation threaten autonomy, freedom, and meaning in life, and if so, how? • What are the epistemic, affective, and political harms and risks associated with online manipulation? • What are legal and regulatory perspectives on online manipulation? The Philosophy of Online Manipulation brings these various considerations together to offer philosophically robust answers to critical questions concerning our online interactions with one another and with autonomous systems. It will be of interest to researchers and advanced students working in moral philosophy, digital ethics, philosophy of technology, and the ethics of manipulation. (shrink)

Nudging works through dispositions to decide with specific heuristics, and has three component parts. A nudge is a feature of an environment that enables such a disposition; a person is nudged when such a disposition is triggered; and a person performs a nudged action when such a disposition manifests in action. This analysis clarifies an autonomy-based worry about nudging as used in public policy or for private profit: that a person’s ability to reason well is undermined when she is nudged. (...) Reasoning well is a component of self-guidance, which is an aspect of autonomy and so something there is reason to promote, preserve, or respect. However, a person can reason well when she is nudged: Many of these heuristics are good rules to reason with, and she can be flexible with respect to them when she takes there to be a better way to reason. Along the way, this paper uncovers several design specifications for responsible nudging, and discusses open empirical questions. However, nudging’s being compatible with reasoning well crystallizes a distinct worry about manipulation: that nudge designers can rely on nudged people guiding themselves toward the designers’ own ends. Manipulation of this sort exploits one aspect of autonomy (namely, self-guidance) to undermine autonomy in other respects. (shrink)

This paper rejects an argument defending the view that the boundary between deception and manipulation is such that some manipulations intended to cause false beliefs count as non-deceptive. On the strongest version of this argument, if a specific behaviour involves compromising the victim’s reasoning, then the behaviour is manipulative but not deceptive, and if it involves exposing the victim to misleading evidence that justifies her false belief, then it is deceptive but not manipulative. This argument has been consistently used as (...) a reason to reject the traditional analysis of human deception, according to which intentionally causing someone to acquire a false belief is sufficient for deception. And because the traditional analysis is also consistent with our most basic intuitions about deception, it does matter whether this argument succeeds. (shrink)

Many philosophical accounts of manipulation are blind to the extent to which actual people fall short of the rational ideal, while prominent accounts in political science are under-inclusive. We offer necessary and sufficient conditions – Suitable Reason and Testimonial Honesty – distinguishing manipulative from non-manipulative influence; develop a ‘hypothetical disclosure test’ to measure the degree of manipulation; and provide further criteria to assess and compare the morality of manipulation across cases. We discuss multiple examples drawn from politics and from public (...) policy with particular attention to recent debates about the ethics and politics of nudge. (shrink)

Franke, in Philosophy & Technology, 37(1), 1–6, (2024), connects the recent debate about manipulative algorithmic transparency with the concerns about problematic pursuits of positive liberty. I argue that the indifference view of manipulative transparency is not aligned with positive liberty, contrary to Franke’s claim, and even if it is, it is not aligned with the risk that many have attributed to pursuits of positive liberty. Moreover, I suggest that Franke’s worry may generalise beyond the manipulative transparency debate to AI ethics (...) in general. (shrink)