The increasing diffusion of novel digital and online sociotechnical systems for arational behavioral influence based on Artificial Intelligence (AI), such as social media, microtargeting advertising, and personalized search algorithms, has brought about new ways of engaging with users, collecting their data and potentially influencing their behavior. However, these technologies and techniques have also raised concerns about the potential for manipulation, as they offer unprecedented capabilities for targeting and influencing individuals on a large scale and in a more subtle, automated and (...) pervasive manner than ever before. This paper, provides a narrative review of the existing literature on manipulation, with a particular focus on the role of AI and associated digital technologies. Furthermore, it outlines an account of manipulation based of four key requirements: intentionality, asymmetry of outcome, non-transparency and violation of autonomy. I argue that while manipulation is not a new phenomenon, the pervasiveness, automaticity, and opacity of certain digital technologies may raise a new type of manipulation, called “digital manipulation”. I call “digital manipulation” any influence exerted through the use of digital technology that is intentionally designed to bypass reason and to produce an asymmetry of outcome between the data processor (or a third party that benefits thereof) and the data subject. Drawing on insights from psychology, sociology, and computer science, I identify key factors that can make manipulation more or less effective, and highlight the potential risks and benefits of these technologies for individuals and society. I conclude that manipulation through AI and associated digital technologies is not qualitatively different from manipulation through human–human interaction in the physical world. However, some functional characteristics make it potentially more likely of evading the subject’s cognitive defenses. This could increase the probability and severity of manipulation. Furthermore, it could violate some fundamental principles of freedom or entitlement related to a person’s brain and mind domain, hence called neurorights. To this end, an account of digital manipulation as a violation of the neuroright to cognitive liberty is presented. (shrink)

Much liberal-democratic thought has concerned itself primarily – even exclusively – with coercive interference in citizens’ lives. But political actors do things – they engage in influential speech, they offer incentives, they mislead other actors, they disrupt the expected functioning of decision-making mechanisms etc. – that fall short of coercion, yet may nonetheless call for normative evaluation and public justification, precisely because they serve to purposively alter citizens’ beliefs, intentions and behaviour. With this article, I explicate a conception of political (...) manipulation to capture this sort of interference, and to distinguish individual manipulation from the manipulation of nonindividual agents like committees, institutions and states. The account, beyond being necessary for further work on the ethics of political manipulation, should prove useful to both normative thinkers interested in power, justice and the ethics of democratic decision-making, and empirical scholars in search of a conceptual apparatus to sharpen their investigations into the exercise of subtle forms of political power. (shrink)

In this paper, we introduce a distinctive kind of psychological abuse we call Tightlacing. We begin by presenting four examples and argue that there is a distinctive form of abuse in these examples that cannot be captured by our existing moral categories. We then outline our diagnosis of this distinctive form of abuse. Tightlacing consists in inducing a mistaken self-conception in others that licenses overburdening demands on them such that victims apply those demands to themselves. We discuss typical Tightlacing strategies (...) and argue that Tightlacing typically is manipulative. Typical tightlacers will be motivated by a strong desire to suppress a kind of behaviour on the victim’s part. We will then differentiate Tightlacing from a related and widely discussed form of psychological abuse, Gaslighting. While Gaslighting focuses on the victim’s epistemic capacities and typically serves to insulate the abuser from potential dissent, Tightlacing focuses on the kind of person the victim is and typically serves to insulate the abuser from confronting ways of behaviour they cannot cope with. While Gaslighting targets the victim’s epistemic self-trust, Tightlacing targets their basic sense of who they are and their sense of entitlement to conduct themselves as who they really are. We finish by diagnosing the wrong-making features of Tightlacing, arguing that Tightlacing, among many secondary wrongs, makes the victim complicit in a denial of their rights as well as an erasure of who they are. (shrink)

Selfish, dishonest or malicious agents may find an interest in manipulating others. While many works deal with designing robust systems or manipulative strategies, few works are interested in defining in a broad sense what is a manipulation and how we can reason with such a notion. In this article, based on a social science literature, we give a general definition of manipulation for multi-agent systems. A manipulation is a deliberate effect of an agent – called manipulator – to instrumentalize another (...) agent – called victim – while making sure to conceal that effect. We present then a logical framework, called KBE, to express and reason about manipulations. Since manipulation relies on deliberate effects, KBE introduces a deliberate BIAT operator which abstracts deliberate consequences of actions. We prove that this logic is sound and complete. Furthermore,we express related notions such as coercion, persuasion, or deception. (shrink)

How social media impacts the autonomy of its users is a topic of increasing focus. However, much of the literature that explores these impacts fails to engage in depth with the philosophical literature on autonomy. This has resulted in a failure to consider the full range of impacts that social media might have on autonomy. A deeper consideration of these impacts is thus needed, given the importance of both autonomy as a moral concept and social media as a feature of (...) contemporary life. By drawing on this philosophical literature, we argue that autonomy is broadly a matter of developing autonomy competencies, having authentic ends and control over key aspects of your own life, and not being manipulated, coerced, and controlled by others. We show how the autonomy of users of social media can be disrespected and harmed through the control that social media can have over its users’ data, attention, and behaviour. We conclude by discussing various recommendations to better regulate social media. (shrink)

Are we being manipulated online? If so, is being manipulated by online technologies and algorithmic systems notably different from human forms of manipulation? And what is under threat exactly when people are manipulated online? This volume provides philosophical and conceptual depth to debates in digital ethics about online manipulation. The contributions explore the ramifications of our increasingly consequential interactions with online technologies such as online recommender systems, social media, user-friendly design, micro-targeting, default-settings, gamification, and real-time profiling. The authors in this (...) volume address four broad and interconnected themes: • What is the conceptual nature of online manipulation? And how, methodologically, should the concept be defined? • Does online manipulation threaten autonomy, freedom, and meaning in life, and if so, how? • What are the epistemic, affective, and political harms and risks associated with online manipulation? • What are legal and regulatory perspectives on online manipulation? The Philosophy of Online Manipulation brings these various considerations together to offer philosophically robust answers to critical questions concerning our online interactions with one another and with autonomous systems. It will be of interest to researchers and advanced students working in moral philosophy, digital ethics, philosophy of technology, and the ethics of manipulation. (shrink)

Conceptual engineering is thought to face an ‘implementation challenge’: the challenge of securing uptake of engineered concepts. But is the fact that implementation is challenging really a defect to be overcome? What kind of picture of political life would be implied by making engineering easy to implement? We contend that the ambition to obviate the implementation challenge goes against the very idea of liberal democratic politics. On the picture we draw, the implementation challenge can be overcome by institutionalizing control over (...) conceptual uptake, and there are contexts—such as professions that depend on coordinated conceptual innovation—in which there are good reasons to institutionalize control in this fashion. But the liberal fear of this power to control conceptual uptake ending up in the wrong hands, combined with the democratic demand for freedom of thought as a precondition of genuine consent, yields a liberal democratic rationale for keeping implementation challenging. (shrink)

When we think about what others believe and want, we are usually affected by what we know about their attitudes. If I’m aware that another person believes something, I have an opportunity to agree or disagree with it. If I think that another person wants something, I can endorse or disapprove of her desire. The importance of such reactions to attributed beliefs and desires has thus far been overlooked in philosophy of mind where the focus has been on explanatory and (...) predictive roles of attitude attribution. The primary goal of this thesis is to fill this lacuna and to articulate the indispensability of such reactions – agreement/disagreement and endorsement/disapproval – for social cognition. In the process of doing it I also show how these initial reactions ground certain further possible responses in intersubjective communication: manipulation, negotiation or adoption of attributed attitudes. The second aim of this thesis is to explicate what is it about belief and desire attributions that makes the responses I’ve described possible. Because one can agree or disagree both with beliefs and assertions and endorse or disapprove both desires and requests, I argue that we should understand belief attributions in terms of assertions and desire attributions in terms of requests. When we think what someone believes and wants, we treat her as a conversation partner because her attitudes call for the same responses as speech acts do, even if she hasn’t made any explicit assertion or request herself. In short, beliefs and desires have communicative significance. Given such significance, we can also see what one needs to take into consideration when attributing these attitudes to another. My answer in the thesis is that belief and desire attributions have to be grounded in facts about the behaviour and well-being of attributees and that these facts also constitute the nature of beliefs and desires. (shrink)

Ever-increasing numbers of human interactions with intelligent software agents, online and offline, and their increasing ability to influence humans have prompted a surge in attention toward the concept of (online) manipulation. Several scholars have argued that manipulative influence is always hidden. But manipulation is sometimes overt, and when this is acknowledged the distinction between manipulation and other forms of social influence becomes problematic. Therefore, we need a better conceptualisation of manipulation that allows it to be overt and yet clearly distinct (...) from related concepts of social influence. I argue that manipulation is careless influence, show how this account helps to alleviate the shortcomings of the hidden influence view of manipulation, and derive implications for digital ethics. (shrink)

At the most general level, "manipulation" refers one of many ways of influencing behavior, along with (but to be distinguished from) other such ways, such as coercion and rational persuasion. Like these other ways of influencing behavior, manipulation is of crucial importance in various ethical contexts. First, there are important questions concerning the moral status of manipulation itself; manipulation seems to be mor- ally problematic in ways in which (say) rational persuasion does not. Why is this so? Furthermore, the notion (...) of manipulation has played an increasingly central role in debates about free will and moral responsibility. Despite its significance in these (and other) contexts, however, the notion of manipulation itself remains deeply vexed. I would say notoriously vexed, but in fact direct philosophical treatments of the notion of manipulation are few and far between, and those that do exist are nota- ble for the sometimes widely divergent conclusions they reach concerning what it is. I begin by addressing (though certainly not resolving) the conceptual issue of how to distinguish manipulation from other ways of influencing behavior. Along the way, I also briefly address the (intimately related) question of the moral status of manipulation: what, if anything, makes it morally problematic? Then I discuss the controversial ways in which the notion of manipulation has been employed in contemporary debates about free will and moral responsibility. (shrink)

In this paper we analyze the non-coercive ways in which researchers can use knowledge about the decision-making tendencies of potential participants in order to motivate them to consent to research enrollment. We identify which modes of influence preserve respect for participants’ autonomy and which disrespect autonomy, and apply the umbrella term of manipulation to the latter. We then apply our analysis to a series of cases adapted from the experiences of clinical researchers in order to develop a framework for thinking (...) through the ethics of manipulating people into research participation. All manipulation disrespects autonomy and is therefore pro tanto wrong. However, only deceptive manipulation invalidates the consent that results from it. Use of the other forms of manipulation can be permissible, but only if the outcome of using manipulation is sufficiently good and if the research cannot be carried out using ethically preferable means to obtain consent. (shrink)

Advances in immunotherapy pave the way for vaccines that target not only infections, but also unhealthy behaviors such as smoking. A nicotine vaccine that eliminates the pleasure associated with smoking could potentially be used to prevent children from adopting this addictive and dangerous behavior. This paper offers an ethical analysis of such vaccines. We argue that it would be permissible for parents to give their child a nicotine vaccine if the following conditions are met: (1) the vaccine is expected to (...) result in a net benefit to each individual vaccinated, (2) the expected harms from the side effects of the vaccine are lower than the non-voluntary harms of smoking, and (3) there are no less manipulative methods available that are as effective at preventing smoking initiation. Finally, we show how the framework developed here could be used to analyze the ethics of other chemical interventions designed to modify children’s behavior. (shrink)

This paper discusses the ethics of public health communication. We argue that a number of commonplace tools of public health communication risk qualifying as non-honest and question whether or not using such tools is ethically justified. First, we introduce the concept of honesty and suggest some reasons for thinking it is morally desirable. We then describe a number of common ways in which public health communication presents information about health-promoting interventions. These include the omission of information about the magnitude of (...) benefits people can expect from health-promoting interventions, and failure to report uncertainty associated with the outcomes of interventions. Next we outline some forms of behaviour which are generally recognised by philosophers as being non-honest, including deception, manipulation, and so on. Finally, we suggest that many of the public health communicative practices identified earlier share features with the non-honest behaviours described and suggest this warrants reflection upon whether such non-honesty is justified by the goals of public health communication. (shrink)

We influence each other constantly and in diverse ways. At times ethically, as when we convince others via arguments founded in good reason. At times problematically, as when we coerce others to ac...

Mobile health applications (‘health apps’) that promise the user to help her with some aspect of her health are very popular: for-profit apps such as MyFitnessPal, Fitbit, or Headspace have tens of millions of users each. For-profit health apps are designed and run as optimization systems. One would expect that these health apps aim to optimize the health of the user, but in reality they aim to optimize user engagement and, in effect, conversion. This is problematic, I argue, because digital (...) health environments that aim to optimize user engagement risk being manipulative. To develop this argument, I first provide a brief analysis of the underlying business models and the resulting designs of the digital environments provided by popular for-profit health apps. In a second step, I present a concept of manipulation that can help analyze digital environments such as health apps. In the last part of the article, I use my concept of manipulation to analyze the manipulative potential of for-profit health apps. Although for-profit health can certainly empower their users, the conditions for empowerment also largely overlap with the conditions for manipulation. As a result, we should be cautious when embracing the empowerment discourse surrounding health apps. An additional aim of this article is to contribute to the rapidly growing literature on digital choice architectures and the ethics of influencing behavior through such choice architectures. I take health apps to be a paradigmatic example of digital choice architectures that give rise to ethical questions, so my analysis of the manipulative potential of health apps can also inform the larger literature on digital choice architectures. (shrink)

The paper focuses on the nature and moral status of manipulation. I analyse a popular account of manipulation by Robert Noggle and assess a challenge that has been posed by Moti Gorin. I argue that Noggle’s theory can fend off the challenge. The analysis is instructive in that it enables one to look more closely at the nature of manipulation. I argue, contrary to some proposed accounts, that manipulation essentially involves deception about the manipulator’s intentions. Secondly, since manipulation contains an (...) element of deception, it is, I maintain, prima facie immoral. Finally, I analyse and explain away several examples of allegedly morally non-problematic manipulation. (shrink)

May a government attempt to improve the lives of its citizens by promoting the activities it deems valuable and discouraging those it disvalues? May it engage in such a practice even when doing so is not a requirement of justice in some strict sense, and even when the judgments of value and disvalue in question are likely to be subject to controversy among its citizens? These questions have long stood at the center of debates between political perfectionists and political neutralists. (...) In what follows I address a prominent cluster of arguments against political perfec- tionism—namely, arguments that focus on the coercive dimensions of state action. My main claim is simple: whatever concerns we might have about coercion, arguments from coercion fall short of supporting a thoroughgoing rejection of perfectionism, for the reason that perfectionist policies need not be coercive. The main body of the paper responds, however, to several neutra- list challenges to this last claim. (shrink)

A longstanding debate exists in both academic literature and popular culture about whether non-informative marketing tactics are manipulative. However, given that we tend to believe that some marketing tactics are manipulative and some are not, the question that marketers, their critics, and consumers need to ask themselves is that of how to actually determine whether any particular marketing tactic is manipulative and whether a given manipulative tactic is, in fact, immoral. This article proposes to operationalize criteria that can be used (...) by marketers for making such determinations and attempts to provide some clarification toward our under- standing of the concept of manipulation and the conditions for the moral acceptability of manipulative marketing practices. It argues that a marketing tactic is manipulative if it is intended to motivate by undermining what the marketer believes is his/her audience’s normal decision-making process either by deception or by playing on a vulnerability that the marketer believes exists in his/her audience’s normal decision-making process. Such a tactic is morally objectionable on several grounds, which make it morally impermissible unless outweighed by sufficient “redemptive” moral considerations. (shrink)

Using practical formalism a deontological ethical analysis of peer relations in organizations is developed. This analysis is composed of two types of duties derived from Kant's Categorical Imperative: negative duties to refrain from the use of peers and positive duties to provide help and assistance. The conditions under which these duties pertain are specified through the development of examples and conceptual distinctions. A number of implications are then discussed.

Eyewitness testimony is a powerful form of evidence, and this is especially true in the United States criminal legal system. At the same time, eyewitness misidentification is the greatest contributing factor to wrongful convictions proven by DNA testing. In this paper, I offer a close examination of this tension between the enormous epistemic weight that eyewitness testimony is afforded in the United States criminal legal system and the fact that there are important questions about its reliability as a source of (...) evidence. In Sections 1 and 2, I argue that lineups and interrogations often function by way of extracting testimony from an eyewitness through practices that are manipulative, deceptive, or coercive. I then show, in Section 3, that when testimony that is extracted in these ways is given an unwarranted excess of credibility, the eyewitness in question is the victim of what I call agential testimonial injustice. I conclude that since much of the testimony of eyewitnesses is both extracted and given an excess of credibility, there is a fairly widespread form of epistemic injustice being inflicted upon testifiers in the United States criminal legal system. This calls for reforms along both dimensions—lineups and interrogations should go through a witness's epistemic agency, rather than bypassing, exploiting, or undermining it, and the weight of the resulting testimony should be viewed in the broader context of its significant fallibility. (shrink)

Many destructive business leaders drive their companies into bankruptcy and dissolution, never to be heard from again in the business press. However, it is useful to study these organizations to prevent the same, or similar destructive business from taking on, and destroying, additional businesses. In this article, we describe one type of organization that follows the model of religious cults, which we call secular business cults. Building on Padilla et al., we describe an SBC toxic triangle of (1) Padilla et (...) al.'s societal factors, (2) additional business factors, and (3) antisocial actors that set the stage for the SBC's emergence. We then describe the characteristics of the SBC's operations, such as an ultrastrong culture, formalized manipulation, manipulative hierarchical relationships, competition, and operations efficiency tools, and the frequent overcommunication of deceptive messages. It is our hope that this unique perspective on organizations will inspire research into an overlooked area of unethical behavior in businesses today. In our Part II paper on the topic, we intend to extend the idea of SBCs to manipulative businesses, which are essentially reorganized, well‐established and legally‐complying revisions of SBCs, and then to manipulative industries and societies. (shrink)

This paper analyses the way in which being in possession of an adequate range of options is an essential component of autonomy. I discuss the way in which the conceptualisation of options in terms of basic rights might assist this argument, and apply these ideas to HIV/AIDS clinical research in the developing world. Finally, I suggest that mechanisms should be put in place through which vulnerable research participants can express their views about the relationship between the research in which they (...) are involved, and needs of a worthwhile life. (shrink)

Australia's policy of mandatory indefinite detention of those seeking asylum and arriving without valid documents has led to terrible human rights abuses and cumulative deterioration in health for those incarcerated. We argue that there is an imperative to research and document the plight of those who have suffered at the hands of the Australian government and its agents. However, the normal tools available to those engaged in health research may further erode the rights and well being of this population, requiring (...) a rethink of existing research ethics paradigms to approaches that foster advocacy research and drawing on the voices of those directly affected, including those bestowed with duty of care for this population. (shrink)

In this paper, I offer a novel account of entrapment. This account suggests that the wrongness of pursuing punishment in cases of entrapment consists of two distinct components, one concerning the culpability of the entrapped defendant and the other concerning the legitimacy of the entrapping state to prosecute crimes that it has effectively created. Distinguishing these two components of entrapment, I explain, helps to clarify the moral issues at stake and to resolve some confusions and debates in existing legal analyses (...) of entrapment. (shrink)

ABSTRACT Persons generally must give their informed consent to participate in research. To provide informed consent persons must be given information regarding the study in simple, lay language. Consent must be voluntary, and persons giving consent must be legally competent to consent and possess the capacity to understand and appreciate the information provided. This paper examines the relationship between the obligation to disclose information regarding risks and the requirement that persons have the capacity to understand and appreciate the information. There (...) has been insufficient attention to the extent to which persons must be able to understand and appreciate study information in order to have their consent deemed valid when the information is provided in simple, lay language. This paper argues that (1) the capacity to understand and appreciate information that should be deemed necessary to give valid consent should be defined by the capacity of the typical, cognitively normal adult and (2) the capacity of the typical, cognitively normal adult to understand and appreciate the concept of risk is limited. Therefore, (3) all things being equal, potential subjects must possess a limited capacity to understand and appreciate risk to be deemed competent to consent to research participation. (4) In some cases investigators ought to require that persons possess a greater than typical capacity to understand and appreciate risk. (shrink)

Today's international community may well view covert action and democracy as mutually exclusive policies. This article examines the practice of covert action in American foreign policy in light of events of the mid-1970s and 1980s, focusing on the scandalous misuse of executive authority and lack of accountability associated with covert means. Often manipulative and sometimes anonymous, covert operations raise critical morality concerns in a democratic society. Whether "any form of accountability is likely to be sufficient to bring the unauthorized use (...) of executive power under control" is the crucial issue to be addressed when examining the practicality of covert actions by the executive branch. (shrink)