Personal AI is a new type of AI companion, distinct from the prevailing forms of AI companionship. Instead of playing a narrow and well-defined social role, like friend, lover, caretaker, or colleague, with a set of pre-determined responses and behaviors, Personal AI is engineered to tailor itself to the user, including learning to mirror the user’s unique emotional language and attitudes. This paper identifies two issues with Personal AI. First, like other AI companions, it is deceptive about the presence of (...) their emotions, which undermines the moral value of companionship. Second, Personal AI leads to a distinctly new form of deception concerning the origins of its emotions. Its emotional attitudes appear to belong to it, when in fact they are only reflections of the user. This results in what I dub “emotional bubbles”—the false impression that personal emotions are externally validated—which have at least two troubling implications. First, emotional bubbles prevent us from encountering emotional attitudes that differ from our own, which is likely to cripple emotional growth and the ability to form diverse social and emotional relationships. Second, if we assume, as some philosophers claim, that shared emotions are constitutive of shared values, it follows that Personal AI subverts joint moral deliberation. Users believe their personal values are externally validated, when they are only validated by themselves. Because of the absence of technovirtues able to handle this problem, I suggest that we proceed very cautiously with the development and marketing of Personal AI. (shrink)

AI-based technologies are increasingly pervasive in a number of contexts. Our affective and emotional life makes no exception. In this article, we analyze one way in which AI-based technologies can affect them. In particular, our investigation will focus on affective artificial agents, namely AI-powered software or robotic agents designed to interact with us in affectively salient ways. We build upon the existing literature on affective artifacts with the aim of providing an original analysis of affective artificial agents and their distinctive (...) features. We argue that, unlike comparatively low-tech affective artifacts, affective artificial agents display a specific form of agency, which prevents them from being perceived by their users as extensions of their selves. In addition to this, we claim that their functioning crucially depends on the simulation of human-like emotion-driven behavior and requires a distinctive form of transparency—we call it emotional transparency—that might give rise to ethical and normative tensions. (shrink)

This paper presents an overview of the risks and benefits of Social AI, understood as conversational AI systems that cater to human social needs like romance, companionship, or entertainment. Section 1 of the paper provides a brief history of conversational AI systems and introduces conceptual distinctions to help distinguish varieties of Social AI and pathways to their deployment. Section 2 of the paper adds further context via a brief discussion of anthropomorphism and its relevance to assessment of human-chatbot relationships. Section (...) 3 of the paper provides a survey of potential and in some cases demonstrated harms associated with user interactions with Social AI systems. Finally, Section 4 discusses how the benefits and harms of Social AI can best be addressed, with a primary focus on how frameworks from AI ethics can inform their development. (shrink)

The thesis introduces and criticizes the discourse on technology, with a specific reference to the concept of AI. The discourse on AI is particularly saturated with reified metaphors which drive connotations and delimit understandings of technology in society. To better analyse the discourse on AI, the thesis proposes the concept of “Expected AI”, a composite signifier filled with historical and sociocultural connotations, and numerous referent objects. Relying on cultural semiotics, science and technology studies, and a diverse selection of heuristic concepts, (...) the thesis delves beneath the surface of AI discourse and demonstrates the hidden political, social, cultural, and ecological dangers of AI. The entanglement of the discourse(s) with (science) fiction, folklore, myth, and religion impacts how AI is perceived and received, as well as the expectations to AI-enabled technologies now and in the future. The thesis also proposes a more ethical and comprehensive ontological model for AI systems. The model describes AI systems as complex figurations, considering their socio-material organisation, global economic-material becoming, and impact on the environment, social institutions, and the semiosphere. The dissertation argues that AI should be understood not just as an object or sociotechnical system, but as its entire product chain encompassing people and cultures, as well as the used resources and impact (both material-ecological and semiotic) on a planetary scale. (shrink)

Advances in artificial intelligence (AI) create an increasing similarity between the performance of AI systems or AI-based robots and human communication. They raise the questions: whether it is possible to communicate with, understand, and even empathically perceive artificial agents; whether we should ascribe actual subjectivity and thus quasi-personal status to them beyond a certain level of simulation; what will be the impact of an increasing dissolution of the distinction between simulated and real encounters. (1) To answer these questions, the paper (...) argues that the precondition for actually understanding others consists in the implicit assumption of the subjectivity of our counterpart, which makes shared feelings and a „we-intentionality” possible. This assumption is ultimately based on the presupposition of a shared form of life, conceived here as „conviviality.” (2) The possibility that future artificial agents could meet these preconditions is refuted on the basis of embodied and enactive cognition, which links subjectivity and consciousness to the aliveness of an organism. (3) Even if subjectivity is in principle impossible for artificial agents, the distinction between simulated and real subjectivity might nevertheless become increasingly blurred. Here, possible consequences are discussed, especially using the example of virtual psychotherapy. Finally, the paper makes case for a mindful appproach to the language we use to talk about artificial systems and pleads for preventing a systematic pretense of subjectivity. (shrink)

This chapter defends the view that manipulated behaviour is explained by an injustice. Injustices that explain manipulated behaviour need not involve agential features such as intentionality. Therefore, technology can manipulate us, even if technological artefacts like robots, intelligent software agents, or other ‘mere tools’ lack agential features such as intentionality. The chapter thus sketches a comprehensive account of manipulated behaviour related to but distinct from existing accounts of manipulative behaviour. It then builds on that account to defend the possibility that (...) we are being manipulated by technology. (shrink)

Debates about terrorism and technology often focus on the potential uses of technology by non-state terrorist actors and by states as forms of counterterrorism. Yet, little has been written about how technology shapes how we think about terrorism. In this chapter I argue that technology, and the language we use to talk about technology, constrains and shapes our understanding of the nature, scope, and impact of terrorism, particularly in relation to state terrorism. After exploring the ways in which technology shapes (...) moral thinking, I use two case studies to demonstrate how technology simultaneously hides and enables terroristic forms of state violence: police control technologies and Unmanned Aerial Vehicles (UAVs), or drones. In both these cases, I argue that features of these technologies, combined with a narrative of precision and efficiency, serves to mask the terroristic nature of the violence that these practices inflict and reinforces the moral exclusion of those against whom these technologies are deployed. In conclusion, I propose that identifying acts of terrorism requires a focus on the impact of technologies of violence (whether they are “high tech” or not) on those most affected, regardless of whether users of these technologies conceive of their actions as terroristic. (shrink)

Mark Coeckelbergh (Int J Soc Robot 1:217–221, 2009) argues that robot ethics should investigate what interaction with robots can do to humans rather than focusing on the robot’s moral status. We should ask what robots do to our sociality and whether human–robot interaction can contribute to the human good and human flourishing. This paper extends Coeckelbergh’s call and investigate what it means to live with disembodied AI-powered agents. We address the following question: Can the human–AI interaction contribute to our moral (...) development? We present an empirically informed philosophical analysis of how the AI personal assistant Siri changes its users’ way of life, based on the responses obtained from 20 semi-directive individual interviews with Siri users. We identify changes in the users’ social interaction associated with the adoption of Siri. These changes include: (1) the indirect effect of reducing opportunities of human interaction, (2) the second-order effect of diminished expectations toward each other in a community, and (3) the acquired preference to obtain hassle-free interaction with Siri over human interaction. We examine them in relation to concerns that are voiced in the current debates over the rise of AI, namely the suspicion that humans could become overly reliant on AI (Danaher 2019) and the worry that social AI could impede on moral development (Fröding and Peterson, Ethics Inf Technol 23:207–214, 2012; Li, Ethics Inf Technol 23:543–550, 2021). We analyze the ethical costs that come from these changes in light of virtue ethics and address potential objections along the way. We end by offering directions for thinking about how to live with AI personal assistant while preserving favorable conditions for moral development. (shrink)

This article discusses the application of artificially intelligent robots within eldercare and explores a series of ethical considerations, including the challenges that AI (Artificial Intelligence) technology poses to traditional Chinese Confucian filial piety. From the perspective of Confucian ethics, the paper argues that robots cannot adequately fulfill duties of care. Due to their detachment from personal relationships and interactions, the “emotions” of AI robots are merely performative reactions in different situations, rather than actual emotional abilities. No matter how “humanized” robots (...) become, it is difficult to establish genuine empathy and a meaningful relationship with them for this reason. Even so, we acknowledge that AI robots are a significant tool in managing the demands of elder care and the growth of care poverty, and as such, we attempt to outline some parameters within which care robotics could be acceptable within a Confucian ethical system. Finally, the paper discusses the social impact and ethical considerations brought on by the interaction between humans and machines. It is observed that the relationship between humans and technology has always had both utopian and dystopian aspects, and robotic elder care is no exception. AI caregiver robots will likely become a part of elder care, and the transformation of these robots from “service providers” to “companions” seems inevitable. In light of this, the application of AI-augmented robotic elder care will also eventually change our understanding of interpersonal relationships and traditional requirements of filial piety. (shrink)

The implementation of care robotics in care settings is identified by some authors as a disruptive innovation, in the sense that it will upend the praxis of care. It is an open ethical question whether this alleged disruption will also have a transformative impact on established ethical concepts and principles. One prevalent worry is that the implementation of care robots will turn deception into a routine component of elderly care, at least to the extent that these robots will function as (...) simulacra for something that they are not (i.e. human caregivers). At face value, this may indeed seem to indicate a concern for how this technology may upend existing practices and relationships within a care setting. Yet, on closer inspection, this reaction may rather point to a rediscovery and a revaluation of a particularly well-entrenched value or virtue, i.e. veracity. The virtue of veracity is one of the values that is mobilized to argue against a substitution of human caregivers (while a combination of care robots and human caregivers is much more accepted). The subject of this paper is to explore how the moral panic surrounding care robots should not so much be interpreted as an anticipated and probable disruptor in a care setting, but rather as a sensitizing – in a way conservationist – argument that identifies veracity as an established value that is supposed to be protected and advanced in present day and future care settings. (shrink)

If existing or future robots appear to have some capacity, state or property, how can we determine whether they truly have it or whether we are deceived into believing so? John Danaher addresses this question by formulating his approach to what he refers to as superficial state deception (SSD) from the perspective of his theory termed ethical behaviourism (EB), which was initially designed to determine the moral status of robots. In summary, Danaher believes that focusing on behaviour is sufficient to (...) determine whether SSD occurs. My general claim is that Danaher’s approach to SSD based on EB is implausible since it results in the impossibility of conceptualizing SSD, e.g., it does not enable determining whether or not SSD occurs in a particular case. Moreover, I show how Danaher’s approach to SSD needs to be transformed to become plausible. To make my point, I (1) examine the main features of EB and distinguish its two versions by showing how Danaher revised the original EB in response to criticism; (2) discuss Danaher’s approach to the problem of deception from the perspective of EB; (3) criticize that approach by showing that it requires revisions analogous to those that have already been recommended in reference to EB, and (4) propose an alternative method for determining the presence of SSD that covers diverse, plausible approaches to SSD. (shrink)