The present paper aims at contributing to the ethical debate on the impacts of artificial intelligence (AI) systems on human autonomy. More specifically, it intends to offer a clearer understanding of the design challenges to the effort of aligning driving automation technologies to this ethical value. After introducing the discussion on the ambiguous impacts that AI systems exert on human autonomy, the analysis zooms in on how the problem has been discussed in the literature on connected and automated vehicles (CAVs). (...) On this basis, it is claimed that the issue has been mainly tackled on a fairly general level, and mostly with reference to the controversial issue of crash-optimization algorithms, so that only limited design insights have been drawn. However, integrating ethical analysis and design practices is critical to pursue the implementation of such an important ethical value into CAV technologies. To this aim, it is argued, a more applied approach targeted at examining the impacts on human autonomy of current CAV functions should also be explored. As an example of the intricacy of this task, the case of automated route planning is discussed in some detail. (shrink)

Artificial Social Agents (ASAs), which are AI software driven entities programmed with rules and preferences to act autonomously and socially with humans, are increasingly playing roles in society. As their sophistication grows, humans will share greater amounts of personal information, thoughts, and feelings with ASAs, which has significant ethical implications. We conducted a study to investigate what ethical principles are of relative importance when people engage with ASAs and whether there is a relationship between people’s values and the ethical principles (...) they prioritise. The study uses scenarios, embedded with the five AI4People ethical principles (Beneficence, Non-maleficence, Autonomy, Justice, and Explicability), involving ASAs taking on roles traditionally played by humans to understand whether these roles and behaviours (including dialogues) are seen as acceptable or unacceptable. Results from 268 participants reveal the greatest sensitivity to ASA behaviours that relate to Autonomy, Justice, Explicability, and the privacy of their personal data. Models were created using Schwartz’s Refined Values as a possible indicator of how stakeholders discern and prioritise the different AI4People ethical principles when interacting with ASAs. Our findings raise issues around the ethical acceptability of ASAs for nudging and changing behaviour due to participants’ desire for autonomy and their concerns over deceptive ASA behaviours such as pretending to have memories and emotions. (shrink)

Social robots are robots that can interact socially with humans. As social robots and the artificial intelligence that powers them becomes more advanced, they will likely take on more social and work roles. This has many important ethical implications. In this paper, we focus on one of the most central of these, the impacts that social robots can have on human autonomy. We argue that, due to their physical presence and social capacities, there is a strong potential for social robots (...) to enhance human autonomy as well as several ways they can inhibit and disrespect it. We argue that social robots could improve human autonomy by helping us to achieve more valuable ends, make more authentic choices, and improve our autonomy competencies. We also argue that social robots have the potential to harm human autonomy by instead leading us to achieve fewer valuable ends ourselves, make less authentic choices, decrease our autonomy competencies, make our autonomy more vulnerable, and disrespect our autonomy. Whether the impacts of social robots on human autonomy are positive or negative overall will depend on the design, regulation, and use we make of social robots in the future. (shrink)

Can Artificial Intelligence be more effective than human instruction for the moral enhancement of people? The author argues that it only would be if the use of this technology were aimed at increasing the individual's capacity to reflectively decide for themselves, rather than at directly influencing behaviour. To support this, it is shown how a disregard for personal autonomy, in particular, invalidates the main proposals for applying new technologies, both biomedical and AI-based, to moral enhancement. As an alternative to these (...) proposals, this article proposes a virtual assistant that, through dialogue, neutrality and virtual reality technologies, can teach users to make better moral decisions on their own. The author concludes that, as long as certain precautions are taken in its design, such an assistant could do this better than a human instructor adopting the same educational methodology. (shrink)

This paper offers a theoretical framework that can be used to derive viable engineering strategies for the design and development of robots that can nudge people towards moral improvement. The framework relies on research in developmental psychology and insights from Stoic ethics. Stoicism recommends contemplative practices that over time help one develop dispositions to behave in ways that improve the functioning of mechanisms that are constitutive of moral cognition. Robots can nudge individuals towards these practices and can therefore help develop (...) the dispositions to, for example, extend concern to others, avoid parochialism, etc. (shrink)

Several proposals for moral enhancement would use AI to augment (auxiliary enhancement) or even supplant (exhaustive enhancement) human moral reasoning or judgment. Exhaustive enhancement proposals conceive AI as some self-contained oracle whose superiority to our own moral abilities is manifest in its ability to reliably deliver the ‘right’ answers to all our moral problems. We think this is a mistaken way to frame the project, as it presumes that we already know many things that we are still in the process (...) of working out, and reflecting on this fact reveals challenges even for auxiliary proposals that eschew the oracular approach. We argue there is nonetheless a substantial role that ‘AI mentors’ could play in our moral education and training. Expanding on the idea of an AI Socratic Interlocutor, we propose a modular system of multiple AI interlocutors with their own distinct points of view reflecting their training in a diversity of concrete wisdom traditions. This approach minimizes any risk of moral disengagement, while the existence of multiple modules from a diversity of traditions ensures pluralism is preserved. We conclude with reflections on how all this relates to the broader notion of moral transcendence implicated in the project of AI moral enhancement, contending it is precisely the whole concrete socio-technical system of moral engagement that we need to model if we are to pursue moral enhancement. (shrink)

This article explores the implications of what it means to moralize about future technological innovations. Specifically, I have been invited to comment on three papers that attempt to think about what seems to be an impending social reality: the availability of life-like sex robots. In response, I explore what it means to moralize about future technological innovations from a secular perspective, i.e., a perspective grounded in an immanent, socio-historically contingent view. I review the arguments of Nancy Jecker, Mark Howard and (...) Robert Sparrow, and Wang Jue and respond to their arguments concerning the permissible limits of human-robot sexual interaction. I argue that we are in a poor epistemic position regarding what the actual future human response will be towards sex robots and how it affects society generally. Given this poor epistemic position, I argue that moralizing about future trends like human-robot sex is difficult because we do not have the relevant facts to work with. Furthermore, I remain skeptical as to policy recommendations based on socio-historically contingent moral viewpoints both because they do not carry in principle moral authority to say what future others may or may not do with their property and they may not even appeal to future secular others, insofar as secular morality is plural and consistently develops anew. (shrink)

A major approach to the ethics of artificial intelligence is to use social choice, in which the AI is designed to act according to the aggregate views of society. This is found in the AI ethics of “coherent extrapolated volition” and “bottom–up ethics”. This paper shows that the normative basis of AI social choice ethics is weak due to the fact that there is no one single aggregate ethical view of society. Instead, the design of social choice AI faces three (...) sets of decisions: standing, concerning whose ethics views are included; measurement, concerning how their views are identified; and aggregation, concerning how individual views are combined to a single view that will guide AI behavior. These decisions must be made up front in the initial AI design—designers cannot “let the AI figure it out”. Each set of decisions poses difficult ethical dilemmas with major consequences for AI behavior, with some decision options yielding pathological or even catastrophic results. Furthermore, non-social choice ethics face similar issues, such as whether to count future generations or the AI itself. These issues can be more important than the question of whether or not to use social choice ethics. Attention should focus on these issues, not on social choice. (shrink)

Robotics is currently not only a cutting-edge research area, but is potentially disruptive to all domains of our lives—for better and worse. While legislation is struggling to keep pace with the development of these new artifacts, our intellectual limitations and physical laws seem to present the only hard demarcation lines, when it comes to state-of-the-art R&D. To better understand the possible implications, the paper at hand critically investigates underlying processes and structures of robotics in the context of Heidegger’s and Nishitani’s (...) accounts of science and technology. Furthermore, the analysis draws on Bauman’s theory of modernity in an attempt to assess the potential risk of large-scale robot integration. The paper will highlight undergirding mechanisms and severe challenges imposed upon our socio-cultural lifeworlds by massive robotic integration. Admittedly, presenting a mainly melancholic account, it will, however, also explore the possibility of robotics forcing us to reassess our position and to solve problems, which we seem unable to tackle without facing existential crises. (shrink)

This paper critically assesses the possibility of moral enhancement with ambient intelligence technologies and artificial intelligence presented in Savulescu and Maslen (2015). The main problem with their proposal is that it is not robust enough to play a normative role in users’ behavior. A more promising approach, and the one presented in the paper, relies on an artifi-cial moral reasoning engine, which is designed to present its users with moral arguments grounded in first-order normative theories, such as Kantianism or utilitarianism, (...) that reason-responsive people can be persuaded by. This proposal can play a normative role and it is also a more promising avenue towards moral enhancement. It is more promising because such a system can be designed to take advantage of the sometimes undue trust that people put in automated technologies. We could therefore expect a well-designed moral reasoner system to be able to persuade people that may not be persuaded by similar arguments from other people. So, all things considered, there is hope in artificial intelli-gence for moral enhancement, but not in artificial intelligence that relies solely on ambient intelligence technologies. (shrink)

Many of our daily activities are now made more convenient and efficient by virtual assistants, and the day when they can be designed to instruct us in certain skills, such as those needed to make moral judgements, is not far off. In this paper we ask to what extent it would be ethically acceptable for these so-called virtual assistants for moral enhancement to use subtle strategies, known as “nudges”, to influence our decisions. To achieve our goal, we will first characterise (...) nudges in their standard use and discuss the debate they have generated around their possible manipulative character, establishing three conditions of manipulation. Secondly, we ask whether nudges can occur in moral virtual assistants that are not manipulative. After critically analysing some proposed virtual assistants, we argue in favour of one of them, given that by pursuing an open and neutral moral enhancement, it promotes and respects the autonomy of the person as much as possible. Thirdly, we analyse how nudges could enhance the functioning of such an assistant, and evaluate them in terms of their degree of threat to the subject’s autonomy and their level of transparency. Finally, we consider the possibility of using motivational nudges, which not only help us in the formation of moral judgements but also in our moral behaviour. (shrink)

Los recientes avances en neurociencia, y en las ciencias de la vida en general, han hecho posible no solo tratar y restaurar las funciones dañadas, sino la posibilidad de algún día mejorar las funciones en la población saludable. Las amplias aplicaciones de la biotecnología y medicación farmacológica se dirigen a los mecanismos próximos que se consideran deficitarios en los pacientes. Recientemente, se ha mostrado como algunas de estas medicaciones pueden afectar comportamientos complejos humanos, como la moral en personas saludables, lo (...) cual abre la puerta para un entendimiento más profundo de la naturaleza humana. ¿Hasta que punto es éticamente permisible alterar la neuroquímica o modular una región cerebral para intervenir en el comportamiento moral?, ¿es intrínsecamente malo la biomejora moral?, ¿cómo podemos conocer los efectos adversos y cómo sopesarlos en un análisis coste/beneficio de una intervención?, ¿la biomejora moral supone una amenaza a la libertad? Considerando dos posturas opuestas, DeGrazia y Harris, sobre los retos éticos de la biomejora moral se sugiere que si los principios tradicionales de la bioética como la seguridad, efectividad y universalidad se cumplen no hay nada malo en la biomejora moral per se y, por consiguiente, es éticamente permisible. (shrink)

The increase in the spread of conversational agents urgently requires to tackle the ethical issues linked to their design. In fact, developers frequently include in their products cues that trigger social biases in order to maximize the performance and the quality of human-machine interactions. The present paper discusses whether and to what extent it is ethically sound to intentionally trigger gender biases through the design of virtually embodied conversational agents. After outlining the complex dynamics involving social biases, social robots, and (...) design, we evaluate the ethics of integrating gender cues in conversational agents, analysing four different approaches to the problem. Finally, we suggest which approach in our opinion might have the best chances to reduce the negative effects of biases and discriminatory visions of gender dynamics. (shrink)

The rapid adoption and implementation of artificial intelligence in medicine creates an ontologically distinct situation from prior care models. There are both potential advantages and disadvantages with such technology in advancing the interests of patients, with resultant ontological and epistemic concerns for physicians and patients relating to the instatiation of AI as a dependent, semi- or fully-autonomous agent in the encounter. The concept of libertarian paternalism potentially exercised by AI (and those who control it) has created challenges to conventional assessments (...) of patient and physician autonomy. The unclear legal relationship between AI and its users cannot be settled presently, an progress in AI and its implementation in patient care will necessitate an iterative discourse to preserve humanitarian concerns in future models of care. This paper proposes that physicians should neither uncritically accept nor unreasonably resist developments in AI but must actively engage and contribute to the discourse, since AI will affect their roles and the nature of their work. One’s moral imaginative capacity must be engaged in the questions of beneficence, autonomy, and justice of AI and whether its integration in healthcare has the potential to augment or interfere with the ends of medical practice. (shrink)

Moral bioenhancement, nudge-designed environments, and ambient persuasive technologies may help people behave more consistently with their deeply held moral convictions. Alternatively, they may aid people in overcoming cognitive and affective limitations that prevent them from appreciating a situation’s moral dimensions. Or they may simply make it easier for them to make the morally right choice by helping them to overcome sources of weakness of will. This paper makes two assumptions. First, technologies to improve people’s moral capacities are realizable. Second, such (...) technologies will actually help people get morality right and behave more consistently with whatever the ‘real’ right thing to do turns out to be. The paper then considers whether or not humanity loses anything valuable, particularly opportunities for moral progress, when being moral is made much easier by eliminating difficult moral deliberation and internal moral struggle. Ultimately, the worry that moral struggle has value as a catalyst for moral progress is rejected. Moral progress is understood here as the discovery and application of new values or sensitization to new sources of harm. (shrink)

An under-examined aspect of human–robot interaction that warrants further exploration is whether robots should be permitted to influence a user’s behavior for that person’s own good. Yet an even more controversial practice could be on the horizon, which is allowing a robot to “nudge” a user’s behavior for the good of society. In this article, we examine the feasibility of creating companion robots that would seek to nurture a user’s empathy toward other human beings. As more and more computing devices (...) subtly and overtly influence human behavior, it is important to draw attention to whether it would be ethically appropriate for roboticists to pursue this type of design pathway. Our primary focus is on whether a companion robot could encourage humans to perform charitable acts; this design possibility illustrates the range of socially just actions that a robot could potentially elicit from a user and what the associated ethical concerns may be. (shrink)

The ethics of emerging forms of artificial intelligence has become a prolific subject in both academic and public spheres. A great deal of these concerns flow from the need to ensure that these technologies do not cause harm—physical, emotional or otherwise—to the human agents with which they will interact. In the literature, this challenge has been met with the creation of artificial moral agents: embodied or virtual forms of artificial intelligence whose decision procedures are constrained by explicit normative principles, requiring (...) the implementation of what is commonly called artificial morality into these agents. To date, the types of reasoning structures and principles which inform artificial morality have been of two kinds: first, an ethically maximal vision of artificial morality which relies on the strict implementation of traditional moral theories such as Kantian deontology or Utilitarianism, and second, a more minimalist vision which applies stochastic AI techniques to large data sets of human moral preferences so as to illicit or intuit general principles and preferences for the design of artificial morality. Taken individually, each approach is unable to fully answer the challenge of producing inoffensive behavior in artificial moral agents, most especially since both forms are unable to strike a balance between the ideal set of constraints which morality imposes on one hand, and the types of constraints public acceptability imposes, on the other. We provide an alternative approach to the design of artificial morality, the Ethical Valence Theory, whose purpose is to accommodate this balance, and apply this approach to the case of autonomous vehicles. (shrink)