The enhancement of human traits has received academic attention for decades, but only recently has moral enhancement using biomedical means – moral bioenhancement (MB) – entered the discussion. After explaining why we ought to take the possibility of MB seriously, the paper considers the shape and content of moral improvement, addressing at some length a challenge presented by reasonable moral pluralism. The discussion then proceeds to this question: Assuming MB were safe, effective, and universally available, would it be morally desirable? (...) In particular, would it pose an unacceptable threat to human freedom? After defending a negative answer to the latter question – which requires an investigation into the nature and value of human freedom – and arguing that there is nothing inherently wrong with MB, the paper closes with reflections on what we should value in moral behaviour. (shrink)

This paper explores the ambiguous impact of new information and communications technologies on the cultivation of moral skills in human beings. Just as twentieth century advances in machine automation resulted in the economic devaluation of practical knowledge and skillsets historically cultivated by machinists, artisans, and other highly trained workers , while also driving the cultivation of new skills in a variety of engineering and white collar occupations, ICTs are also recognized as potential causes of a complex pattern of economic deskilling, (...) reskilling, and upskilling. In this paper, I adapt the conceptual apparatus of sociological debates over economic deskilling to illuminate a different potential for technological deskilling/upskilling, namely the ability of ICTs to contribute to the moral deskilling of human users, a potential that exists alongside rich but currently underrealized possibilities for moral reskilling and/or upskilling. I flesh out this general hypothesis by means of examples involving automated weapons technology, new media practices, and social robotics. I conclude that since moral skills are essential prerequisites for the effective development of practical wisdom and virtuous character, and since market and cultural forces are not presently aligned to bring about the more salutary of the ambiguous potentials presented here, the future shape of these developments warrants our close attention—and perhaps active intervention. (shrink)

Ethical decision-making descriptive theoretical models often conflict with each other and typically lack comprehensiveness. To address this deficiency, a revised EDM model is proposed that consolidates and attempts to bridge together the varying and sometimes directly conflicting propositions and perspectives that have been advanced. To do so, the paper is organized as follows. First, a review of the various theoretical models of EDM is provided. These models can generally be divided into rationalist-based ; and non-rationalist-based. Second, the proposed model, called (...) ‘Integrated Ethical Decision Making,’ is introduced in order to fill the gaps and bridge the current divide in EDM theory. The individual and situational factors as well as the process of the proposed model are then described. Third, the academic and managerial implications of the proposed model are discussed. Finally, the limitations of the proposed model are presented. (shrink)

We describe a form of moral artificial intelligence that could be used to improve human moral decision-making. We call it the “artificial moral advisor”. The AMA would implement a quasi-relativistic version of the “ideal observer” famously described by Roderick Firth. We describe similarities and differences between the AMA and Firth’s ideal observer. Like Firth’s ideal observer, the AMA is disinterested, dispassionate, and consistent in its judgments. Unlike Firth’s observer, the AMA is non-absolutist, because it would take into account the human (...) agent’s own principles and values. We argue that the AMA would respect and indeed enhance individuals’ moral autonomy, help individuals achieve wide and a narrow reflective equilibrium, make up for the limitations of human moral psychology in a way that takes conservatives’ objections to human bioenhancement seriously, and implement the positive functions of intuitions and emotions in human morality without their downsides, such as biases and prejudices. (shrink)

During the past decade, the “script” concept, indicating how technologies prescribe human actions, has acquired a central place in STS. Until now, the concept has mainly functioned in descriptive settings. This article will deploy it in a normative setting. When technologies coshape human actions, they give material answers to the ethical question of how to act. This implies that engineers are doing “ethics by other means”: they materialize morality. The article will explore the implications of this insight for engineering ethics. (...) It first augments the script concept by developing the notion of technological mediation. After this, it investigates how the concept of mediation could be made fruitful for design ethics. It discusses how the ambition to design behaviorinfluencing technologies raises moral questions itself and elaborates two methods for anticipating technological mediation in the design process: performing mediation analyses and using an augmented version of constructive technology assessment. (shrink)

BackgroundHealthcare providers have to make ethically complex clinical decisions which may be a source of stress. Researchers have recently introduced Artificial Intelligence (AI)-based applications to assist in clinical ethical decision-making. However, the use of such tools is controversial. This review aims to provide a comprehensive overview of the reasons given in the academic literature for and against their use.MethodsPubMed, Web of Science, Philpapers.org and Google Scholar were searched for all relevant publications. The resulting set of publications was title and abstract (...) screened according to defined inclusion and exclusion criteria, resulting in 44 papers whose full texts were analysed using the Kuckartz method of qualitative text analysis.ResultsArtificial Intelligence might increase patient autonomy by improving the accuracy of predictions and allowing patients to receive their preferred treatment. It is thought to increase beneficence by providing reliable information, thereby, supporting surrogate decision-making. Some authors fear that reducing ethical decision-making to statistical correlations may limit autonomy. Others argue that AI may not be able to replicate the process of ethical deliberation because it lacks human characteristics. Concerns have been raised about issues of justice, as AI may replicate existing biases in the decision-making process.ConclusionsThe prospective benefits of using AI in clinical ethical decision-making are manifold, but its development and use should be undertaken carefully to avoid ethical pitfalls. Several issues that are central to the discussion of Clinical Decision Support Systems, such as justice, explicability or human–machine interaction, have been neglected in the debate on AI for clinical ethics so far.Trial registrationThis review is registered at Open Science Framework (https://osf.io/wvcs9). (shrink)

This paper offers a theoretical framework that can be used to derive viable engineering strategies for the design and development of robots that can nudge people towards moral improvement. The framework relies on research in developmental psychology and insights from Stoic ethics. Stoicism recommends contemplative practices that over time help one develop dispositions to behave in ways that improve the functioning of mechanisms that are constitutive of moral cognition. Robots can nudge individuals towards these practices and can therefore help develop (...) the dispositions to, for example, extend concern to others, avoid parochialism, etc. (shrink)

This paper critically assesses the possibility of moral enhancement with ambient intelligence technologies and artificial intelligence presented in Savulescu and Maslen (2015). The main problem with their proposal is that it is not robust enough to play a normative role in users’ behavior. A more promising approach, and the one presented in the paper, relies on an artifi-cial moral reasoning engine, which is designed to present its users with moral arguments grounded in first-order normative theories, such as Kantianism or utilitarianism, (...) that reason-responsive people can be persuaded by. This proposal can play a normative role and it is also a more promising avenue towards moral enhancement. It is more promising because such a system can be designed to take advantage of the sometimes undue trust that people put in automated technologies. We could therefore expect a well-designed moral reasoner system to be able to persuade people that may not be persuaded by similar arguments from other people. So, all things considered, there is hope in artificial intelli-gence for moral enhancement, but not in artificial intelligence that relies solely on ambient intelligence technologies. (shrink)

Several proposals for moral enhancement would use AI to augment (auxiliary enhancement) or even supplant (exhaustive enhancement) human moral reasoning or judgment. Exhaustive enhancement proposals conceive AI as some self-contained oracle whose superiority to our own moral abilities is manifest in its ability to reliably deliver the ‘right’ answers to all our moral problems. We think this is a mistaken way to frame the project, as it presumes that we already know many things that we are still in the process (...) of working out, and reflecting on this fact reveals challenges even for auxiliary proposals that eschew the oracular approach. We argue there is nonetheless a substantial role that ‘AI mentors’ could play in our moral education and training. Expanding on the idea of an AI Socratic Interlocutor, we propose a modular system of multiple AI interlocutors with their own distinct points of view reflecting their training in a diversity of concrete wisdom traditions. This approach minimizes any risk of moral disengagement, while the existence of multiple modules from a diversity of traditions ensures pluralism is preserved. We conclude with reflections on how all this relates to the broader notion of moral transcendence implicated in the project of AI moral enhancement, contending it is precisely the whole concrete socio-technical system of moral engagement that we need to model if we are to pursue moral enhancement. (shrink)

Artificial intelligence (AI) systems are increasingly being used in healthcare, thanks to the high level of performance that these systems have proven to deliver. So far, clinical applications have focused on diagnosis and on prediction of outcomes. It is less clear in what way AI can or should support complex clinical decisions that crucially depend on patient preferences. In this paper, we focus on the ethical questions arising from the design, development and deployment of AI systems to support decision-making around (...) cardiopulmonary resuscitation and the determination of a patient’s Do Not Attempt to Resuscitate status (also known as code status). The COVID-19 pandemic has made us keenly aware of the difficulties physicians encounter when they have to act quickly in stressful situations without knowing what their patient would have wanted. We discuss the results of an interview study conducted with healthcare professionals in a university hospital aimed at understanding the status quo of resuscitation decision processes while exploring a potential role for AI systems in decision-making around code status. Our data suggest that (1) current practices are fraught with challenges such as insufficient knowledge regarding patient preferences, time pressure and personal bias guiding care considerations and (2) there is considerable openness among clinicians to consider the use of AI-based decision support. We suggest a model for how AI can contribute to improve decision-making around resuscitation and propose a set of ethically relevant preconditions—conceptual, methodological and procedural—that need to be considered in further development and implementation efforts. (shrink)

For several years, scholars have (for good reason) been largely preoccupied with worries about the use of artificial intelligence and machine learning (AI/ML) tools to make decisions about us. Only recently has significant attention turned to a potentially more alarming problem: the use of AI/ML to influence our decision-making. The contexts in which we make decisions—what behavioral economists call our choice architectures—are increasingly technologically-laden. Which is to say: algorithms increasingly determine, in a wide variety of contexts, both the sets of (...) options we choose from and the way those options are framed. Moreover, artificial intelligence and machine learning (AI/ML) makes it possible for those options and their framings—the choice architectures—to be tailored to the individual chooser. They are constructed based on information collected about our individual preferences, interests, aspirations, and vulnerabilities, with the goal of influencing our decisions. At the same time, because we are habituated to these technologies we pay them little notice. They are, as philosophers of technology put it, transparent to us—effectively invisible. I argue that this invisible layer of technological mediation, which structures and influences our decision-making, renders us deeply susceptible to manipulation. Absent a guarantee that these technologies are not being used to manipulate and exploit, individuals will have little reason to trust them. (shrink)

The intention of this paper is to point to the dilemma humanity may face in light of AI advancements. The dilemma is whether to create a world with less evil or maintain the human status of moral agents. This dilemma may arise as a consequence of using automated decision-making systems for high-stakes decisions. The use of automated decision-making bears the risk of eliminating human moral agency and autonomy and reducing humans to mere moral patients. On the other hand, it also (...) has the potential to bring tremendous benefits to humanity by decreasing human-induced harm in the world. After presenting how this dilemma may arise, I explore general avenues for addressing it. I will argue that we do not have to solve this dilemma in an all-or-nothing fashion and that a more nuanced approach may be suitable. However, the main point I want to highlight is that we need to have a principled way of addressing this dilemma, which is currently missing. (shrink)

We argue that the growing prevalence of statistical machine learning in everyday decision making – from creditworthiness to police force allocation – effectively replaces many of our humdrum practical judgments and that this will eventually undermine our capacity for making such judgments. We lean on Aristotle’s famous account of how phronesis and moral virtues develop to make our case. If Aristotle is right that the habitual exercise of practical judgment allows us to incrementally hone virtues, and if AI saves us (...) time by taking over some of those practical judgments, or if its pattern recognition capacities are very good at learning that kind of behavior – we risk innovating ourselves out of moral competence with the introduction of AI. (shrink)

This chapter examines the possibility of using AI technologies to improve human moral reasoning and decision-making, especially in the context of purchasing and consumer decisions. We characterize such AI technologies as artificial ethics assistants (AEAs). We focus on just one part of the AI-aided moral improvement question: the case of the individual who wants to improve their morality, where what constitutes an improvement is evaluated by the individual’s own values. We distinguish three broad areas in which an individual might think (...) their own moral reasoning and decision-making could be improved: one’s actions, character, or other evaluable attributes fall short of one’s values and moral beliefs; one sometimes misjudges or is uncertain about what the right thing to do is in particular situations, given one’s values; one is uncertain about some fundamental moral questions or recognizes a possibility that some of one’s core moral beliefs and values are mistaken. We sketch why one might think that AI tools could be used to support moral improvement in those areas, and describe two types of assistance: preparatory assistance, including advice and training supplied in advance of moral deliberation; and on-the-spot assistance, including on-the-spot advice and facilitation of moral functioning over the course of moral deliberation. Then, we turn to some of the ethical issues that AEAs might raise, looking in particular at three under-appreciated problems posed by the use of AI for moral self-improvement: namely, reliance on sensitive moral data; the inescapability of outside influences on AEAs; and AEA usage prompting the user to adopt beliefs and make decisions without adequate reasons. (shrink)

Moral bioenhancement, nudge-designed environments, and ambient persuasive technologies may help people behave more consistently with their deeply held moral convictions. Alternatively, they may aid people in overcoming cognitive and affective limitations that prevent them from appreciating a situation’s moral dimensions. Or they may simply make it easier for them to make the morally right choice by helping them to overcome sources of weakness of will. This paper makes two assumptions. First, technologies to improve people’s moral capacities are realizable. Second, such (...) technologies will actually help people get morality right and behave more consistently with whatever the ‘real’ right thing to do turns out to be. The paper then considers whether or not humanity loses anything valuable, particularly opportunities for moral progress, when being moral is made much easier by eliminating difficult moral deliberation and internal moral struggle. Ultimately, the worry that moral struggle has value as a catalyst for moral progress is rejected. Moral progress is understood here as the discovery and application of new values or sensitization to new sources of harm. (shrink)

Recent theories on cognitive science have stressed the significance of moral intuition as a counter to and complementary part of moral reasoning in decision making. Thus, the aim of this paper is to create an integrated framework that can account for both intuitive and reflective cognitive processes, in order to explore the antecedents of ethical decision making. To do that, we build on Scholasticism, an important medieval school of thought from which descends the main pillars of the modern Catholic social (...) doctrine. Particularly, the focus will be on the scholastic concept of synderesis, which is an innate human faculty that constantly inclines decision makers toward universal moral principles. Managerial implications are discussed, stressing how a rediscovery of decision makers’ intuitive moral judgments could be relevant in the reflective thinking practice of managers’ ethical reasoning, thus saving them from rational insensitivity to ethical dilemmas. (shrink)

Systematic killing has long been associated with some of the darkest episodes in human history. Increasingly, however, it is framed as a desirable outcome in war, particularly in the context of military AI and lethal autonomy. Autonomous weapons systems, defenders argue, will surpass humans not only militarily but also morally, enabling a more precise and dispassionate mode of violence, free of the emotion and uncertainty that too often weaken compliance with the rules and standards of war. We contest this framing. (...) Drawing on the history of systematic killing, we argue that lethal autonomous weapons systems reproduce, and in some cases intensify, the moral challenges of the past. Autonomous violence incentivizes a moral devaluation of those targeted and erodes the moral agency of those who kill. Both outcomes imperil essential restraints on the use of military force. (shrink)

In this paper, we examine the qualitative moral impact of machine learning-based clinical decision support systems in the process of medical diagnosis. To date, discussions about machine learning in this context have focused on problems that can be measured and assessed quantitatively, such as by estimating the extent of potential harm or calculating incurred risks. We maintain that such discussions neglect the qualitative moral impact of these technologies. Drawing on the philosophical approaches of technomoral change and technological mediation theory, which (...) explore the interplay between technologies and morality, we present an analysis of concerns related to the adoption of machine learning-aided medical diagnosis. We analyze anticipated moral issues that machine learning systems pose for different stakeholders, such as bias and opacity in the way that models are trained to produce diagnoses, changes to how health care providers, patients, and developers understand their roles and professions, and challenges to existing forms of medical legislation. Albeit preliminary in nature, the insights offered by the technomoral change and the technological mediation approaches expand and enrich the current discussion about machine learning in diagnostic practices, bringing distinct and currently underexplored areas of concern to the forefront. These insights can contribute to a more encompassing and better informed decision-making process when adapting machine learning techniques to medical diagnosis, while acknowledging the interests of multiple stakeholders and the active role that technologies play in generating, perpetuating, and modifying ethical concerns in health care. (shrink)

Our contemporary condition is deeply infused with scientific-technological rationales. These influence and shape our ethical reasoning on war, including the moral status of civilians and the moral choices available to us. In this article, I discuss how technology shapes and directs the moral choices available to us by setting parameters for moral deliberation. I argue that technology has moral significance for just war thinking, yet this is often overlooked in attempts to assess who is liable to harm in war and (...) to what extent. This omission produces an undue deference to technological authority, reducing combatants, civilians and scenarios to data points. If we are to develop a maximally restrictive framework for harming civilians in war, which in my view should be a goal of just war thinking, then it is imperative that the scientific-technological dimension of contemporary war is given due attention. (shrink)

In the past decades, computers have become more and more involved in society by the rise of ubiquitous systems, increasing the number of interactions between humans and IT systems. At the same time, the technology itself is getting more complex, enabling devices to act in a way that previously only humans could, based on developments in the fields of both robotics and artificial intelligence. This results in a situation in which many autonomous, intelligent and context-aware systems are involved in decisions (...) that affect their environment. These relations between people, machines, and decisions can take many different forms, but thus far, a systematic account of machine-assisted moral decisions is lacking. This paper investigates the concept of machine-assisted moral decisions from the perspective of technological mediation. It is argued that modern machines do not only have morality in the sense of mediating the actions of humans, but that, by making their own decisions within their relations with humans, mediate morality itself. A classification is proposed to differentiate between four different types of moral relations. The moral aspects within the decisions these systems make are combined into three dimensions that describe the distinct characteristics of different types of moral mediation by machines. Based on this classification, specific guidelines for moral behavior can be provided for these systems. (shrink)

Robots and other smart machines are increasingly interwoven into the social fabric of our society, with the area and scope of their application continuing to expand. As we become accustomed to interacting through and with robots, we also begin to supplement or replace existing human–human interactions with human–machine interactions. This article aims to discuss the impacts of the shift from human–human interactions to human–machine interactions in one facet of our self-constitution, i.e., morality. More specifically, it sets out to explore whether (...) and how the shift to human–machine interactions can affect our moral cultivation. I shall structure the article around what Shannon Vallor calls technology-driven moral deskilling, i.e., the phenomenon of technology negatively affecting individual moral cultivation, and shall also attempt to offer a Confucian response to the problem. I first elaborate in detail Vallor’s idea of technology-driven moral deskilling. Next, I discuss three paradigms of virtue acquisition identified by Nancy E. Snow, i.e., the “folk” paradigm, the skill-and-expertise paradigm, and the Confucian paradigm, and show how the Confucian paradigm can help us to respond to technology-driven moral deskilling. Finally, I introduce the idea of Confucian rituals (li) and argue for the ritualizing of machines as an answer to technology-driven moral deskilling. (shrink)

BackgroundSocial cognition and competence are a key part of daily interactions and essential for satisfying relationships and well-being. Pediatric neurological and psychological conditions can affect social cognition and require assessment and remediation of social skills. To adequately approximate the complex and dynamic nature of real-world social interactions, innovative tools are needed. The aim of this study was to document the performance of adolescents on two versions of a serious video game presenting realistic, everyday, socio-moral conflicts, and to explore whether their (...) performance is associated with empathy or sense of presence, factors known to influence social cognition.MethodsParticipants first completed a pre-test measure of socio-moral reasoning based on three dilemmas from a previously validated computer task. Then, they either played an evaluative version or an adaptive version of a video game presenting nine social situations in which they made socio-moral decisions and provided justifications. In the evaluative version, participants’ audio justifications were recorded verbatim and coded manually to obtain a socio-moral reasoning maturity score. In the adaptive version, tailored feedback and social reinforcements were provided based on participant responses. An automatic coding algorithm developed using artificial intelligence was used to determine socio-moral maturity level in real-time and to provide a basis for the feedback and reinforcements in the game. All participants then completed a three-dilemma post-test assessment.ResultsThose who played the adaptive version showed improved SMR across the pre-test, in-game and post-test moral maturity scores, F = 9.81, pHF < 0.001, ϵ2 = 0.21, but those who played the Evaluative version did not. Socio-moral reasoning scores from both versions combined did not correlate with empathy or sense of presence during the game, though results neared significance. The study findings support preliminary validation of the game as a promising method for assessing and remediating social skills during adolescence. (shrink)

Artificial intelligence in military operations comes in two kinds. First, there is narrow or specific intelligence – the autonomous ability to identify an instance of a species of target, and to track its changes of position. Second, there is broad or general intelligence – the autonomous ability to choose a species of target, identify instances, track their movements, decide when to strike them, learn from errors, and improve initial choices. These two kinds of artificial intelligence raise ethical questions mainly because (...) of two features: the physical distance they put between the human agents deploying them and their targets, and their ability to act independently of those agents. The main ethical questions these features raise are three. First, how to maintain the traditional martial virtues of fortitude and chivalry while operating lethal weapons at a safe distance? Second, how much autonomy to grant a machine? And third, what risks to take with the possibility of technical error? This paper considers each of these questions in turn. (shrink)

This paper is based on a five-year study of the ethical principles considered by executives, middle managers, and students as appropriate guidelines for making decisions. Out of the fourteen principles surveyed, nine seem to be standards that can be applied with no further thought or research required by the decision maker. The other six principles may suggest decisions makers need clearer guidelines as to what to do or what not to do when faced with an ethical dilemma that exists outside (...) defined systems of corporate ethics. (shrink)