Our ability to locate moral responsibility is often thought to be a necessary condition for conducting morally permissible medical practice, engaging in a just war, and other high-stakes endeavors. Yet, with increasing reliance upon artificially intelligent systems, we may be facing a wideningresponsibility gap, which, some argue, cannot be bridged by traditional concepts of responsibility. How then, if at all, can we make use of crucial emerging technologies? According to Colin Allen and Wendell Wallach, the advent of so-called ‘artificial moral (...) agents’ (AMAs) is inevitable. Still, this notion may seem to push back the problem, leaving those who have an interest in developing autonomous technology with a dilemma. We may need to scale-back our efforts at deploying AMAs (or at least maintain human oversight); otherwise, we must rapidly and drastically update our moral and legal norms in a way that ensures responsibility for potentially avoidable harms. This paper invokes contemporary accounts of responsibility in order to show how artificially intelligent systems might be held responsible. Although many theorists are concerned enough to develop artificial conceptions ofagencyor to exploit our present inability to regulate valuable innovations, the proposal here highlights the importance of—and outlines a plausible foundation for—a workable notion ofartificial moral responsibility. (shrink)

Machine learning-driven clinical decision support systems (ML-CDSSs) seem impressively promising for future routine and emergency care. However, reflection on their clinical implementation reveals a wide array of ethical challenges. The preferences, concerns and expectations of professional stakeholders remain largely unexplored. Empirical research, however, may help to clarify the conceptual debate and its aspects in terms of their relevance for clinical practice. This study explores, from an ethical point of view, future healthcare professionals’ attitudes to potential changes of responsibility and decision-making (...) authority when using ML-CDSS. Twenty-seven semistructured interviews were conducted with German medical students and nursing trainees. The data were analysed based on qualitative content analysis according to Kuckartz. Interviewees’ reflections are presented under three themes the interviewees describe as closely related: (self-)attribution of responsibility, decision-making authority and need of (professional) experience. The results illustrate the conceptual interconnectedness of professional responsibility and its structural and epistemic preconditions to be able to fulfil clinicians’ responsibility in a meaningful manner. The study also sheds light on the four relata of responsibility understood as a relational concept. The article closes with concrete suggestions for the ethically sound clinical implementation of ML-CDSS. (shrink)

Artificial intelligence (AI) based clinical decision support systems (CDSS) are becoming ever more widespread in healthcare and could play an important role in diagnostic and treatment processes. For this reason, AI‐based CDSS has an impact on the doctor–patient relationship, shaping their decisions with its suggestions. We may be on the verge of a paradigm shift, where the doctor–patient relationship is no longer a dual relationship, but a triad. This paper analyses the role of AI‐based CDSS for shared decision‐making to better (...) comprehend its promises and associated ethical issues. Moreover, it investigates how certain AI implementations may instead foster the inappropriate paradigm of paternalism. Understanding how AI relates to doctors and influences doctor–patient communication is essential to promote more ethical medical practice. Both doctors' and patients' autonomy need to be considered in the light of AI. (shrink)

With automation of routine decisions coupled with more intricate and complex information architecture operating this automation, concerns are increasing about the trustworthiness of these systems. These concerns are exacerbated by a class of artificial intelligence that uses deep learning, an algorithmic system of deep neural networks, which on the whole remain opaque or hidden from human comprehension. This situation is commonly referred to as the black box problem in AI. Without understanding how AI reaches its conclusions, it is an open (...) question to what extent we can trust these systems. The question of trust becomes more urgent as we delegate more and more decision-making to and increasingly rely on AI to safeguard significant human goods, such as security, healthcare, and safety. Models that “open the black box” by making the non-linear and complex decision process understandable by human observers are promising solutions to the black box problem in AI but are limited, at least in their current state, in their ability to make these processes less opaque to most observers. A philosophical analysis of trust will show why transparency is a necessary condition for trust and eventually for judging AI to be trustworthy. A more fruitful route for establishing trust in AI is to acknowledge that AI is situated within a socio-technical system that mediates trust, and by increasing the trustworthiness of these systems, we thereby increase trust in AI. (shrink)

Trust is one of the big buzzwords in debates about the shaping of society, democracy, and emerging technologies. For example, one prominent idea put forward by the High‐Level Expert Group on Artificial Intelligence appointed by the European Commission is that artificial intelligence should be trustworthy. In this essay, we explore the notion of trust and argue that both proponents and critics of trustworthy AI have flawed pictures of the nature of trust. We develop an approach to understanding trust in AI (...) that does not conceive of trust merely as an accelerator for societal acceptance of AI technologies. Instead, we argue, trust is granted through leaps of faith. For this reason, trust remains precarious, fragile, and resistant to promotion through formulaic approaches. We also highlight the significance of distrust in societal deliberation, as it is relevant to trust in various and intricate ways. Among the fruitful aspects of distrust is that it enables individuals to forgo technology if desired, to constrain its power, and to exercise meaningful human control. (shrink)

Research on the ethics of algorithms has grown substantially over the past decade. Alongside the exponential development and application of machine learning algorithms, new ethical problems and solutions relating to their ubiquitous use in society have been proposed. This article builds on a review of the ethics of algorithms published in 2016, 2016). The goals are to contribute to the debate on the identification and analysis of the ethical implications of algorithms, to provide an updated analysis of epistemic and normative (...) concerns, and to offer actionable guidance for the governance of the design, development and deployment of algorithms. (shrink)

Making good decisions in extremely complex and difficult processes and situations has always been both a key task as well as a challenge in the clinic and has led to a large amount of clinical, legal and ethical routines, protocols and reflections in order to guarantee fair, participatory and up-to-date pathways for clinical decision-making. Nevertheless, the complexity of processes and physical phenomena, time as well as economic constraints and not least further endeavours as well as achievements in medicine and healthcare (...) continuously raise the need to evaluate and to improve clinical decision-making. This article scrutinises if and how clinical decision-making processes are challenged by the rise of so-called artificial intelligence-driven decision support systems (AI-DSS). In a first step, this article analyses how the rise of AI-DSS will affect and transform the modes of interaction between different agents in the clinic. In a second step, we point out how these changing modes of interaction also imply shifts in the conditions of trustworthiness, epistemic challenges regarding transparency, the underlying normative concepts of agency and its embedding into concrete contexts of deployment and, finally, the consequences for (possible) ascriptions of responsibility. Third, we draw first conclusions for further steps regarding a ‘meaningful human control’ of clinical AI-DSS. (shrink)

Artificial intelligence (AI) is expected to revolutionise the practice of medicine. Recent advancements in the field of deep learning have demonstrated success in variety of clinical tasks: detecting diabetic retinopathy from images, predicting hospital readmissions, aiding in the discovery of new drugs, etc. AI’s progress in medicine, however, has led to concerns regarding the potential effects of this technology on relationships of trust in clinical practice. In this paper, I will argue that there is merit to these concerns, since AI (...) systems can be relied on, and are capable of reliability, but cannot be trusted, and are not capable of trustworthiness. Insofar as patients are required to rely on AI systems for their medical decision-making, there is potential for this to produce a deficit of trust in relationships in clinical practice. (shrink)

We conducted focus groups to assess patient attitudes toward research on medical practices in the context of usual care. We found that patients focus on the implications of this research for their relationship with and trust in their physicians. Patients view research on medical practices as separate from usual care, demanding dissemination of information and in most cases, individual consent. Patients expect information about this research to come through their physician, whom they rely on to identify and filter associated risks. (...) In general, patients support this research, but worry that participation in research involving randomization may undermine individualized care that acknowledges their unique medical histories. These findings suggest the need for public education on variation in practice among physicians and the need for a collaborative approach to the governance of research on medical practices that addresses core values of trust, transparency, and partnership. (shrink)

The notion of “responsibility gap” with artificial intelligence (AI) was originally introduced in the philosophical debate to indicate the concern that “learning automata” may make more difficult or impossible to attribute moral culpability to persons for untoward events. Building on literature in moral and legal philosophy, and ethics of technology, the paper proposes a broader and more comprehensive analysis of the responsibility gap. The responsibility gap, it is argued, is not one problem but a set of at least four interconnected (...) problems – gaps in culpability, moral and public accountability, active responsibility—caused by different sources, some technical, other organisational, legal, ethical, and societal. Responsibility gaps may also happen with non-learning systems. The paper clarifies which aspect of AI may cause which gap in which form of responsibility, and why each of these gaps matter. It proposes a critical review of partial and non-satisfactory attempts to address the responsibility gap: those which present it as a new and intractable problem (“fatalism”), those which dismiss it as a false problem (“deflationism”), and those which reduce it to only one of its dimensions or sources and/or present it as a problem that can be solved by simply introducing new technical and/or legal tools (“solutionism”). The paper also outlines a more comprehensive approach to address the responsibility gaps with AI in their entirety, based on the idea of designing socio-technical systems for “meaningful human control", that is systems aligned with the relevant human reasons and capacities. (shrink)

Research on the ethics of algorithms has grown substantially over the past decade. Alongside the exponential development and application of machine learning algorithms, new ethical problems and solutions relating to their ubiquitous use in society have been proposed. This article builds on a review of the ethics of algorithms published in 2016, 2016). The goals are to contribute to the debate on the identification and analysis of the ethical implications of algorithms, to provide an updated analysis of epistemic and normative (...) concerns, and to offer actionable guidance for the governance of the design, development and deployment of algorithms. (shrink)

Zusammenfassung Die rasanten Entwicklungen im Bereich der Künstlichen Intelligenz und Robotik stellen nicht nur die Ethik, sondern auch das Recht vor neue Herausforderungen, gerade im Bereich der Medizin und Pflege. Grundsätzlich hat der Einsatz von KI dabei das Potenzial, sowohl die Heilbehandlungen als auch den adäquaten Umgang im Rahmen der Pflege zu erleichtern, wenn nicht sogar zu verbessern. Verwaltungsaufgaben, die Überwachung von Vitalfunktionen und deren Parameter sowie die Untersuchung von Gewebeproben etwa könnten autonom ablaufen. In Diagnostik und Therapie können Systeme (...) die behandelnde Ärztin unterstützen. Intelligente Betten ermöglichen eine Frühmobilisierung der Patient:innen bei gleichzeitig geringerem Personalaufwand. Gleichwohl birgt der Einsatz der Systeme rechtliche Herausforderungen. So besteht das Risiko einer Verletzung der beteiligten Personen. Im Gegensatz zu herkömmlichen Technologien „leidet“ KI unter der „Black-Box-Problematik“: Die von den Systemen generierten Ergebnisse sind nicht mehr vollständig vorhersehbar und nachvollziehbar. Der Einsatz birgt unbekannte und unkalkulierbare Risiken, was sich insbesondere auf die zivilrechtliche Haftung und strafrechtliche Verantwortung auswirkt. Wem die Entscheidungen der Systeme normativ zuzurechnen sind, ist eine Kernfrage des juristischen Diskurses. Die aus praktikabilitätsgründen naheliegende Wahl, dem letztentscheidenden Menschen das Verhalten eines KI-Systems zuzurechnen, überzeugt nicht in allen Fällen, sondern degradiert ihn häufig zum symbolischen Haftungsknecht und legt ihm einseitig die Risiken der Technologien auf. Weiterhin ergeben sich im Bereich der Medizin und Pflege Fragen hinsichtlich der Zulassung von KI-Systemen, da die Maschinen während der Nutzung weiterlernen und so ihren strukturellen Aufbau kontinuierlich verändern. Es ist daher angebracht, sich frühzeitig mit dem Konfliktpotential aus ethischer und rechtlicher Sicht auseinander zu setzen, um einer potenziellen gesellschaftlichen Angst vor derartigen Systemen vorzubeugen und einen praxisgerechten Handlungsrahmen zu schaffen. (shrink)

ZusammenfassungSystemmedizinische Ansätze zeichnen sich durch die Integration großer Datenmengen aus vielfältigen Datenquellen aus und führen systembiologische und medizinische Forschungsansätze mit informationswissenschaftlichen Methoden und prädiktiven Verfahren mathematischer Modellierung zusammen. Hieraus resultiert eine enge Kooperation von Ärzten und Naturwissenschaftlern, wobei insbesondere die Expertise nicht-ärztlicher Forscher zunehmend an Bedeutung für die Datenaufbereitung und -interpretation gewinnt. Aus ethischer Perspektive wirft diese Entwicklung Fragen nach der konkreten Gestaltung einer systemmedizinischen Zusammenarbeit sowie möglichen Rollenveränderungen und neuen Verantwortungszuschreibungen an Ärzte und nicht-ärztliche Forscher auf. Um diese Fragen (...) mit Blick auf die Erfahrungen und Perspektiven der beteiligten Akteursgruppen zu beleuchten, führten wir eine qualitative Interviewstudie mit Ärzten und nicht-ärztlichen Forschern aus unterschiedlichen systemmedizinischen Kontexten durch. Aus dem Interviewmaterial ließen sich zwei Konzeptionen von Systemmedizin rekonstruieren. Die erste ist durch eine eindeutige arbeitsteilige Rollentrennung zwischen Ärzten und Forschern charakterisiert: Der Forscher fungiert als Dienstleister, der Arzt als translationaler, interdisziplinär ausgerichteter Mediziner. Die zweite zeichnet sich durch eine weitreichende Aufhebung der Rollentrennung von Ärzten und Forschern aus: Die Berufsgruppen agieren als interdisziplinäres Team mit einer engen wechselseitigen inhaltlichen und methodischen Zusammenarbeit der Akteure. In beiden Konzeptionen werden Rollenkonflikte von Ärzten und Forschern deutlich, die insbesondere auf die Diskrepanz zwischen dem Arzt- und Forscherethos und die mit ihnen je verknüpften spezifischen Handlungsnormen und Ziele zurückzuführen sind. Ferner besteht mit Blick auf die dem Arzt und Forscher jeweils zukommende Verantwortung gegenüber Patienten vielfältiger normativer Klärungsbedarf, insbesondere hinsichtlich der Frage, welche Verantwortung den Akteursgruppen gerechtfertigter Weise zugeschrieben werden kann. Diesbezüglich erscheint eine Differenzierung von versorgungsnäheren und grundlagenforschungsorientierten Arbeitsfeldern angeraten. (shrink)

In his contribution, ‘Represent me: please! Towards an ethics of digital twins in medicine’1, Braun shows that there is a fundamental ambivalence inherent in digital twins: they can either open up new freedoms for the simulated persons, or, conversely, endanger and restrict their freedom. To prevent digital twins from restricting people’s freedom, Braun suggests a strong focus on control. Braun’s focus on control is insufficient, I will argue, because his idea of control works only retroactively: it can only react to (...) restrictions of freedom that have already occurred, but it cannot prevent them, nor can it restore any lost freedom. Control is therefore insufficient to defend people’s freedom. To prove my claim, I will first use an everyday example to illustrate the fundamental ambivalence inherent in digital twins. Second, I will show that digital twins endanger people’s freedom in a hermeneutic way. Finally, I will demonstrate why controllability cannot prevent this hermeneutic endangerment. I understand digital twins, like Braun, as simulation of a person, as their dynamic and precise effigy. Simulated twins use machine-learning and artificial intelligence capabilities to make predictive analyses about the simulated person and the future development of her health. Imagine: You are eating dinner. Meanwhile, your partner lectures you about the calorie density and nutrient composition of your food, warns you that your portion size exceeds your daily calorie requirement by x calories. It calculates how your dinner will negatively impact your body mass index and physical fitness, and predicts what the consequences will be for your overall health. Based on these data and predictions, it creates an individualised diet plan for you—and at the same time shows you by how much you could improve your vital values by sticking to that plan. This example helps to illustrate the fundamental …. (shrink)