The use of machine learning, or “artificial intelligence” (AI) in medicine is widespread and growing. In this paper, I focus on a specific proposed clinical application of AI: using models to predict incapacitated patients’ treatment preferences. Drawing on results from machine learning, I argue this proposal faces a special moral problem. Machine learning researchers owe us assurance on this front before experimental research can proceed. In my conclusion I connect this concern to broader issues in AI safety.

How shall we decide for others who cannot decide for themselves? And who—or what, in the case of artificial intelligence — should make the decision? The present issue of the journal tackles several interrelated topics, many of them having to do with surrogate decision making. For example, the feature article by Jardas et al 1 explores the potential use of artificial intelligence to predict incapacitated patients’ likely treatment preferences based on their sociodemographic characteristics, raising questions about the means by which (...) we come to decide for others. And a clinical ethics round table led by Wilkinson and Pillay2 examines the case of a premature baby on life support whose primary surrogate is herself incapacitated. Together, these examples force us to think more deeply about the meaning and significance of taken-for-granted concepts: respect for autonomy, substituted judgement, best interests. We’ll consider the baby first and then turn to AI. “Baby T” is a critically ill newborn delivered prematurely by emergency caesarean section. The mother had entered into a surrogacy arrangement with a same-sex male couple, the intended parents, who were to take over the baby’s care after birth—just as soon as a formal parental order could be obtained through the court. Until then, the birth mother, who had used her own eggs to conceive Baby T along with sperm from an unidentified donor, would have legal and ethical responsibility to decide about the baby’s care. Unfortunately, she too was in critical condition, having fallen unconscious prior to delivery due to a sudden brain haemorrhage. She remained unconscious, and thus incapacitated, during a crucial period in which time-sensitive decisions about Baby T’s care needed to be made, including whether to continue life support. Given the mother’s incapacity, who should …. (shrink)

What proper role should considerations of risk, particularly to research subjects, play when it comes to conducting research on human enhancement in the military context? We introduce the currently visible military enhancement techniques (1) and the standard discussion of risk for these (2), in particular what we refer to as the ‘Assumption’, which states that the demands for risk-avoidance are higher for enhancement than for therapy. We challenge the Assumption through the introduction of three categories of enhancements (3): therapeutic, preventive, (...) and pure enhancements. This demands a revision of the Assumption (4), alongside which we propose some further general principles bearing on how to balance risks and benefits in the context of military enhancement research. We identify a particular type of therapeutic enhancements as providing a more responsible path to human trials of the relevant interventions than pure enhancement applications. Finally, we discuss some possible objections to our line of thought (5). While acknowledging their potential insights, we ultimately find them to be unpersuasive, at least provided that our proposal is understood as fully non-coercive towards the candidates for such therapeutic enhancement trials. (shrink)

Like many, I find the idea of relying on patient preference predictors in life-or-death cases ethically troubling. As part of his stimulating discussion, Sharadin1 diagnoses such unease as a worry that using PPPs disrespects patients’ autonomy, by treating their most intimate and significant desires as if they were caused by their demographic traits. I agree entirely with Sharadin’s ‘debunking’ response to this concern: we can use statistical correlations to predict others’ preferences without thereby assuming any causal claim. However, I suspect (...) that, for at least some of us, our unease about PPPs stems from a different kind of ‘autonomy’ concern. In this commentary, then, I will explore this concern, and show how it relates to Sharadin’s work. Very many of our preferences are caused, ultimately, by facts which are outside our control, such as our demographic features. However, I suggest that we can still act autonomously on the basis of such preferences, when they are preferences which we endorse. Imagine, for example, that Jane has grown up in a church-growing environment, which has shaped many of her preferences, …. (shrink)

This book sheds light on various ethical challenges military and humanitarian health care personnel face while working in adverse conditions. Contexts of armed conflict, hybrid wars or other forms of violence short of war, as well as natural disasters, all have in common that ordinary circumstances can no longer be taken for granted. Hence, the provision of health care has to adapt, for example, to a different level of risk, to scarce resources, or uncommon approaches due to external incentives or (...) requirements. This affects the practice of health care as well as its ethics. This book offers a panoramic overview on various challenges healthcare faces in extraordinary situations and provides new insights from practitioners’ as well as from academic scholars’ perspectives. (shrink)

Except in extraordinary circumstances, patients' clinical care should reflect their preferences. Incapacitated patients cannot report their preferences. This is a problem. Extant solutions to the problem are inadequate: surrogates are unreliable, and advance directives are uncommon. In response, some authors have suggested developing algorithmic "patient preference predictors" (PPPs) to inform care for incapacitated patients. In a recent paper, Earp et al. propose a new twist on PPPs. Earp et al. suggest we personalize PPPs using modern machine learning (ML) techniques. In (...) this paper, I argue that, contrary to Earp et al.'s claim, personalized patient preference predictors are neither technically feasible nor ethically desirable. (shrink)

The patient preference predictor is a proposed computer-based algorithm that would predict the treatment preferences of decisionally incapacitated patients. Incorporation of a PPP into the decision-making process has the potential to improve implementation of the substituted judgement standard by providing more accurate predictions of patients’ treatment preferences than reliance on surrogates alone. Yet, critics argue that methods for making treatment decisions for incapacitated patients should be judged on a number of factors beyond simply providing them with the treatments they would (...) have chosen for themselves. These factors include the extent to which the decision-making process recognises patients’ freedom to choose and relies on evidence the patient themselves would take into account when making treatment decisions. These critics conclude that use of a PPP should be rejected on the grounds that it is inconsistent with these factors, especially as they relate to proper respect for patient autonomy. In this paper, we review and evaluate these criticisms. We argue that they do not provide reason to reject use of a PPP, thus supporting efforts to develop a full-scale PPP and to evaluate it in practice. There are no data in this work. (shrink)

When a patient lacks sufficient capacity to make a certain treatment decision, whether because of deficits in their ability to make a judgement that reflects their values or to make a decision that reflects their judgement or both, the decision must be made by a surrogate. Often the best way to respect the patient’s autonomy, in such cases, is for the surrogate to make a ‘substituted’ judgement on behalf of the patient, which is the decision that best reflects the patient’s (...) values and identity-infused beliefs as they were when the patient last possessed capacity. It is by now well-known that there is troubling evidence that surrogates do only slightly better than chance at making accurate substituted judgements. The use of a patient preference predictor, an algorithm that predicts patient preferences based on characteristics like the patient’s age, gender, socioeconomic status and education level, could help. Rid and Wendler have done important work making a moral case for using the PPP.1 If it were shown to enhance accuracy, it is not at all obvious what objections there could be to its use. Yet several subtle criticisms of the PPP have emerged in the literature. In their feature article, Jardas, Wasserman and Wendler select six of the most prominent objections to the PPP in the literature and defend the PPP against them.2 Along the way they make significant contributions to broader issues concerning the proper uses of statistical evidence and limits on the kinds of preferences there can be reasons of autonomy to respect. Since I found the authors’ answers to the final two objections—that the PPP relies on naked statistical evidence and that it uses non-endorsed reasons—to be especially interesting and provocative, I will examine those answers and raise a few questions about …. (shrink)

Artificial intelligence (AI) systems are quickly gaining ground in healthcare and clinical decision-making. However, it is still unclear in what way AI can or should support decision-making that is based on incapacitated patients’ values and goals of care, which often requires input from clinicians and loved ones. Although the use of algorithms to predict patients’ most likely preferred treatment has been discussed in the medical ethics literature, no example has been realised in clinical practice. This is due, arguably, to the (...) lack of a structured approach to the epistemological, ethical and pragmatic challenges arising from the design and use of such algorithms. The present paper offers a new perspective on the problem by suggesting that preference predicting AIs be viewed as sociotechnical systems with distinctive life-cycles. We explore how both known and novel challenges map onto the different stages of development, highlighting interdisciplinary strategies for their resolution. (shrink)

In their contribution ‘Ethics of the algorithmic prediction of goal of care preferences’1 Ferrario et al elaborate a from theory to practice contribution concerning the realisation of artificial intelligence (AI)-based patient preference prediction (PPP) systems. Such systems are intended to help find the treatment that the patient would have chosen in clinical situations—especially in the intensive care or emergency units—where the patient is no longer capable of making that decision herself. The authors identify several challenges that complicate their effective development, (...) application and evaluation—and offer solutions to them. One of these issues is the question of who should ultimately use said systems. While it is undisputed that clinicians should use these AI systems for their decision-making process, there is an ongoing debate about whether next of kin should use them as well. The authors advocate that ‘’access should be provided to both clinicians and loved ones with due explanations and as desired’. We will disagree with this assessment and explain in our commentary why it is important that surrogates provide their own assessments with as little external (AI) influence as possible. Why do next of kin actually participate in the process of preference finding and treatment decision-making? A key reason is that clinicians usually …. (shrink)

New informatics technologies are becoming increasingly important in medical practice. Machine learning (ML) and deep learning (DL) systems enable data analysis and the formulation of medical recomm...