An AI-based “patient preference predictor” (PPP) is a proposed method for guiding healthcare decisions for patients who lack decision-making capacity. The proposal is to use correlations between sociodemographic data and known healthcare preferences to construct a model that predicts the unknown preferences of a particular patient. In this paper, I highlight a distinction that has been largely overlooked so far in debates about the PPP–that between algorithmic prediction and decision-making–and argue that much of the recent philosophical disagreement stems from this (...) oversight. I show how three prominent objections to the PPP only challenge its use as the sole determinant of a choice, and actually support its use as a source of evidence about patient preferences to inform human decision-making. The upshot is that we should adopt the evidential conception of the PPP and shift our evaluation of this technology towards the ethics of algorithmic prediction, rather than decision-making. (shrink)

The use of machine learning, or “artificial intelligence” (AI) in medicine is widespread and growing. In this paper, I focus on a specific proposed clinical application of AI: using models to predict incapacitated patients’ treatment preferences. Drawing on results from machine learning, I argue this proposal faces a special moral problem. Machine learning researchers owe us assurance on this front before experimental research can proceed. In my conclusion I connect this concern to broader issues in AI safety.

Progressing beyond the given has been a key modern tendency. Yet modern societies are currently facing the problem of how to put limits on progress, expansion, and growth, live within them, and preserve (rather than transcend) the present. Drawing on economic sociology scholarship on valuation and morality in economic life, this article develops and applies the term economization to analyze the enactment of limits on progress. The question of end-of-life care—when to stop medical efforts to prolong life, postpone death, and (...) advance the scientific frontier—serves as an illustrative empirical case that sheds light on limit-setting in general. My analysis of this case combines historical, ethnographic, and in-depth interview data on US palliative care clinicians, who specialize in making life-and-death decisions in acute care hospitals. (shrink)

What proper role should considerations of risk, particularly to research subjects, play when it comes to conducting research on human enhancement in the military context? We introduce the currently visible military enhancement techniques (1) and the standard discussion of risk for these (2), in particular what we refer to as the ‘Assumption’, which states that the demands for risk-avoidance are higher for enhancement than for therapy. We challenge the Assumption through the introduction of three categories of enhancements (3): therapeutic, preventive, (...) and pure enhancements. This demands a revision of the Assumption (4), alongside which we propose some further general principles bearing on how to balance risks and benefits in the context of military enhancement research. We identify a particular type of therapeutic enhancements as providing a more responsible path to human trials of the relevant interventions than pure enhancement applications. Finally, we discuss some possible objections to our line of thought (5). While acknowledging their potential insights, we ultimately find them to be unpersuasive, at least provided that our proposal is understood as fully non-coercive towards the candidates for such therapeutic enhancement trials. (shrink)

The standard approach to treatment decision making for incapacitated patients often fails to provide treatment consistent with the patient’s preferences and values and places significant stress on surrogate decision makers. These shortcomings provide compelling reason to search for methods to improve current practice. Shared decision making between surrogates and clinicians has important advantages, but it does not provide a way to determine patients’ treatment preferences. Hence, shared decision making leaves families with the stressful challenge of identifying the patient’s preferred treatment (...) option. To address this concern, the present paper proposes to incorporate the use of a “Patient Preference Predictor” (PPP) into the shared decision-making process between surrogates and clinicians. A PPP would predict which treatment option a given incapacitated patient would most likely prefer, based on the individual’s characteristics and information on what treatment preferences are correlated with these characteristics. Use of a PPP is likely to increase the chances that incapacitated patients are treated consistent with their preferences and values and might reduce the stress and burden on their surrogates. Including a PPP in the shared decision-making process therefore has the potential to realize important ethical goals for making treatment decisions for incapacitated patients. The present paper justifies this approach on conceptual and normative grounds. (shrink)

Monitoring and assistive technologies (MATs) are being used more frequently in healthcare. A central ethical concern is the compatibility of these systems with the moral preferences of their users—an issue especially relevant to participatory approaches within the ethics-by-design debate. However, users’ incapacity to communicate preferences or to participate in design processes, e.g., due to dementia, presents a hurdle for participatory ethics-by-design approaches. In this paper, we explore the question of how the value preferences of users in the field of dementia (...) care can be integrated into AI-based MATs. First, we briefly introduce different ethics-by-design approaches and explain the challenges they face in dementia care. Next, we introduce a novel approach for addressing these challenges. Through a qualitative content analysis of interviews with persons with dementia and their family caregivers (n = 27), we identified multiple ideal–typical value preference profiles. We then applied these profiles in a computer simulation, by which we also introduce the concept of ethical compliance quantification to scale the moral preferences of the interviewees for the simulation. Finally, we discuss study results, the advantages of using computer simulations, and general study limitations before drawing conclusions for further research. The study contributes to the ongoing debate on participatory ethics-by-design by defining on the basis of empirical data ideal–typical value preference frameworks that can be used to guide MAT actions and their outcomes. Going forward, MAT end-users with dementia and other communication-impaired persons could be enabled to choose a value profile that best fits their moral preferences. (shrink)

When making substituted judgments for incapacitated patients, surrogates often struggle to guess what the patient would want if they had capacity. Surrogates may also agonize over having the (sole) responsibility of making such a determination. To address such concerns, a Patient Preference Predictor (PPP) has been proposed that would use an algorithm to infer the treatment preferences of individual patients from population-level data about the known preferences of people with similar demographic characteristics. However, critics have suggested that even if such (...) a PPP were more accurate, on average, than human surrogates in identifying patient preferences, the proposed algorithm would nevertheless fail to respect the patient’s (former) autonomy since it draws on the ‘wrong’ kind of data: namely, data that are not specific to the individual patient and which therefore may not reflect their actual values, or their reasons for having the preferences they do. Taking such criticisms on board, we here propose a new approach: the Personalized Patient Preference Predictor (P4). The P4 is based on recent advances in machine learning, which allow technologies including large language models to be more cheaply and efficiently ‘fine-tuned’ on person-specific data. The P4, unlike the PPP, would be able to infer an individual patient’s preferences from material (e.g., prior treatment decisions) that is in fact specific to them. Thus, we argue, in addition to being potentially more accurate at the individual level than the previously proposed PPP, the predictions of a P4 would also more directly reflect each patient’s own reasons and values. In this article, we review recent discoveries in artificial intelligence research that suggest a P4 is technically feasible, and argue that, if it is developed and appropriately deployed, it should assuage some of the main autonomy-based concerns of critics of the original PPP. We then consider various objections to our proposal and offer some tentative replies. (shrink)

BackgroundHealthcare providers have to make ethically complex clinical decisions which may be a source of stress. Researchers have recently introduced Artificial Intelligence (AI)-based applications to assist in clinical ethical decision-making. However, the use of such tools is controversial. This review aims to provide a comprehensive overview of the reasons given in the academic literature for and against their use.MethodsPubMed, Web of Science, Philpapers.org and Google Scholar were searched for all relevant publications. The resulting set of publications was title and abstract (...) screened according to defined inclusion and exclusion criteria, resulting in 44 papers whose full texts were analysed using the Kuckartz method of qualitative text analysis.ResultsArtificial Intelligence might increase patient autonomy by improving the accuracy of predictions and allowing patients to receive their preferred treatment. It is thought to increase beneficence by providing reliable information, thereby, supporting surrogate decision-making. Some authors fear that reducing ethical decision-making to statistical correlations may limit autonomy. Others argue that AI may not be able to replicate the process of ethical deliberation because it lacks human characteristics. Concerns have been raised about issues of justice, as AI may replicate existing biases in the decision-making process.ConclusionsThe prospective benefits of using AI in clinical ethical decision-making are manifold, but its development and use should be undertaken carefully to avoid ethical pitfalls. Several issues that are central to the discussion of Clinical Decision Support Systems, such as justice, explicability or human–machine interaction, have been neglected in the debate on AI for clinical ethics so far.Trial registrationThis review is registered at Open Science Framework (https://osf.io/wvcs9). (shrink)

Artificial intelligence (AI) systems are quickly gaining ground in healthcare and clinical decision-making. However, it is still unclear in what way AI can or should support decision-making that is based on incapacitated patients’ values and goals of care, which often requires input from clinicians and loved ones. Although the use of algorithms to predict patients’ most likely preferred treatment has been discussed in the medical ethics literature, no example has been realised in clinical practice. This is due, arguably, to the (...) lack of a structured approach to the epistemological, ethical and pragmatic challenges arising from the design and use of such algorithms. The present paper offers a new perspective on the problem by suggesting that preference predicting AIs be viewed as sociotechnical systems with distinctive life-cycles. We explore how both known and novel challenges map onto the different stages of development, highlighting interdisciplinary strategies for their resolution. (shrink)

Patient preference predictors (PPPs) promise to provide medical professionals with a new solution to the problem of making treatment decisions on behalf of incapacitated patients. I show that the use of PPPs faces a version of a normative problem familiar from legal scholarship: the problem of naked statistical evidence. I sketch two sorts of possible reply, vindicating and debunking, and suggest that our reply to the problem in the one domain ought to mirror our reply in the other. The conclusion (...) is thus conditional: if we think the problem of naked statistical evidence is a serious problem in the legal domain, then we should be concerned about the symmetrical problem for PPPs. (shrink)

Background: Current practice frequently fails to provide care consistent with the preferences of decisionally-incapacitated patients. It also imposes significant emotional burden on their surrogates. Algorithmic-based patient preference predictors (PPPs) have been proposed as a possible way to address these two concerns. While previous research found that patients strongly support the use of PPPs, the views of surrogates are unknown. The present study thus assessed the views of experienced surrogates regarding the possible use of PPPs as a means to help make (...) treatment decisions for decisionally-incapacitated patients. -/- Methods: This qualitative study used semi-structured interviews to determine the views of experienced surrogates [n=26] who were identified from two academic medical centers and two community hospitals. The primary outcomes were respondents’ overall level of support for the idea of using PPPs and the themes related to their views on how a PPP should be used, if at all, in practice. -/- Results: Overall, 21 participants supported the idea of using PPPs. The remaining five indicated that they would not use a PPP because they made decisions based on the patient’s best interests, not based on substituted judgment. Major themes which emerged were that surrogates, not the patient’s preferences, should determine how treatment decisions are made, and concern that PPPs might be used to deny expensive care or be biased against minority groups. -/- Conclusions: Surrogates, like patients, strongly support the idea of using PPPs to help make treatment decisions for decisionally-incapacitated patients. These findings provide support for developing a PPP and assessing it in practice. At the same time, patients and surrogates disagree over whose preferences should determine how treatment decisions are made, including whether to use a PPP. These findings reveal a fundamental disagreement regarding the guiding principles for surrogate decision-making. Future research is needed to assess this disagreement and consider ways to address it. (shrink)

A significant proportion of elderly and psychiatric patients do not have the capacity to make health care decisions. We suggest that machine learning technologies could be harnessed to integrate data mined from electronic health records (EHRs) and social media in order to estimate the confidence of the prediction that a patient would consent to a given treatment. We call this process, which takes data about patients as input and derives a confidence estimate for a particular patient’s predicted health care-related decision (...) as an output, the autonomy algorithm. We suggest that the proposed algorithm would result in more accurate predictions than existing methods, which are resource intensive and consider only small patient cohorts. This algorithm could become a valuable tool in medical decision-making processes, augmenting the capacity of all people to make health care decisions in difficult situations. (shrink)

The patient preference predictor is a proposed computer-based algorithm that would predict the treatment preferences of decisionally incapacitated patients. Incorporation of a PPP into the decision-making process has the potential to improve implementation of the substituted judgement standard by providing more accurate predictions of patients’ treatment preferences than reliance on surrogates alone. Yet, critics argue that methods for making treatment decisions for incapacitated patients should be judged on a number of factors beyond simply providing them with the treatments they would (...) have chosen for themselves. These factors include the extent to which the decision-making process recognises patients’ freedom to choose and relies on evidence the patient themselves would take into account when making treatment decisions. These critics conclude that use of a PPP should be rejected on the grounds that it is inconsistent with these factors, especially as they relate to proper respect for patient autonomy. In this paper, we review and evaluate these criticisms. We argue that they do not provide reason to reject use of a PPP, thus supporting efforts to develop a full-scale PPP and to evaluate it in practice. There are no data in this work. (shrink)

Since surrogates are not good at predicting patient preferences, and since these decisions can cause surrogates distress, some have claimed we need an alternative way to make decisions for incapaci...

The Patient Preference Predictor (PPP) proposal places a high priority on the accuracy of predicting patients’ preferences and finds the performance of surrogates inadequate. However, the quest to develop a highly accurate, individualized statistical model has significant obstacles. First, it will be impossible to validate the PPP beyond the limit imposed by 60%–80% reliability of people’s preferences for future medical decisions—a figure no better than the known average accuracy of surrogates. Second, evidence supports the view that a sizable minority of (...) persons may not even have preferences to predict. Third, many, perhaps most, people express their autonomy just as much by entrusting their loved ones to exercise their judgment than by desiring to specifically control future decisions. Surrogate decision making faces none of these issues and, in fact, it may be more efficient, accurate, and authoritative than is commonly assumed. (shrink)

The concept of a Patient Preference Predictor—an algorithm that supplements or replaces the process of surrogate decision-making for incapacitated patients—was first suggested a decade ago (Rid and...