Tensions between respect for autonomy and paternalism loom large in Ferrario et al ’s discussion of artificial intelligence (AI)-based preference predictors.1 To be sure, their analysis (rightfully) brings out the moral matter of respecting patient preferences. My point here, however, is that their consideration of AI-based preference predictors in treatment of incapacitated patients opens more fundamental moral questions about the desirability of over-ruling considered patient preferences, not only if these are disclosed by surrogates, but possibly also in treating competent patients. (...) I do not advocate such an evolution—the moral desirability of that calls for a much broader debate, one in which the meaning of ‘doing good’ in medicine, and how this intersects with normative views on ‘the goal(s) of medicine’ would be central elements. While my aim in this piece is more modest, I nonetheless hope to approach it sideways, by indicating how the contribution by Ferrario et al reopens discussion about paternalism and the normativity of preferences in medicine. This follows from what these authors give as reason for endorsing this technology in care for incapacitated patients. Their argument for employing such tools in case of incapacitation is based on the premise that advice of surrogate decision-makers about care preferences is suboptimal because of their biased perception …. (shrink)

In their contribution ‘Ethics of the algorithmic prediction of goal of care preferences’1 Ferrario et al elaborate a from theory to practice contribution concerning the realisation of artificial intelligence (AI)-based patient preference prediction (PPP) systems. Such systems are intended to help find the treatment that the patient would have chosen in clinical situations—especially in the intensive care or emergency units—where the patient is no longer capable of making that decision herself. The authors identify several challenges that complicate their effective development, (...) application and evaluation—and offer solutions to them. One of these issues is the question of who should ultimately use said systems. While it is undisputed that clinicians should use these AI systems for their decision-making process, there is an ongoing debate about whether next of kin should use them as well. The authors advocate that ‘’access should be provided to both clinicians and loved ones with due explanations and as desired’. We will disagree with this assessment and explain in our commentary why it is important that surrogates provide their own assessments with as little external (AI) influence as possible. Why do next of kin actually participate in the process of preference finding and treatment decision-making? A key reason is that clinicians usually …. (shrink)

Nietzche claimed that once we know why to live, we’ll suffer almost any how.1 Artificial intelligence (AI) is used widely for the how, but Ferrario et al now advocate using AI for the why.2 Here, I offer my doubts on practical grounds but foremost on ethical ones. Practically, individuals already vacillate over the why, wavering with time and circumstance. That AI could provide prosthetics (or orthotics) for human agency feels unrealistic here, not least because ‘answers’ would be largely unverifiable. Ethically, (...) the concern is that AI stands to frack our humanity. We form a fragile ecosystem of ethical subjects, our responsiveness to others’ suffering, enabled by our own. To deliberate together for incapacitated others is among those solemn privileges that verify our humanity. Having AI mine these delicate pain-forests risks treating our suffering as the new oil—to be extracted and exploited, but beyond our vision and at our cost. Let’s briefly develop each idea, starting with the how/why distinction. This is palpable, even for more prosaic questions like how or why to drive. The former admits of increasingly sophisticated technological fixes and nudge; the latter often remains very particular and personal. How much greater then, the difference between …. (shrink)

Most jurisdictions require a patient to consent to any medical intervention. Clinicians ask a patient, ‘Given the pain and distress associated with our intervention and the predicted likelihood of this best-case outcome, do you want to accept the treatment?’ When a patient is incapable of deciding, clinicians may ask people who know the patient to say what the patient would decide; this is substituted judgement. In contrast, asking the same people to say how the person would make the decision is (...) substituted interests.1 The UK Mental Capacity Act 2005 uses the latter approach, and my comments appertain to that approach. When a patient lacks capacity, the question facing clinicians is, ‘Given the likely outcome of this treatment plan for the patient’s life in the long term, would the patient decide to accept the plan?’ This answer depends on the patient’s wishes, values and other factors they would use when deciding. The proposed development of an algorithm involves the following:2. (shrink)

Artificial intelligence (AI) systems are quickly gaining ground in healthcare and clinical decision-making. However, it is still unclear in what way AI can or should support decision-making that is based on incapacitated patients’ values and goals of care, which often requires input from clinicians and loved ones. Although the use of algorithms to predict patients’ most likely preferred treatment has been discussed in the medical ethics literature, no example has been realised in clinical practice. This is due, arguably, to the (...) lack of a structured approach to the epistemological, ethical and pragmatic challenges arising from the design and use of such algorithms. The present paper offers a new perspective on the problem by suggesting that preference predicting AIs be viewed as sociotechnical systems with distinctive life-cycles. We explore how both known and novel challenges map onto the different stages of development, highlighting interdisciplinary strategies for their resolution. (shrink)