In response to Ferrario _et al_’s 1 work entitled ‘Ethics of the algorithmic prediction of goal of care preferences: from theory to practice’, we would like to point out an area of concern: the risk of artificial intelligence (AI) paternalism in their proposed framework. Accordingly, in this commentary, we underscore the importance of the implementation of safeguards for AI algorithms before they are deployed in clinical practice. The goal of documenting a living will and advanced directives is to convey personal (...) preferences regarding the acceptance of therapies, including life support, for future use in case of losing one’s capacity. This is standard practice in the care of incapacitated critically ill patients as it is considered to extend the individual’s autonomy. Notably, most of the documents that intensivists encounter in clinical practice are written in a generic fashion and lack context. This problem usually leads to the reliance on... (shrink)

The contention that medical artificial intelligence (AI) should be ‘explainable’ is widespread in contemporary philosophy and in legal and best practice documents. Yet critics argue that ‘explainability’ is not a stable concept; non-explainable AI is often more accurate; mechanisms intended to improve explainability do not improve understanding and introduce new epistemic concerns; and explainability requirements are ad hoc where human medical decision-making is often opaque. A recent ‘political response’ to these issues contends that AI used in high-stakes scenarios, including medical (...) AI, must be explainable to meet basic standards of legitimacy: People are owed reasons for decisions that impact their vital interests, and this requires explainable AI. This article demonstrates why the political response fails. Attending to systemic considerations, as its proponents desire, suggests that the political response is subject to the same criticisms as other arguments for explainable AI and presents new issues. It also suggests that decision-making about non-explainable medical AI can meet public reason standards. The most plausible version of the response amounts to a simple claim that public reason demands reasons why AI is permitted. But that does not actually support explainable AI or respond to criticisms of strong requirements for explainable medical AI. (shrink)