Advanced AI systems are rapidly making their way into medical research and practice, and, arguably, it is only a matter of time before they will surpass human practitioners in terms of accuracy, reliability, and knowledge. If this is true, practitioners will have a prima facie epistemic and professional obligation to align their medical verdicts with those of advanced AI systems. However, in light of their complexity, these AI systems will often function as black boxes: the details of their contents, calculations, (...) and procedures cannot be meaningfully understood by human practitioners. When AI systems reach this level of complexity, we can also speak of black-box medicine. In this paper, we want to argue that black-box medicine conflicts with core ideals of patient-centered medicine. In particular, we claim, black-box medicine is not conducive for supporting informed decision-making based on shared information, shared deliberation, and shared mind between practitioner and patient. (shrink)

Conversational artificial intelligence (CAI) presents many opportunities in the psychotherapeutic landscape—such as therapeutic support for people with mental health problems and without access to care. The adoption of CAI poses many risks that need in-depth ethical scrutiny. The objective of this paper is to complement current research on the ethics of AI for mental health by proposing a holistic, ethical, and epistemic analysis of CAI adoption. First, we focus on the question of whether CAI is rather a tool or an (...) agent. This question serves as a framework for the subsequent ethical analysis of CAI focusing on topics of (self-) knowledge, (self-)understanding, and relationships. Second, we propose further conceptual and ethical analysis regarding human-AI interaction and argue that CAI cannot be considered as an equal partner in a conversation as is the case with a human therapist. Instead, CAI’s role in a conversation should be restricted to specific functions. (shrink)

Physicians and other medical practitioners make untold numbers of judgments about patient care on a daily, weekly, and monthly basis. These judgments fall along a number of spectrums, from the mundane to the tragic, from the obvious to the challenging. Under the rubric of evidence-based medicine, these judgments will be informed by the robust conclusions of medical research. In the ideal circumstance, medical research makes the best decision obvious to the trained professional. Even when practice approximates this ideal, it does (...) so unevenly. Judgments in medical practice are always accompanied by uncertainty, and this uncertainty is a fickle companion—constant in its presence but inconstant in its expression. This feature of medical judgments gives rise to the moral responsibility of medical practitioners to be epistemically humble. This requires the recognition and communication of the uncertainty that accompanies their judgment as well as a commitment to avoiding intuitive innovations. (shrink)

The notion of “responsibility gap” with artificial intelligence (AI) was originally introduced in the philosophical debate to indicate the concern that “learning automata” may make more difficult or impossible to attribute moral culpability to persons for untoward events. Building on literature in moral and legal philosophy, and ethics of technology, the paper proposes a broader and more comprehensive analysis of the responsibility gap. The responsibility gap, it is argued, is not one problem but a set of at least four interconnected (...) problems – gaps in culpability, moral and public accountability, active responsibility—caused by different sources, some technical, other organisational, legal, ethical, and societal. Responsibility gaps may also happen with non-learning systems. The paper clarifies which aspect of AI may cause which gap in which form of responsibility, and why each of these gaps matter. It proposes a critical review of partial and non-satisfactory attempts to address the responsibility gap: those which present it as a new and intractable problem (“fatalism”), those which dismiss it as a false problem (“deflationism”), and those which reduce it to only one of its dimensions or sources and/or present it as a problem that can be solved by simply introducing new technical and/or legal tools (“solutionism”). The paper also outlines a more comprehensive approach to address the responsibility gaps with AI in their entirety, based on the idea of designing socio-technical systems for “meaningful human control", that is systems aligned with the relevant human reasons and capacities. (shrink)

Objective: To examine the role of explainability in machine learning for healthcare (MLHC), and its necessity and significance with respect to effective and ethical MLHC application. Study Design and Setting: This commentary engages with the growing and dynamic corpus of literature on the use of MLHC and artificial intelligence (AI) in medicine, which provide the context for a focused narrative review of arguments presented in favour of and opposition to explainability in MLHC. Results: We find that concerns regarding explainability are (...) not limited to MLHC, but rather extend to numerous well-validated treatment interventions as well as to human clinical judgment itself. We examine the role of evidence-based medicine in evaluating unexplainable treatments and technologies, and highlight the analogy between the concept of explainability in MLHC and the related concept of mechanistic reasoning in evidence-based medicine. Conclusion: Ultimately, we conclude that the value of explainability in MLHC is not intrinsic, but is instead instrumental to achieving greater imperatives such as performance and trust. We caution against the uncompromising pursuit of explainability, and advocate instead for the development of robust empirical methods to successfully evaluate increasingly inexplicable algorithmic systems. (shrink)

The application of artificial intelligence and machine learning technologies in healthcare have immense potential to improve the care of patients. While there are some emerging practices surro...

Although decision‐making algorithms are not new to medicine, the availability of vast stores of medical data, gains in computing power, and breakthroughs in machine learning are accelerating the pace of their development, expanding the range of questions they can address, and increasing their predictive power. In many cases, however, the most powerful machine learning techniques purchase diagnostic or predictive accuracy at the expense of our ability to access “the knowledge within the machine.” Without an explanation in terms of reasons or (...) a rationale for particular decisions in individual cases, some commentators regard ceding medical decision‐making to black box systems as contravening the profound moral responsibilities of clinicians. I argue, however, that opaque decisions are more common in medicine than critics realize. Moreover, as Aristotle noted over two millennia ago, when our knowledge of causal systems is incomplete and precarious—as it often is in medicine—the ability to explain how results are produced can be less important than the ability to produce such results and empirically verify their accuracy. (shrink)

In this paper we argue that ill persons are particularly vulnerable to epistemic injustice in the sense articulated by Fricker. Ill persons are vulnerable to testimonial injustice through the presumptive attribution of characteristics like cognitive unreliability and emotional instability that downgrade the credibility of their testimonies. Ill persons are also vulnerable to hermeneutical injustice because many aspects of the experience of illness are difficult to understand and communicate and this often owes to gaps in collective hermeneutical resources. We then argue (...) that epistemic injustice arises in part owing to the epistemic privilege enjoyed by the practitioners and institutions of contemporary healthcare services—the former owing to their training, expertise, and third-person psychology, and the latter owing to their implicit privileging of certain styles of articulating and evidencing testimonies in ways that marginalise ill persons. We suggest that a phenomenological toolkit may be part of an effort to ameliorate epistemic injustice. (shrink)

Evidence-based medicine has always required integration of patient values with ‘best’ clinical evidence. It is widely recognized that scientific practices and discoveries, including those of EBM, are value-laden. But to date, the science of EBM has focused primarily on methods for reducing bias in the evidence, while the role of values in the different aspects of the EBM process has been almost completely ignored.

It is generally accepted that the therapeutic relationship between professionals and patients is one of trust. Nonetheless, some patient groups carry certain social vulnerabilities that can be exacerbated when they extend trust to health-care professionals. In exploring the epistemic and ethical implications of expert status, this paper examines how calls to trust may increase epistemic oppression and perpetuate the vulnerability of people with impairments. It critically evaluates the processes through which epistemic communities are formed or determined, and examines the institutional (...) structure and power relations that contribute to or sustain such self-affirming ideological communities. It also argues for a two-way collaborative approach between professionals and people with impairments that recognizes the role of local knowledge in enhancing professionals’ own understanding of their theoretical framework and promoting responsive practices and policies. (shrink)

Evidence-based psychiatry (EBP) has arisen through the application of evidence-based medicine (EBM) to psychiatry. However, there may be aspects of psychiatric disorders and treatments that do not conform well to the assumptions of EBM. This paper reviews the ongoing debate about evidence-based psychiatry and investigates the applicability, to psychiatry, of two basic methodological features of EBM: prognostic homogeneity of clinical trial groups and quantification of trial outcomes. This paper argues that EBM may not be the best way to pursue psychiatric (...) knowledge given the particular features of psychiatric disorders and their treatments. As a result, psychiatry may have to develop its own standards for rigour and validity. This paper concludes that EBM has had a powerful influence on how psychiatry investigates and understands mental disorders. Psychiatry could influence EBM in return, reshaping it in ways that are more clinically useful and congruent with patients’ needs. (shrink)

ML-HCAs have the potential to significantly change an entire healthcare system. It is not even necessary to presume that this will be disruptive but sufficient to assume that the mere adaptation of...

This article reports the findings of AI4People, an Atomium—EISMD initiative designed to lay the foundations for a “Good AI Society”. We introduce the core opportunities and risks of AI for society; present a synthesis of five ethical principles that should undergird its development and adoption; and offer 20 concrete recommendations—to assess, to develop, to incentivise, and to support good AI—which in some cases may be undertaken directly by national or supranational policy makers, while in others may be led by other (...) stakeholders. If adopted, these recommendations would serve as a firm foundation for the establishment of a Good AI Society. (shrink)