Artificial intelligence (AI) has attracted an increasing amount of attention, both positive and negative. Its potential applications in healthcare are indeed manifold and revolutionary, and within the realm of medical imaging and radiology (which will be the focus of this paper), significant increases in accuracy and speed, as well as significant savings in cost, stand to be gained through the adoption of this technology. Because of its novelty, a norm of keeping humans “in the loop” wherever AI mechanisms are deployed (...) has become synonymous with good ethical practice in some circles. It has been argued that keeping humans “in the loop” is important for reasons of safety, accountability, and the maintenance of institutional trust. However, as the application of machine learning for the detection of lumbar spinal stenosis (LSS) in this paper’s case study reveals, there are some scenarios where an insistence on keeping humans in the loop (or in other words, the resistance to automation) seems unwarranted and could possibly lead us to miss out on very real and important opportunities in healthcare—particularly in low-resource settings. It is important to acknowledge these opportunity costs of resisting automation in such contexts, where better options may be unavailable. Using an AI model based on convolutional neural networks developed by a team of researchers at NUH/NUS medical school in Singapore for automated detection and classification of the lumbar spinal canal, lateral recess, and neural foraminal narrowing in an MRI scan of the spine to diagnose LSS, we will aim to demonstrate that where certain criteria hold (e.g., the AI is as accurate or better than human experts, risks are low in the event of an error, the gain in wellbeing is significant, and the task being automated is not essentially or importantly human), it is both morally permissible and even desirable to kick the humans out of the loop. (shrink)

Artificial intelligence (AI) holds considerable promise for transforming clinical diagnostics. While much has been written both about public attitudes toward the use of AI tools in medicine and about uncertainty regarding legal liability that may be delaying its adoption, the interface of these two issues has so far drawn less attention. However, understanding this interface is essential to determining how jury behaviour is likely to influence adoption of AI by physicians. One distinctive concern identified in this paper is a ‘negative (...) outcome penalty paradox’ (NOPP) in which physicians risk being penalised by juries in cases with negative outcomes, whether they overrule AI determinations or accept them. The paper notes three reasons why AI in medicine is uniquely susceptible to the NOPP and urges serious further consideration of this complex dilemma. (shrink)

Background Despite continuous performance improvements, especially in clinical contexts, a major challenge of Artificial Intelligence based Decision Support Systems (AI-DSS) remains their degree of epistemic opacity. The conditions of and the solutions for the justified use of the occasionally unexplainable technology in healthcare are an active field of research. In March 2024, the European Union agreed upon the Artificial Intelligence Act (AIA), requiring medical AI-DSS to be ad-hoc explainable or to use post-hoc explainability methods. The ethical debate does not seem (...) to settle on this requirement yet. This systematic review aims to outline and categorize the positions and arguments in the ethical debate. Methods We conducted a literature search on PubMed, BASE, and Scopus for English-speaking scientific peer-reviewed publications from 2016 to 2024. The inclusion criterion was to give explicit requirements of explainability for AI-DSS in healthcare and reason for it. Non-domain-specific documents, as well as surveys, reviews, and meta-analyses were excluded. The ethical requirements for explainability outlined in the documents were qualitatively analyzed with respect to arguments for the requirement of explainability and the required level of explainability. Results The literature search resulted in 1662 documents; 44 documents were included in the review after eligibility screening of the remaining full texts. Our analysis showed that 17 records argue in favor of the requirement of explainable AI methods (xAI) or ad-hoc explainable models, providing 9 categories of arguments. The other 27 records argued against a general requirement, providing 11 categories of arguments. Also, we found that 14 works advocate the need for context-dependent levels of explainability, as opposed to 30 documents, arguing for context-independent, absolute standards. Conclusions The systematic review of reasons shows no clear agreement on the requirement of post-hoc explainability methods or ad-hoc explainable models for AI-DSS in healthcare. The arguments found in the debate were referenced and responded to from different perspectives, demonstrating an interactive discourse. Policymakers and researchers should watch the development of the debate closely. Conversely, ethicists should be well informed by empirical and technical research, given the frequency of advancements in the field. (shrink)

The impact of machine learning in healthcare on patient informed consent is now the subject of significant inquiry in bioethics. However, the topic has predominantly been considered in the context of black box diagnostic or treatment recommendation algorithms. The impact of machine learning involved in healthcare resource allocation on patient consent remains undertheorized. This paper will establish where patient consent is relevant in healthcare resource allocation, before exploring the impact on informed consent from the introduction of black box machine learning (...) into resource allocation. It will then consider the arguments for informing patients about the use of machine learning in resource allocation, before exploring the challenge of whether individual patients could principally contest algorithmic prioritization decisions involving black box machine learning. Finally, this paper will examine how different forms of opacity in machine learning involved in resource allocation could be a barrier to patient consent to clinical decision-making in different healthcare contexts. (shrink)

Wenn Künstliche Intelligenz (KI) Texte generieren kann, was sagt das darüber, was ein Text ist? Worin unterscheiden sich von Menschen geschriebene und mittels KI generierte Texte? Welche Erwartungen, Befürchtungen und Hoffnungen hegen Wissenschaften, wenn in ihren Diskursen KI-generierte Texte rezipiert werden und Anerkennung finden, deren Urheberschaft und Originalität nicht mehr eindeutig definierbar sind? Wie verändert sich die Arbeit mit Quellen und welche Konsequenzen ergeben sich daraus für die Kriterien wissenschaftlicher Textarbeit und das Verständnis von Wissenschaft insgesamt? Welche Chancen, Grenzen und (...) Risiken besitzen KI-Textgeneratoren aus Sicht von Technikethik und Technikfolgenabschätzung? Und welche Kriterien sind für einen ethisch bewussten Umgang mit KI-Textgeneratoren im Bildungskontext anzulegen? Diese und ähnliche Fragen verdeutlichen die Komplexität und Tragweite der Herausforderung, die der stetig wachsende und zunehmend selbstverständliche Einsatz von KI-Textgeneratoren für das zukünftige Arbeiten mit und an Texten mit sich bringt. Hierbei kristallisieren sich Problematiken heraus, die sich an ein breites Spektrum von Disziplinen wenden, deren spezifischen Diskurse in diesem Band beleuchtet werden. (shrink)