The impact of machine learning in healthcare on patient informed consent is now the subject of significant inquiry in bioethics. However, the topic has predominantly been considered in the context of black box diagnostic or treatment recommendation algorithms. The impact of machine learning involved in healthcare resource allocation on patient consent remains undertheorized. This paper will establish where patient consent is relevant in healthcare resource allocation, before exploring the impact on informed consent from the introduction of black box machine learning (...) into resource allocation. It will then consider the arguments for informing patients about the use of machine learning in resource allocation, before exploring the challenge of whether individual patients could principally contest algorithmic prioritization decisions involving black box machine learning. Finally, this paper will examine how different forms of opacity in machine learning involved in resource allocation could be a barrier to patient consent to clinical decision-making in different healthcare contexts. (shrink)

From epileptic seizures to depressive symptoms, predictive neurotechnologies are used for a large range of applications. In this article we focus on advisory devices; namely, predictive neurotechnology programmed to detect specific neural events (e.g., epileptic seizure) and advise users to take necessary steps to reduce or avoid the impact of the forecasted neuroevent. Receiving advise from a predictive device is not without ethical concerns. The problem with predictive neural devices, in particular advisory ones, is the risk of seeing one’s autonomous (...) choice supplanted by the predictions instead of being supplemented by it. For users, there is a potential shift from being assisted by the system to being over-dependent on the technology. In other terms, it introduces ethical issues associated with epistemic dependency. In this article, we examine the notion of epistemic authority in relation to predictive neurotechnologies. Section 1 of our article explores and defines the concept of epistemic authority. In section 2, we illustrate how predictive devices are best conceived of as epistemic authorities and we explore the subject-device epistemic relationship. In section 3, we spell out the risk of harms interconnected with epistemic deferral. We conclude by stressing a set of preliminary measures to prepare users for the authoritative nature of predictive devices. (shrink)

The study of rare diseases has long been an area of challenge for medical researchers, with agonizingly slow movement towards improved understanding of pathophysiology and treatments compared with more common illnesses. The push towards evidence-based medicine (EBM), which prioritizes certain types of evidence over others, poses a particular issue when mapped onto rare diseases, which may not be feasibly investigated using the methodologies endorsed by EBM, due to a number of constraints. While other trial designs have been suggested to overcome (...) these limitations (with varying success), perhaps the most recent and enthusiastically adopted is the application of artificial intelligence to rare disease data. This paper critically examines the pitfalls of EBM (and its trial design offshoots) as it pertains to rare diseases, exploring the current landscape of AI as a potential solution to these challenges. This discussion is also taken a step further, providing philosophical commentary on the weaknesses and dangers of AI algorithms applied to rare disease research. While not proposing a singular solution, this article does provide a thoughtful reminder that no ‘one-size-fits-all’ approach exists in the complex world of rare diseases. We must balance cautious optimism with critical evaluation of new research paradigms and technology, while at the same time not neglecting the ever-important aspect of patient values and preferences, which may be challenging to incorporate into computer-driven models. (shrink)

In the ever-evolving landscape of Artificial Intelligence in Healthcare (AIH), understanding the entities and legal frameworks governing its research and development is crucial. This report delves into the intricacies of AIH in Malaysia, undertaking a comprehensive literature search on scientific databases, government portals, and news sources. Additionally, bibliometric analysis has been concurrently conducted to discern trends and developments in AIH over the years. Notably, the interest in AIH has seen a consistent rise since 2017, marked by a growing number of (...) use cases (25 reported here) developed by both local and foreign innovators and applicators. Despite this surge in research and adoption, Malaysia lacks direct legislation specifically addressing AIH technologies, leaving them subject to 11 existing laws. This lack of clear oversight is compounded by the insufficient expertise within local regulatory and ethical bodies to effectively assess AIH research and deployment. The resultant challenges include bureaucratic hurdles for AIH innovators and applicators, raising ethical concerns related to patient autonomy, privacy, data management, AI robustness, and liability. To address these issues, this paper recommends: (1) adopting international ethical guidelines for AIH, (2) enhancing public awareness and education on AI technologies, and (3) promoting AIH research through clinical or silent trials to improve oversight and foster innovation. (shrink)

Providing healthcare services frequently involves cognitively demanding tasks, including diagnoses and analyses as well as complex decisions about treatments and therapy. From a global perspective, ethically significant inequalities exist between regions where the expert knowledge required for these tasks is scarce or abundant. One possible strategy to diminish such inequalities and increase healthcare opportunities in expert-scarce settings is to provide healthcare solutions involving digital technologies that do not necessarily require the presence of a human expert, e.g., in the form of (...) artificial intelligent decision-support systems (AI-DSS). Such algorithmic decision-making, however, is mostly developed in resource- and expert-abundant settings to support healthcare experts in their work. As a practical consequence, the normative standards and requirements for such algorithmic decision-making in healthcare require the technology to be at least as explainable as the decisions made by the experts themselves. The goal of providing healthcare in settings where resources and expertise are scarce might come with a normative pull to lower the normative standards of using digital technologies in order to provide at least some healthcare in the first place. We scrutinize this tendency to lower standards in particular settings from a normative perspective, distinguish between different types of absolute and relative, local and global standards of explainability, and conclude by defending an ambitious and practicable standard of local relative explainability. (shrink)

Recent publications on digital health technologies highlight the importance of ‘responsible’ use. References to the concept of responsibility are, however, frequently made without providing clear definitions of responsibility, thus leaving room for ambiguities. Addressing these uncertainties is critical since they might lead to misunderstandings, impacting the quality and safety of healthcare delivery. Therefore, this study investigates how responsibility is interpreted in the context of using digital health technologies, including artificial intelligence (AI), telemonitoring, wearables and mobile apps. We conducted a scoping (...) review with a systematic search in PubMed, Web of Science, Embase, CINAHL and Philosopher’s Index. A total of 34 articles were included and categorized using a theoretical framework of responsibility aspects, and revealed two main findings. First, we found that digital health technologies can expand and shift existing ‘role responsibilities’ among caregivers, patients and technology. Second, moral responsibility is often equated with liability or accountability, without clear justification. Articles describe new ways in which physicians can be held accountable, particularly in the context of AI, and discuss the emergence of a ‘responsibility gap’ where no-one can be fully responsible for AI-generated outcomes. The literature also shows that m-Health technologies can increase patients’ accountability for their own health. However, there was limited discussion in the reviewed literature on whether these attributions of accountability are appropriate. We conclude with implications for practice and suggestions for expanding the theoretical framework of moral responsibility, recommending further study on responsibility of collectives and artificial entities, and on the role of virtue in digital health. (shrink)

This manuscript draws on the moral norms arising from the nuanced accounts of epistemic (in)justice and social identity in relational autonomy to normatively assess and articulate the ethical problems associated with using AI in patient care in light of the Black Box problem. The article also describes how black-boxed AI may be used within the healthcare system. The manuscript highlights what needs to happen to align AI with the moral norms it draws on. Deeper thinking – from other backgrounds other (...) than decolonial scholarship and relational autonomy – about the impact of AI on the human experience needs to be done to appreciate any other barriers that may exist. Future studies can take up this task. (shrink)

A large part of the explainable AI literature focuses on what explanations are in general, what algorithmic explainability is more specifically, and how to code these principles of explainability into AI systems. Much less attention has been devoted to the question of why algorithmic decisions and systems should be explainable and whether there ought to be a right to explanation and why. We therefore explore the normative landscape of the need for AI to be explainable and individuals having a right (...) to such explanation. This exploration is particularly relevant to the medical domain where the (im)possibility of explainable AI is high on both the research and practitioners’ agenda. The dominant intuition overall is that explainability has and should play a key role in the health context. Notwithstanding the strong normative intuition for having a right to explanation, intuitions can be wrong. So, we need more than an appeal to intuitions when it comes to explaining the normative significance of having a right to explanation when being subject to AI-based decision-making. The aim of the paper is therefore to provide an account of what might underlie the normative intuition. We defend the ‘symmetry thesis’ according to which there is no special normative reason to have a right to explanation when ‘machines’ in the broad sense, make decisions, recommend treatment, discover tumors, and so on. Instead, we argue that we have a right to explanation in cases that involve automated processing that significantly affect our core deliberative agency and which we do not understand, because we have a general moral right to explanation when choices are made which significantly affect us but which we do not understand. (shrink)

Background Despite continuous performance improvements, especially in clinical contexts, a major challenge of Artificial Intelligence based Decision Support Systems (AI-DSS) remains their degree of epistemic opacity. The conditions of and the solutions for the justified use of the occasionally unexplainable technology in healthcare are an active field of research. In March 2024, the European Union agreed upon the Artificial Intelligence Act (AIA), requiring medical AI-DSS to be ad-hoc explainable or to use post-hoc explainability methods. The ethical debate does not seem (...) to settle on this requirement yet. This systematic review aims to outline and categorize the positions and arguments in the ethical debate. Methods We conducted a literature search on PubMed, BASE, and Scopus for English-speaking scientific peer-reviewed publications from 2016 to 2024. The inclusion criterion was to give explicit requirements of explainability for AI-DSS in healthcare and reason for it. Non-domain-specific documents, as well as surveys, reviews, and meta-analyses were excluded. The ethical requirements for explainability outlined in the documents were qualitatively analyzed with respect to arguments for the requirement of explainability and the required level of explainability. Results The literature search resulted in 1662 documents; 44 documents were included in the review after eligibility screening of the remaining full texts. Our analysis showed that 17 records argue in favor of the requirement of explainable AI methods (xAI) or ad-hoc explainable models, providing 9 categories of arguments. The other 27 records argued against a general requirement, providing 11 categories of arguments. Also, we found that 14 works advocate the need for context-dependent levels of explainability, as opposed to 30 documents, arguing for context-independent, absolute standards. Conclusions The systematic review of reasons shows no clear agreement on the requirement of post-hoc explainability methods or ad-hoc explainable models for AI-DSS in healthcare. The arguments found in the debate were referenced and responded to from different perspectives, demonstrating an interactive discourse. Policymakers and researchers should watch the development of the debate closely. Conversely, ethicists should be well informed by empirical and technical research, given the frequency of advancements in the field. (shrink)

Due to its enormous potential, artificial intelligence (AI) can transform healthcare on a seemingly infinite scale. However, as we continue to explore the immense potential of AI, it is vital to consider the ethical concerns associated with its development and deployment. One specific concern that has been flagged in the literature is the responsibility gap (RG) due to the introduction of AI in healthcare. When the use of an AI algorithm or system results in a negative outcome for a patient(s), (...) to whom can or should responsibility for that outcome be assigned? Although the concept of the RG was introduced in Anglo-American and European philosophy, this paper aims to broaden the debate by providing an Ubuntu-inspired perspective on the RG. Ubuntu, deeply rooted in African philosophy, calls for collective responsibility, and offers a uniquely forward-looking approach to address the alleged RG caused by AI in healthcare. An Ubuntu-inspired perspective can serve as a valuable guide and tool when addressing the alleged RG. Incorporating Ubuntu into the AI ethics discourse can contribute to a more ethical and responsible integration of AI in healthcare. (shrink)