The application of artificial intelligence and machine learning technologies in healthcare have immense potential to improve the care of patients. While there are some emerging practices surro...

In the research ethics literature, there is strong disagreement about the ethical acceptability of placebo-controlled trials, particularly when a tested therapy aims to alleviate a condition for which standard treatment exists. Recently, this disagreement has given rise to debate over the moral appropriateness of the principle of clinical equipoise for medical research. Underlying these debates are two fundamentally different visions of the moral obligations that investigators owe their subjects.Some commentators and ethics documents claim that physicians, whether acting as care givers (...) or researchers, have the same duty of beneficence towards their patients and subjects: namely, that they must provide optimal medical care. In discussing placebo surgery in research on refractory Parkinson's disease, Peter Clark succinctly states this view: “The researcher has an ethical responsibility to act in the best interest of subjects.”. (shrink)

In the research ethics literature, there is strong disagreement about the ethical acceptability of placebo-controlled trials, particularly when a tested therapy aims to alleviate a condition for which standard treatment exists. Recently, this disagreement has given rise to debate over the moral appropriateness of the principle of clinical equipoise for medical research. Underlying these debates are two fundamentally different visions of the moral obligations that investigators owe their subjects.Some commentators and ethics documents claim that physicians, whether acting as care givers (...) or researchers, have the same duty of beneficence towards their patients and subjects: namely, that they must provide optimal medical care. In discussing placebo surgery in research on refractory Parkinson's disease, Peter Clark succinctly states this view: “The researcher has an ethical responsibility to act in the best interest of subjects.”. (shrink)

We argued last year in this journal that extensive integration of research and care is a worthy goal of health system design, and we second the call from Ruth Faden and colleagues to move toward learning health care systems. As they recognize, learning health care systems demand the coordination of research and medical ethics—two sets of normative commitments that have long been considered distinct. In offering a novel ethics framework for such systems, Faden et al. advance the scholarly debate about (...) how best to do this and challenge us to think more deeply about its practical implications. However, we believe that at least three issues they raise require additional attention. First, the research‐care distinction has more normative bite than they are willing to concede. Second, the role for independent oversight requires clarification. Finally, they neglect the concept of stewardship as a guiding obligation of learning health care systems. (shrink)

We argued last year in this journal that extensive integration of research and care is a worthy goal of health system design, and we second the call from Ruth Faden and colleagues to move toward learning health care systems. As they recognize, learning health care systems demand the coordination of research and medical ethics—two sets of normative commitments that have long been considered distinct. In offering a novel ethics framework for such systems, Faden et al. advance the scholarly debate about (...) how best to do this and challenge us to think more deeply about its practical implications. However, we believe that at least three issues they raise require additional attention. First, the research‐care distinction has more normative bite than they are willing to concede. Second, the role for independent oversight requires clarification. Finally, they neglect the concept of stewardship as a guiding obligation of learning health care systems. (shrink)

Calls are increasing for American health care to be organized as a learning health care system, defined by the Institute of Medicine as a health care system “in which knowledge generation is so embedded into the core of the practice of medicine that it is a natural outgrowth and product of the healthcare delivery process and leads to continual improvement in care.” We applaud this conception, and in this paper, we put forward a new ethics framework for it. No such (...) framework has previously been articulated. The goals of our framework are twofold: to support the transformation to a learning health care system and to help ensure that learning activities carried out within such a system are conducted in an ethically acceptable fashion. (shrink)

Artificial intelligence (AI) systems have demonstrated impressive performance across a variety of clinical tasks. However, notoriously, sometimes these systems are 'black boxes'. The initial response in the literature was a demand for 'explainable AI'. However, recently, several authors have suggested that making AI more explainable or 'interpretable' is likely to be at the cost of the accuracy of these systems and that prioritising interpretability in medical AI may constitute a 'lethal prejudice'. In this paper, we defend the value of interpretability (...) in the context of the use of AI in medicine. Clinicians may prefer interpretable systems over more accurate black boxes, which in turn is sufficient to give designers of AI reason to prefer more interpretable systems in order to ensure that AI is adopted and its benefits realised. Moreover, clinicians may be justified in this preference. Achieving the downstream benefits from AI is critically dependent on how the outputs of these systems are interpreted by physicians and patients. A preference for the use of highly accurate black box AI systems, over less accurate but more interpretable systems , may itself constitute a form of lethal prejudice that may diminish the benefits of AI to - and perhaps even harm - patients. (shrink)

In recent years, there has been a surge of high-profile publications on applications of artificial intelligence (AI) systems for medical diagnosis and prognosis. While AI provides various opportunities for medical practice, there is an emerging consensus that the existing studies show considerable deficits and are unable to establish the clinical benefit of AI systems. Hence, the view that the clinical benefit of AI systems needs to be studied in clinical trials—particularly randomised controlled trials (RCTs)—is gaining ground. However, an issue that (...) has been overlooked so far in the debate is that, compared with drug RCTs, AI RCTs require methodological adjustments, which entail ethical challenges. This paper sets out to develop a systematic account of the ethics of AI RCTs by focusing on the moral principles of clinical equipoise, informed consent and fairness. This way, the objective is to animate further debate on the (research) ethics of medical AI. (shrink)

In recent years, a plethora of high-profile scientific publications has been reporting about machine learning algorithms outperforming clinicians in medical diagnosis or treatment recommendations. This has spiked interest in deploying relevant algorithms with the aim of enhancing decision-making in healthcare. In this paper, we argue that instead of straightforwardly enhancing the decision-making capabilities of clinicians and healthcare institutions, deploying machines learning algorithms entails trade-offs at the epistemic and the normative level. Whereas involving machine learning might improve the accuracy of medical (...) diagnosis, it comes at the expense of opacity when trying to assess the reliability of given diagnosis. Drawing on literature in social epistemology and moral responsibility, we argue that the uncertainty in question potentially undermines the epistemic authority of clinicians. Furthermore, we elucidate potential pitfalls of involving machine learning in healthcare with respect to paternalism, moral responsibility and fairness. At last, we discuss how the deployment of machine learning algorithms might shift the evidentiary norms of medical diagnosis. In this regard, we hope to lay the grounds for further ethical reflection of the opportunities and pitfalls of machine learning for enhancing decision-making in healthcare. (shrink)

Calls are increasing for American health care to be organized as a learning health care system, defined by the Institute of Medicine as a health care system “in which knowledge generation is so embedded into the core of the practice of medicine that it is a natural outgrowth and product of the healthcare delivery process and leads to continual improvement in care.” We applaud this conception, and in this paper, we put forward a new ethics framework for it. No such (...) framework has previously been articulated. The goals of our framework are twofold: to support the transformation to a learning health care system and to help ensure that learning activities carried out within such a system are conducted in an ethically acceptable fashion. (shrink)

This article considers the issue of opacity as a problem for socially consequential mechanisms of classification and ranking, such as spam filters, credit card fraud detection, search engines, news trends, market segmentation and advertising, insurance or loan qualification, and credit scoring. These mechanisms of classification all frequently rely on computational algorithms, and in many cases on machine learning algorithms to do this work. In this article, I draw a distinction between three forms of opacity: opacity as intentional corporate or state (...) secrecy, opacity as technical illiteracy, and an opacity that arises from the characteristics of machine learning algorithms and the scale required to apply them usefully. The analysis in this article gets inside the algorithms themselves. I cite existing literatures in computer science, known industry practices, and do some testing and manipulation of code as a form of lightweight code audit. I argue that recognizing the distinct forms of opacity that may be coming into play in a given application is a key to determining which of a variety of technical and non-technical solutions could help to prevent harm. (shrink)

A special report of The Hastings Center and the Association of American Medical Colleges addressed the ethical oversight of learning health systems, which seek to combine high‐quality patient care with routine data collection aimed at improving patient outcomes. The report contained two position papers, authored by a number of distinguished bioethicists, and several commentaries. The position papers urged two changes. First, they urged a rethinking of our approach to the regulation of human subjects research, so as to make it easier (...) in the future for learning health systems to function well. Second, they argued that the rethinking required dispensing with a strict distinction between research and therapy, which has been a major tenet of bioethics since the Belmont Report, which explicated basic ethical principles governing human subjects research. We fully support the objectives of the authors, and we agree that the learning health system is an important advance that serves patients well. Unnecessary regulatory burden ought not impede this progress. We disagree, however, that the best way to bring about these needed changes in the regulatory environment is to reject the basic distinction between research and treatment. Unfortunately, we find the arguments in favor of that strategy to be, in places, reminiscent of what we take to be basic conceptual errors that hampered the ethical understanding of human subjects research prior to adoption of the Belmont Report. To see why one need not reject the research‐treatment distinction in order to promote learning health systems, we first investigate in some detail the arguments offered for eliminating the distinction. We next turn to an issue not addressed by those authors, namely, the relationship between the physician or investigator and the patient or subject, to illustrate why the distinction is important and what is lost if it is jettisoned. (shrink)

Advanced AI systems are rapidly making their way into medical research and practice, and, arguably, it is only a matter of time before they will surpass human practitioners in terms of accuracy, reliability, and knowledge. If this is true, practitioners will have a prima facie epistemic and professional obligation to align their medical verdicts with those of advanced AI systems. However, in light of their complexity, these AI systems will often function as black boxes: the details of their contents, calculations, (...) and procedures cannot be meaningfully understood by human practitioners. When AI systems reach this level of complexity, we can also speak of black-box medicine. In this paper, we want to argue that black-box medicine conflicts with core ideals of patient-centered medicine. In particular, we claim, black-box medicine is not conducive for supporting informed decision-making based on shared information, shared deliberation, and shared mind between practitioner and patient. (shrink)

This article examines the potential implications of the implementation of artificial intelligence in health care for both its delivery and the medical profession. To this end, the first section explores the basic features of AI and the yet theoretical concept of autonomous AI followed by an overview of current and developing AI applications. Against this background, the second section discusses the transforming roles of physicians and changes in the patient–physician relationship that could be a consequence of gradual expansion of AI (...) in health care. Subsequently, an examination of the responsibilities physicians should assume in this process is explored. The third section describes conceivable practical and ethical challenges that implementation of a single all-encompassing AI healthcare system would pose. The fourth section presents arguments for regulation of AI in health care to ensure that these applications do not violate basic ethical principles and that human control of AI will be preserved in the future. In the final section, fundamental components of a moral framework from which such regulation may be derived are brought forward, and some possible strategies for building a moral framework are discussed. (shrink)

In Deep Medicine, Eric Topol argues that the development of artificial intelligence (AI) for healthcare will lead to a dramatic shift in the culture and practice of medicine. Topol claims that, rather than replacing physicians, AI could function alongside of them in order to allow them to devote more of their time to face-to-face patient care. Unfortunately, these high hopes for AI-enhanced medicine fail to appreciate a number of factors that, we believe, suggest a radically different picture for the future (...) of healthcare. Far from facilitating a return to “the golden age of doctoring”, the role of economic and institutional considerations in determining how medical AI will be used mean that it is likely to further erode therapeutic relationships and threaten professional and patient satisfaction. (shrink)

Objectives: Machine learning (ML) has the potential to facilitate “continual learning” in medicine, in which an ML system continues to evolve in response to exposure to new data over time, even after being deployed in a clinical setting. In this article, we provide a tutorial on the range of ethical issues raised by the use of such “adaptive” ML systems in medicine that have, thus far, been neglected in the literature. -/- Target audience: The target audiences for this tutorial are (...) the developers of ML AI systems, healthcare regulators, the broader medical informatics community, and practicing clinicians. -/- Scope: Discussions of adaptive ML systems to date have overlooked the distinction between 2 sorts of variance that such systems may exhibit—diachronic evolution (change over time) and synchronic variation (difference between cotemporaneous instantiations of the algorithm at different sites)—and underestimated the significance of the latter. We highlight the challenges that diachronic evolution and synchronic variation present for the quality of patient care, informed consent, and equity, and discuss the complex ethical trade-offs involved in the design of such systems. (shrink)

What does Artificial Intelligence (AI) have to contribute to health care? And what should we be looking out for if we are worried about its risks? In this paper we offer a survey, and initial evaluation, of hopes and fears about the applications of artificial intelligence in medicine. AI clearly has enormous potential as a research tool, in genomics and public health especially, as well as a diagnostic aid. It’s also highly likely to impact on the organisational and business practices (...) of healthcare systems in ways that are perhaps under-appreciated. Enthusiasts for AI have held out the prospect that it will free physicians up to spend more time attending to what really matters to them and their patients. We will argue that this claim depends upon implausible assumptions about the institutional and economic imperatives operating in contemporary healthcare settings. We will also highlight important concerns about privacy, surveillance, and bias in big data, as well as the risks of over trust in machines, the challenges of transparency, the deskilling of healthcare practitioners, the way AI reframes healthcare, and the implications of AI for the distribution of power in healthcare institutions. We will suggest that two questions, in particular, are deserving of further attention from philosophers and bioethicists. What does care look like when one is dealing with data as much as people? And, what weight should we give to the advice of machines in our own deliberations about medical decisions? (shrink)

An axiom of medical research ethics is that a protocol is moral only if it has a “favorable risk-benefit ratio”. This axiom is usually interpreted in the following way: a medical research protocol is moral only if it has a positive expected value -- that is, if it is likely to do more good (to both subjects and society) than harm. I argue that, thus interpreted, the axiom has two problems. First, it is unusable, because it requires us to know (...) more about the potential outcomes of research than we ever could. Second, it is false, because it conflicts with the so-called “soft paternalist” principles of liberal democracy. In place of this flawed rule I propose a new way of making risk-benefit assessments, one that does comport with the principles of liberalism. I argue that a protocol is moral only if it would be entered into by competent subjects who are informed about the protocol. The new rule this eschews all pseudo-utilitarian calculation about the protocol’s likely harms and benefits. (shrink)

Various publications claim that medical AI systems perform as well, or better, than clinical experts. However, there have been very few controlled trials and the quality of existing studies has been called into question. There is growing concern that existing studies overestimate the clinical benefits of AI systems. This has led to calls for more, and higher-quality, randomized controlled trials of medical AI systems. While this a welcome development, AI RCTs raise novel methodological challenges that have seen little discussion. We (...) discuss some of the challenges arising in the context of AI RCTs and make some suggestions for how to meet them. (shrink)