Making good decisions in extremely complex and difficult processes and situations has always been both a key task as well as a challenge in the clinic and has led to a large amount of clinical, legal and ethical routines, protocols and reflections in order to guarantee fair, participatory and up-to-date pathways for clinical decision-making. Nevertheless, the complexity of processes and physical phenomena, time as well as economic constraints and not least further endeavours as well as achievements in medicine and healthcare (...) continuously raise the need to evaluate and to improve clinical decision-making. This article scrutinises if and how clinical decision-making processes are challenged by the rise of so-called artificial intelligence-driven decision support systems (AI-DSS). In a first step, this article analyses how the rise of AI-DSS will affect and transform the modes of interaction between different agents in the clinic. In a second step, we point out how these changing modes of interaction also imply shifts in the conditions of trustworthiness, epistemic challenges regarding transparency, the underlying normative concepts of agency and its embedding into concrete contexts of deployment and, finally, the consequences for (possible) ascriptions of responsibility. Third, we draw first conclusions for further steps regarding a ‘meaningful human control’ of clinical AI-DSS. (shrink)

Within the ethical debate on Machine Learning-driven decision support systems (ML_CDSS), notions such as “human in the loop” or “meaningful human control” are often cited as being necessary for ethical legitimacy. In addition, ethical principles usually serve as the major point of reference in ethical guidance documents, stating that conflicts between principles need to be weighed and balanced against each other. Starting from a neo-Kantian viewpoint inspired by Onora O'Neill, this article makes a concrete suggestion of how to interpret the (...) role of the “human in the loop” and to overcome the perspective of rivaling ethical principles in the evaluation of AI in health care. We argue that patients should be perceived as “fellow workers” and epistemic partners in the interpretation of ML_CDSS outputs. We further highlight that a meaningful process of integrating (rather than weighing and balancing) ethical principles is most appropriate in the evaluation of medical AI. (shrink)

Artificial intelligence and machine learning have the potential to revolutionize the delivery of health care. But designing machine learning‐based decision support systems is not a merely technical challenge. It also requires attention to bioethical principles. As AI and machine learning advance, bioethical frameworks need to be tailored to address the problems that these evolving systems might pose, and the development of these automated systems also needs to be tailored to incorporate bioethical principles.

From predicting medical conditions to administering health behavior interventions, artificial intelligence technologies are being developed to enhance patient care and outcomes. However, as Mélanie Terrasse and coauthors caution in an article in this issue of the Hastings Center Report, an overreliance on virtual technologies may depersonalize medical interactions and erode therapeutic relationships. The increasing expectation that patients will be actively engaged in their own care, regardless of the patients’ desire, technological literacy, and economic means, may also violate patients’ autonomy and (...) exacerbate access. Moreover, since AI design is both a technical and social process, algorithms may mirror human biases, calling into question the vision of AI technologies surpassing human judgment and avoiding prejudices in decision‐making. The best answer to these problems is to develop AI health technologies as part of a culture of health care quality improvement, responding to existing needs while being proactive about potential technical and ethical problems that can arise from the technologies’ design and implementation. (shrink)

Although decision‐making algorithms are not new to medicine, the availability of vast stores of medical data, gains in computing power, and breakthroughs in machine learning are accelerating the pace of their development, expanding the range of questions they can address, and increasing their predictive power. In many cases, however, the most powerful machine learning techniques purchase diagnostic or predictive accuracy at the expense of our ability to access “the knowledge within the machine.” Without an explanation in terms of reasons or (...) a rationale for particular decisions in individual cases, some commentators regard ceding medical decision‐making to black box systems as contravening the profound moral responsibilities of clinicians. I argue, however, that opaque decisions are more common in medicine than critics realize. Moreover, as Aristotle noted over two millennia ago, when our knowledge of causal systems is incomplete and precarious—as it often is in medicine—the ability to explain how results are produced can be less important than the ability to produce such results and empirically verify their accuracy. (shrink)

Artificial intelligence (AI) technologies in medicine are gradually changing biomedical research and patient care. High expectations and promises from novel AI applications aiming to positively impact society raise new ethical considerations for patients and caregivers who use these technologies. Based on a qualitative content analysis of semi-structured interviews and focus groups with healthcare professionals (HCPs), patients, and family members of patients with Parkinson’s Disease (PD), the present study investigates participant views on the comparative benefits and problems of using human versus (...) AI predictive computer vision health monitoring, as well as participants’ ethical concerns regarding these technologies. Participants presumed that AI monitoring would enhance information sharing and treatment, but voiced concerns about data ownership, data protection, commercialization of patient data, and privacy at home. They highlighted that privacy issues at home and data security issues are often linked and should be investigated together. Findings may help technologists, HCPs, and policymakers determine how to incorporate stakeholders’ intersecting but divergent concerns into developing and implementing AI PD monitoring tools. (shrink)

Artificial intelligence (AI) has garnered tremendous attention in health care, and many hope that AI can enhance our health system’s ability to care for people with chronic and degenerative conditions, including Parkinson’s Disease (PD). This paper reports the themes and lessons derived from a qualitative study with people living with PD, family caregivers, and health care providers regarding the ethical dimensions of using AI to monitor, assess, and predict PD symptoms and progression. Thematic analysis identified ethical concerns at four intersecting (...) levels: personal, interpersonal, professional/institutional, and societal levels. Reflecting on potential benefits of predictive algorithms that can continuously collect and process longitudinal data, participants expressed a desire for more timely, ongoing, and accurate information that could enhance management of day-to-day fluctuations and facilitate clinical and personal care as their disease progresses. Nonetheless, they voiced concerns about intersecting ethical questions around evolving illness identities, familial and professional care relationships, privacy, and data ownership/governance. The multi-layer analysis provides a helpful way to understand the ethics of using AI in monitoring and managing PD and other chronic/degenerative conditions. (shrink)