Making good decisions in extremely complex and difficult processes and situations has always been both a key task as well as a challenge in the clinic and has led to a large amount of clinical, legal and ethical routines, protocols and reflections in order to guarantee fair, participatory and up-to-date pathways for clinical decision-making. Nevertheless, the complexity of processes and physical phenomena, time as well as economic constraints and not least further endeavours as well as achievements in medicine and healthcare (...) continuously raise the need to evaluate and to improve clinical decision-making. This article scrutinises if and how clinical decision-making processes are challenged by the rise of so-called artificial intelligence-driven decision support systems (AI-DSS). In a first step, this article analyses how the rise of AI-DSS will affect and transform the modes of interaction between different agents in the clinic. In a second step, we point out how these changing modes of interaction also imply shifts in the conditions of trustworthiness, epistemic challenges regarding transparency, the underlying normative concepts of agency and its embedding into concrete contexts of deployment and, finally, the consequences for (possible) ascriptions of responsibility. Third, we draw first conclusions for further steps regarding a ‘meaningful human control’ of clinical AI-DSS. (shrink)

Although decision‐making algorithms are not new to medicine, the availability of vast stores of medical data, gains in computing power, and breakthroughs in machine learning are accelerating the pace of their development, expanding the range of questions they can address, and increasing their predictive power. In many cases, however, the most powerful machine learning techniques purchase diagnostic or predictive accuracy at the expense of our ability to access “the knowledge within the machine.” Without an explanation in terms of reasons or (...) a rationale for particular decisions in individual cases, some commentators regard ceding medical decision‐making to black box systems as contravening the profound moral responsibilities of clinicians. I argue, however, that opaque decisions are more common in medicine than critics realize. Moreover, as Aristotle noted over two millennia ago, when our knowledge of causal systems is incomplete and precarious—as it often is in medicine—the ability to explain how results are produced can be less important than the ability to produce such results and empirically verify their accuracy. (shrink)

In recent years, increasingly advanced artificial intelligence (AI), and in particular machine learning, has shown great promise as a tool in various healthcare contexts. Yet as machine learning in medicine has become more useful and more widely adopted, concerns have arisen about the “black-box” nature of some of these AI models, or the inability to understand—and explain—the inner workings of the technology. Some critics argue that AI algorithms must be explainable to be responsibly used in the clinical encounter, while supporters (...) of AI dismiss the importance of explainability and instead highlight the many benefits the application of this technology could have for medicine. However, this dichotomy fails to consider the particular ways in which machine learning technologies mediate relations in the clinical encounter, and in doing so, makes explainability more of a problem than it actually is. We argue that postphenomenology is a highly useful theoretical lens through which to examine black-box AI, because it helps us better understand the particular mediating effects this type of technology brings to clinical encounters and moves beyond the explainability stalemate. Using a postphenomenological approach, we argue that explainability is more of a concern for physicians than it is for patients, and that a lack of explainability does not introduce a novel concern to the physician–patient encounter. Explainability is just one feature of technological mediation and need not be the central concern on which the use of black-box AI hinges. (shrink)

Artificial intelligence and machine learning have the potential to revolutionize the delivery of health care. But designing machine learning‐based decision support systems is not a merely technical challenge. It also requires attention to bioethical principles. As AI and machine learning advance, bioethical frameworks need to be tailored to address the problems that these evolving systems might pose, and the development of these automated systems also needs to be tailored to incorporate bioethical principles.

Artificial intelligence (AI) technologies in medicine are gradually changing biomedical research and patient care. High expectations and promises from novel AI applications aiming to positively impact society raise new ethical considerations for patients and caregivers who use these technologies. Based on a qualitative content analysis of semi-structured interviews and focus groups with healthcare professionals (HCPs), patients, and family members of patients with Parkinson’s Disease (PD), the present study investigates participant views on the comparative benefits and problems of using human versus (...) AI predictive computer vision health monitoring, as well as participants’ ethical concerns regarding these technologies. Participants presumed that AI monitoring would enhance information sharing and treatment, but voiced concerns about data ownership, data protection, commercialization of patient data, and privacy at home. They highlighted that privacy issues at home and data security issues are often linked and should be investigated together. Findings may help technologists, HCPs, and policymakers determine how to incorporate stakeholders’ intersecting but divergent concerns into developing and implementing AI PD monitoring tools. (shrink)

From predicting medical conditions to administering health behavior interventions, artificial intelligence technologies are being developed to enhance patient care and outcomes. However, as Mélanie Terrasse and coauthors caution in an article in this issue of the Hastings Center Report, an overreliance on virtual technologies may depersonalize medical interactions and erode therapeutic relationships. The increasing expectation that patients will be actively engaged in their own care, regardless of the patients’ desire, technological literacy, and economic means, may also violate patients’ autonomy and (...) exacerbate access. Moreover, since AI design is both a technical and social process, algorithms may mirror human biases, calling into question the vision of AI technologies surpassing human judgment and avoiding prejudices in decision‐making. The best answer to these problems is to develop AI health technologies as part of a culture of health care quality improvement, responding to existing needs while being proactive about potential technical and ethical problems that can arise from the technologies’ design and implementation. (shrink)

Artificial intelligence (AI) has garnered tremendous attention in health care, and many hope that AI can enhance our health system’s ability to care for people with chronic and degenerative conditions, including Parkinson’s Disease (PD). This paper reports the themes and lessons derived from a qualitative study with people living with PD, family caregivers, and health care providers regarding the ethical dimensions of using AI to monitor, assess, and predict PD symptoms and progression. Thematic analysis identified ethical concerns at four intersecting (...) levels: personal, interpersonal, professional/institutional, and societal levels. Reflecting on potential benefits of predictive algorithms that can continuously collect and process longitudinal data, participants expressed a desire for more timely, ongoing, and accurate information that could enhance management of day-to-day fluctuations and facilitate clinical and personal care as their disease progresses. Nonetheless, they voiced concerns about intersecting ethical questions around evolving illness identities, familial and professional care relationships, privacy, and data ownership/governance. The multi-layer analysis provides a helpful way to understand the ethics of using AI in monitoring and managing PD and other chronic/degenerative conditions. (shrink)