In this paper I argue that the search for explainable models and interpretable decisions in AI must be reformulated in terms of the broader project of offering a pragmatic and naturalistic account of understanding in AI. Intuitively, the purpose of providing an explanation of a model or a decision is to make it understandable to its stakeholders. But without a previous grasp of what it means to say that an agent understands a model or a decision, the explanatory strategies will (...) lack a well-defined goal. Aside from providing a clearer objective for XAI, focusing on understanding also allows us to relax the factivity condition on explanation, which is impossible to fulfill in many machine learning models, and to focus instead on the pragmatic conditions that determine the best fit between a model and the methods and devices deployed to understand it. After an examination of the different types of understanding discussed in the philosophical and psychological literature, I conclude that interpretative or approximation models not only provide the best way to achieve the objectual understanding of a machine learning model, but are also a necessary condition to achieve post hoc interpretability. This conclusion is partly based on the shortcomings of the purely functionalist approach to post hoc interpretability that seems to be predominant in most recent literature. (shrink)

Although decision‐making algorithms are not new to medicine, the availability of vast stores of medical data, gains in computing power, and breakthroughs in machine learning are accelerating the pace of their development, expanding the range of questions they can address, and increasing their predictive power. In many cases, however, the most powerful machine learning techniques purchase diagnostic or predictive accuracy at the expense of our ability to access “the knowledge within the machine.” Without an explanation in terms of reasons or (...) a rationale for particular decisions in individual cases, some commentators regard ceding medical decision‐making to black box systems as contravening the profound moral responsibilities of clinicians. I argue, however, that opaque decisions are more common in medicine than critics realize. Moreover, as Aristotle noted over two millennia ago, when our knowledge of causal systems is incomplete and precarious—as it often is in medicine—the ability to explain how results are produced can be less important than the ability to produce such results and empirically verify their accuracy. (shrink)

Trust is central to the therapeutic relationship, but the epistemic asymmetries between the expert healthcare provider and the patient make the patient, the trustor, vulnerable to the provider, the trustee. The narratives of pain sufferers provide helpful insights into the experience of pain at the juncture of trust, expert knowledge, and the therapeutic relationship. While stories of pain sufferers having their testimonies dismissed are well documented, pain sufferers continue to experience their testimonies as being epistemically downgraded. This kind of epistemic (...) injustice has received limited treatment in bioethics. In this paper, we examine how a climate of distrust in pain management may facilitate what Fricker calls epistemic injustice. We critically interrogate the processes through which pain sufferers are vulnerable to specific kinds of epistemic injustice, such as testimonial injustice. We also examine how healthcare institutions and practices privilege some kinds of evidence and ways of knowing while excluding certain patient testimonies from epistemic consideration. We argue that providers ought to avoid epistemic injustice in pain management by striving toward epistemic humility. Epistemic humility, as a form of epistemic justice, may be the kind disposition required to correct the harmful prejudices that may arise through testimonial exchange in chronic pain management. (shrink)

Several philosophical issues in connection with computer simulations rely on the assumption that results of simulations are trustworthy. Examples of these include the debate on the experimental role of computer simulations :483–496, 2009; Morrison in Philos Stud 143:33–57, 2009), the nature of computer data Computer simulations and the changing face of scientific experimentation, Cambridge Scholars Publishing, Barcelona, 2013; Humphreys, in: Durán, Arnold Computer simulations and the changing face of scientific experimentation, Cambridge Scholars Publishing, Barcelona, 2013), and the explanatory power of (...) computer simulations :277–292, 2008; Durán in Int Stud Philos Sci 31:27–45, 2017). The aim of this article is to show that these authors are right in assuming that results of computer simulations are to be trusted when computer simulations are reliable processes. After a short reconstruction of the problem of epistemic opacity, the article elaborates extensively on computational reliabilism, a specified form of process reliabilism with computer simulations located at the center. The article ends with a discussion of four sources for computational reliabilism, namely, verification and validation, robustness analysis for computer simulations, a history of successful implementations, and the role of expert knowledge in simulations. (shrink)

This article analyses the phenomenon of epistemic injustice within contemporary healthcare. We begin by detailing the persistent complaints patients make about their testimonial frustration and hermeneutical marginalization, and the negative impact this has on their care. We offer an epistemic analysis of this problem using Miranda Fricker's account of epistemic injustice. We detail two types of epistemic injustice, testimonial and hermeneutical, and identify the negative stereotypes and structural features of modern healthcare practices that generate them. We claim that these stereotypes (...) and structural features render ill persons especially vulnerable to these two types of epistemic injustice. We end by proposing five avenues for further work on epistemic injustice in healthcare. (shrink)

In information societies, operations, decisions and choices previously left to humans are increasingly delegated to algorithms, which may advise, if not decide, about how data should be interpreted and what actions should be taken as a result. More and more often, algorithms mediate social processes, business transactions, governmental decisions, and how we perceive, understand, and interact among ourselves and with the environment. Gaps between the design and operation of algorithms and our understanding of their ethical implications can have severe consequences (...) affecting individuals as well as groups and whole societies. This paper makes three contributions to clarify the ethical importance of algorithmic mediation. It provides a prescriptive map to organise the debate. It reviews the current discussion of ethical aspects of algorithms. And it assesses the available literature in order to identify areas requiring further work to develop the ethics of algorithms. (shrink)

In this paper we argue that ill persons are particularly vulnerable to epistemic injustice in the sense articulated by Fricker. Ill persons are vulnerable to testimonial injustice through the presumptive attribution of characteristics like cognitive unreliability and emotional instability that downgrade the credibility of their testimonies. Ill persons are also vulnerable to hermeneutical injustice because many aspects of the experience of illness are difficult to understand and communicate and this often owes to gaps in collective hermeneutical resources. We then argue (...) that epistemic injustice arises in part owing to the epistemic privilege enjoyed by the practitioners and institutions of contemporary healthcare services—the former owing to their training, expertise, and third-person psychology, and the latter owing to their implicit privileging of certain styles of articulating and evidencing testimonies in ways that marginalise ill persons. We suggest that a phenomenological toolkit may be part of an effort to ameliorate epistemic injustice. (shrink)

We survey several ways in which the structures and norms of medicine and healthcare can generate epistemic injustice.

Computational methods such as computer simulations, Monte Carlo methods, and agent-based modeling have become the dominant techniques in many areas of science. Extending Ourselves contains the first systematic philosophical account of these new methods, and how they require a different approach to scientific method. Paul Humphreys draws a parallel between the ways in which such computational methods have enhanced our abilities to mathematically model the world, and the more familiar ways in which scientific instruments have expanded our access to the (...) empirical world. This expansion forms the basis for a new kind of empiricism, better suited to the needs of science than the older anthropocentric forms of empiricism. Human abilities are no longer the ultimate standard of epistemological correctness.Humphreys also includes arguments for the primacy of properties rather than objects, the need to consider technological constraints when appraising scientific methods, and a detailed account of how the path from computational template to scientific application is constructed. This last feature allows us to hold a form of selective realism in which anti-realist arguments based on formal reconstructions of theories can be avoided. One important consequence of the rise of computational methods is that the traditional organization of the sciences is being replaced by an organization founded on computational templates.Extending Ourselves will be of interest to philosophers of science, epistemologists, and to anyone interested in the role played by computers in modern science. (shrink)

Reasons are given to justify the claim that computer simulations and computational science constitute a distinctively new set of scientific methods and that these methods introduce new issues in the philosophy of science. These issues are both epistemological and methodological in kind.

Organizations such as the EU High-Level Expert Group on AI and the IEEE have recently formulated ethical principles and (moral) values that should be adhered to in the design and deployment of artificial intelligence (AI). These include respect for autonomy, non-maleficence, fairness, transparency, explainability, and accountability. But how can we ensure and verify that an AI system actually respects these values? To help answer this question, I propose an account for determining when an AI system can be said to embody (...) certain values. This account understands embodied values as the result of design activities intended to embed those values in such systems. AI systems are here understood as a special kind of sociotechnical system that, like traditional sociotechnical systems, are composed of technical artifacts, human agents, and institutions but—in addition—contain artificial agents and certain technical norms that regulate interactions between artificial agents and other elements of the system. The specific challenges and opportunities of embedding values in AI systems are discussed, and some lessons for better embedding values in AI systems are drawn. (shrink)

Journal of Social Philosophy, EarlyView.

In recent years, a plethora of high-profile scientific publications has been reporting about machine learning algorithms outperforming clinicians in medical diagnosis or treatment recommendations. This has spiked interest in deploying relevant algorithms with the aim of enhancing decision-making in healthcare. In this paper, we argue that instead of straightforwardly enhancing the decision-making capabilities of clinicians and healthcare institutions, deploying machines learning algorithms entails trade-offs at the epistemic and the normative level. Whereas involving machine learning might improve the accuracy of medical (...) diagnosis, it comes at the expense of opacity when trying to assess the reliability of given diagnosis. Drawing on literature in social epistemology and moral responsibility, we argue that the uncertainty in question potentially undermines the epistemic authority of clinicians. Furthermore, we elucidate potential pitfalls of involving machine learning in healthcare with respect to paternalism, moral responsibility and fairness. At last, we discuss how the deployment of machine learning algorithms might shift the evidentiary norms of medical diagnosis. In this regard, we hope to lay the grounds for further ethical reflection of the opportunities and pitfalls of machine learning for enhancing decision-making in healthcare. (shrink)

Chronic fatigue syndrome or myalgic encephalomyelitis remains a controversial illness category. This paper surveys the state of knowledge and attitudes about this illness and proposes that epistemic concerns about the testimonial credibility of patients can be articulated using Miranda Fricker’s concept of epistemic injustice. While there is consensus within mainstream medical guidelines that there is no known cause of CFS/ME, there is continued debate about how best to conceive of CFS/ME, including disagreement about how to interpret clinical studies of treatments. (...) Against this background, robust qualitative and quantitative research from a range of countries has found that many doctors display uncertainty about whether CFS/ME is real, which may result in delays in diagnosis and treatment for patients. Strikingly, qualitative research evinces that patients with CFS/ME often experience suspicion by health professionals, and many patients vocally oppose the effectiveness, and the conceptualization, of their illness as psychologically treatable. We address the intersection of these issues and healthcare ethics, and claim that this state of affairs can be explained as a case of epistemic injustice. We find evidence that healthcare consultations are fora where patients with CFS/ME may be particularly vulnerable to epistemic injustice. We argue that the marginalization of many patients is a professional failure that may lead to further ethical and practical consequences both for progressive research into CFS/ME, and for ethical care and delivery of current treatments among individuals suffering from this debilitating illness. (shrink)

Technologies that deploy data science methods are liable to result in epistemic harms involving the diminution of individuals with respect to their standing as knowers or their credibility as sources of testimony. Not all harms of this kind are unjust but when they are we ought to try to prevent or correct them. Epistemically unjust harms will typically intersect with other more familiar and well-studied kinds of harm that result from the design, development, and use of data science technologies. However, (...) we argue that epistemic injustices can be distinguished conceptually from more familiar kinds of harm. We argue that epistemic harms are morally relevant even in cases where those who suffer them are unharmed in other ways. Via a series of examples from the criminal justice system, workplace hierarchies, and educational contexts we explain the kinds of epistemic injustice that can result from common uses of data science technologies. (shrink)

Artificial intelligence (AI) is increasingly being developed for use in medicine, including for diagnosis and in treatment decision making. The use of AI in medical treatment raises many ethical issues that are yet to be explored in depth by bioethicists. In this paper, I focus specifically on the relationship between the ethical ideal of shared decision making and AI systems that generate treatment recommendations, using the example of IBM’s Watson for Oncology. I argue that use of this type of system (...) creates both important risks and significant opportunities for promoting shared decision making. If value judgements are fixed and covert in AI systems, then we risk a shift back to more paternalistic medical care. However, if designed and used in an ethically informed way, AI could offer a potentially powerful way of supporting shared decision making. It could be used to incorporate explicit value reflection, promoting patient autonomy. In the context of medical treatment, we need value-flexible AI that can both respond to the values and treatment goals of individual patients and support clinicians to engage in shared decision making. (shrink)

Many critics of medicalization express concern that the process privileges individualised, biologically grounded interpretations of medicalized phenomena, inhibiting understanding and communication of aspects of those phenomena that are less relevant to their biomedical modelling. I suggest that this line of critique views medicalization as a hermeneutical injustice—a form of epistemic injustice that prevents people having the hermeneutical resources available to interpret and communicate significant areas of their experience. Interpreting the critiques in this fashion shows they frequently fail because they: neglect (...) the ways in which medicalization may not obscure, but rather illuminate, individuals’ experiences; and neglect the testimony of those experiencing first-hand medicalized problems, thus may be guilty of perpetrating testimonial injustice. However, I suggest that such arguments are valuable insofar as they highlight the unwarranted epistemic privilege frequently afforded to medical institutions and medicalized models of phenomena, and a consequent need for greater epistemic humility on the part of health workers and researchers. (shrink)

Machine learning (ML) systems play an increasingly relevant role in medicine and healthcare. As their applications move ever closer to patient care and cure in clinical settings, ethical concerns about the responsibility of their use come to the fore. I analyse an aspect of responsible ML use that bears not only an ethical but also a significant epistemic dimension. I focus on ML systems’ role in mediating patient–physician relations. I thereby consider how ML systems may silence patients’ voices and relativise (...) the credibility of their opinions, which undermines their overall credibility status without valid moral and epistemic justification. More specifically, I argue that withholding credibilitydue tohow ML systems operate can be particularly harmful to patients and, apart from adverse outcomes, qualifies as a form of testimonial injustice. I make my case for testimonial injustice in medical ML by considering ML systems currently used in the USA to predict patients’ risk of misusing opioids (automated Prediction Drug Monitoring Programmes, PDMPs for short). I argue that the locus of testimonial injustice in ML-mediated medical encounters is found in the fact that these systems are treated asmarkers of trustworthinesson which patients’ credibility is assessed. I further show how ML-based PDMPs exacerbate and further propagate social inequalities at the expense of vulnerable social groups. (shrink)