We propose a formal framework for interpretable machine learning. Combining elements from statistical learning, causal interventionism, and decision theory, we design an idealised explanation game in which players collaborate to find the best explanation for a given algorithmic prediction. Through an iterative procedure of questions and answers, the players establish a three-dimensional Pareto frontier that describes the optimal trade-offs between explanatory accuracy, simplicity, and relevance. Multiple rounds are played at different levels of abstraction, allowing the players to explore overlapping causal (...) patterns of variable granularity and scope. We characterise the conditions under which such a game is almost surely guaranteed to converge on a optimal explanation surface in polynomial time, and highlight obstacles that will tend to prevent the players from advancing beyond certain explanatory thresholds. The game serves a descriptive and a normative function, establishing a conceptual space in which to analyse and compare existing proposals, as well as design new and improved solutions. (shrink)

Scientists depend on complex computational systems that are often ineliminably opaque, to the detriment of our ability to give scientific explanations and detect artifacts. Some philosophers have s...

We are sceptical of concerns over the opacity of algorithmic decision tools. While transparency and explainability are certainly important desiderata in algorithmic governance, we worry that automated decision-making is being held to an unrealistically high standard, possibly owing to an unrealistically high estimate of the degree of transparency attainable from human decision-makers. In this paper, we review evidence demonstrating that much human decision-making is fraught with transparency problems, show in what respects AI fares little worse or better and argue that (...) at least some regulatory proposals for explainable AI could end up setting the bar higher than is necessary or indeed helpful. The demands of practical reason require the justification of action to be pitched at the level of practical reason. Decision tools that support or supplant practical reasoning should not be expected to aim higher than this. We cast this desideratum in terms of Daniel Dennett’s theory of the “intentional stance” and argue that since the justification of action for human purposes takes the form of intentional stance explanation, the justification of algorithmic decisions should take the same form. In practice, this means that the sorts of explanations for algorithmic decisions that are analogous to intentional stance explanations should be preferred over ones that aim at the architectural innards of a decision tool. (shrink)

Machine learning algorithms may radically improve our ability to diagnose and treat disease. For moral, legal, and scientific reasons, it is essential that doctors and patients be able to understand and explain the predictions of these models. Scalable, customisable, and ethical solutions can be achieved by working together with relevant stakeholders, including patients, data scientists, and policy makers.

Many of the computing systems programmed using Machine Learning are opaque: it is difficult to know why they do what they do or how they work. Explainable Artificial Intelligence aims to develop analytic techniques that render opaque computing systems transparent, but lacks a normative framework with which to evaluate these techniques’ explanatory successes. The aim of the present discussion is to develop such a framework, paying particular attention to different stakeholders’ distinct explanatory requirements. Building on an analysis of “opacity” from (...) philosophy of science, this framework is modeled after accounts of explanation in cognitive science. The framework distinguishes between the explanation-seeking questions that are likely to be asked by different stakeholders, and specifies the general ways in which these questions should be answered so as to allow these stakeholders to perform their roles in the Machine Learning ecosystem. By applying the normative framework to recently developed techniques such as input heatmapping, feature-detector visualization, and diagnostic classification, it is possible to determine whether and to what extent techniques from Explainable Artificial Intelligence can be used to render opaque computing systems transparent and, thus, whether they can be used to solve the Black Box Problem. (shrink)

Deep learning is currently the most prominent and widely successful method in artificial intelligence. Despite having played an active role in earlier artificial intelligence and neural network research, philosophers have been largely silent on this technology so far. This is remarkable, given that deep learning neural networks have blown past predicted upper limits on artificial intelligence performance—recognizing complex objects in natural photographs and defeating world champions in strategy games as complex as Go and chess—yet there remains no universally accepted explanation (...) as to why they work so well. This article provides an introduction to these networks as well as an opinionated guidebook on the philosophical significance of their structure and achievements. It argues that deep learning neural networks differ importantly in their structure and mathematical properties from the shallower neural networks that were the subject of so much philosophical reflection in the 1980s and 1990s. The article then explores several different explanations for their success and ends by proposing three areas of inquiry that would benefit from future engagement by philosophers of mind and science. (shrink)

In this paper I argue that the search for explainable models and interpretable decisions in AI must be reformulated in terms of the broader project of offering a pragmatic and naturalistic account of understanding in AI. Intuitively, the purpose of providing an explanation of a model or a decision is to make it understandable to its stakeholders. But without a previous grasp of what it means to say that an agent understands a model or a decision, the explanatory strategies will (...) lack a well-defined goal. Aside from providing a clearer objective for XAI, focusing on understanding also allows us to relax the factivity condition on explanation, which is impossible to fulfill in many machine learning models, and to focus instead on the pragmatic conditions that determine the best fit between a model and the methods and devices deployed to understand it. After an examination of the different types of understanding discussed in the philosophical and psychological literature, I conclude that interpretative or approximation models not only provide the best way to achieve the objectual understanding of a machine learning model, but are also a necessary condition to achieve post hoc interpretability. This conclusion is partly based on the shortcomings of the purely functionalist approach to post hoc interpretability that seems to be predominant in most recent literature. (shrink)

Recent work on interpretability in machine learning and AI has focused on the building of simplified models that approximate the true criteria used to make decisions. These models are a useful pedagogical device for teaching trained professionals how to predict what decisions will be made by the complex system, and most importantly how the system might break. However, when considering any such model it’s important to remember Box’s maxim that "All models are wrong but some are useful." We focus on (...) the distinction between these models and explanations in philosophy and sociology. These models can be understood as a "do it yourself kit" for explanations, allowing a practitioner to directly answer "what if questions" or generate contrastive explanations without external assistance. Although a valuable ability, giving these models as explanations appears more difficult than necessary, and other forms of explanation may not have the same trade-offs. We contrast the different schools of thought on what makes an explanation, and suggest that machine learning might benefit from viewing the problem more broadly. (shrink)

Understanding is a central aim of science and highly important in present-day society. But what precisely is scientific understanding and how can it be achieved? This book answers these questions, through philosophical analysis and historical case studies, and presents a philosophical theory of scientific understanding that highlights its contextual nature.

Scientific explanations must bear the proper relationship to the world: they must depict what, out in the world, is responsible for the explanandum. But explanations must also bear the proper relationship to their audience: they must be able to create human understanding. With few exceptions, philosophical accounts of explanation either ignore entirely the relationship between explanations and their audience or else demote this consideration to an ancillary role. In contrast, I argue that considering an explanation’s communicative role is crucial to (...) any satisfactory account of explanation. (shrink)

I discuss the bearing on the mind-body problem of some general characteristics of living systems, including the physical basis of metabolism and the relation between living activity and cognitive capacities in simple organisms. I then attempt to describe stages in the history of animal life important to the evolution of subjective experience. Features of the biological basis of cognition are used to criticize arguments against materialism that draw on the conceivability of a separation between mental and physical. I also argue (...) against commonly held views about the "multiple realizability" of mental states of the kind seen in humans. The aim of the paper is to reconfigure and narrow the "explanatory gap" between mental and physical. (shrink)

An analysis of two heuristic strategies for the development of mechanistic models, illustrated with historical examples from the life sciences. In Discovering Complexity, William Bechtel and Robert Richardson examine two heuristics that guided the development of mechanistic models in the life sciences: decomposition and localization. Drawing on historical cases from disciplines including cell biology, cognitive neuroscience, and genetics, they identify a number of "choice points" that life scientists confront in developing mechanistic explanations and show how different choices result in divergent (...) explanatory models. Describing decomposition as the attempt to differentiate functional and structural components of a system and localization as the assignment of responsibility for specific functions to specific structures, Bechtel and Richardson examine the usefulness of these heuristics as well as their fallibility—the sometimes false assumption underlying them that nature is significantly decomposable and hierarchically organized. When Discovering Complexity was originally published in 1993, few philosophers of science perceived the centrality of seeking mechanisms to explain phenomena in biology, relying instead on the model of nomological explanation advanced by the logical positivists (a model Bechtel and Richardson found to be utterly inapplicable to the examples from the life sciences in their study). Since then, mechanism and mechanistic explanation have become widely discussed. In a substantive new introduction to this MIT Press edition of their book, Bechtel and Richardson examine both philosophical and scientific developments in research on mechanistic models since 1993. (shrink)

As Aristotle stated, scientific explanation is based on deductive argument--yet, Wesley C. Salmon points out, not all deductive arguments are qualified explanations. The validity of the explanation must itself be examined. _Four Decades of Scientific Explanation_ provides a comprehensive account of the developments in scientific explanation that transpired in the last four decades of the twentieth century. It continues to stand as the most comprehensive treatment of the writings on the subject during these years. Building on the historic 1948 essay (...) by Carl G. Hempel and Paul Oppenheim, "Studies in the Logic of Explanation,” which introduced the deductive-nomological model on which most work on scientific explanation was based for the following four decades, Salmon goes beyond this model's inherent basis of describing empirical knowledge to tells us “not only _what,_ but also _why_.” Salmon examines the predominant models in chronological order and describes their development, refinement, and criticism or rejection. _Four Decades of Scientific Explanation_ underscores the need for a consensus of approach and ongoing evaluations of methodology in scientific explanation, with the goal of providing a better understanding of natural phenomena. (shrink)

I distinguish three theses associated with the new mechanistic philosophy – concerning causation, explanation and scientific methodology. Advocates of each thesis are identified and relationships among them are outlined. I then look at some recent work on natural selection and mechanisms. There, attention to different kinds of New Mechanism significantly affects of what is at stake.

Scientific understanding, this paper argues, can be analyzed entirely in terms of a mental act of “grasping” and a notion of explanation. To understand why a phenomenon occurs is to grasp a correct explanation of the phenomenon. To understand a scientific theory is to be able to construct, or at least to grasp, a range of potential explanations in which that theory accounts for other phenomena. There is no route to scientific understanding, then, that does not go by way of (...) scientific explanation. (shrink)

The chapters in this book highlight the multifaceted nature of the process of scientific research.

The philosophical theory of scientific explanation proposed here involves a radically new treatment of causality that accords with the pervasively statistical character of contemporary science. Wesley C. Salmon describes three fundamental conceptions of scientific explanation--the epistemic, modal, and ontic. He argues that the prevailing view is untenable and that the modal conception is scientifically out-dated. Significantly revising aspects of his earlier work, he defends a causal/mechanical theory that is a version of the ontic conception. Professor Salmon's theory furnishes a robust (...) argument for scientific realism akin to the argument that convinced twentieth-century physical scientists of the existence of atoms and molecules. To do justice to such notions as irreducibly statistical laws and statistical explanation, he offers a novel account of physical randomness. The transition from the "reviewed view" of scientific explanation to the causal/mechanical model requires fundamental rethinking of basic explanatory concepts. (shrink)

Approaches to explanation -- Causal and explanatory relevance -- The kairetic account of /D making -- The kairetic account of explanation -- Extending the kairetic account -- Event explanation and causal claims -- Regularity explanation -- Abstraction in regularity explanation -- Approaches to probabilistic explanation -- Kairetic explanation of frequencies -- Kairetic explanation of single outcomes -- Looking outward -- Looking inward.

Woodward's long awaited book is an attempt to construct a comprehensive account of causation explanation that applies to a wide variety of causal and explanatory claims in different areas of science and everyday life. The book engages some of the relevant literature from other disciplines, as Woodward weaves together examples, counterexamples, criticisms, defenses, objections, and replies into a convincing defense of the core of his theory, which is that we can analyze causation by appeal to the notion of manipulation.

Carl Craver investigates what we are doing when we sue neuroscience to explain what's going on in the brain.

Offering a new approach to scientific explanation, this book focuses initially on the explaining act itself.

Issues concerning scientific explanation have been a focus of philosophical attention from Pre- Socratic times through the modern period. However, recent discussion really begins with the development of the Deductive-Nomological (DN) model. This model has had many advocates (including Popper 1935, 1959, Braithwaite 1953, Gardiner, 1959, Nagel 1961) but unquestionably the most detailed and influential statement is due to Carl Hempel (Hempel 1942, 1965, and Hempel & Oppenheim 1948). These papers and the reaction to them have structured subsequent discussion concerning (...) scientific explanation to an extraordinary degree. After some general remarks by way of background and orientation (Section 1), this entry describes the DN model and its extensions, and then turns to some well-known objections (Section 2). It next describes a variety of subsequent attempts to develop alternative models of explanation, including Wesley Salmon's Statistical Relevance (Section 3) and Causal Mechanical (Section 4) models and the Unificationist models due to Michael Friedman and Philip Kitcher (Section 5). Section 6 provides a summary and discusses directions for future work. (shrink)

To explain the phenomena in the world of our experience, to answer the question “why?” rather than only the question “what?”, is one of the foremost objectives of all rational inquiry; and especially, scientific research in its various branches strives to go beyond a mere description of its subject matter by providing an explanation of the phenomena it investigates. While there is rather general agreement about this chief objective of science, there exists considerable difference of opinion as to the function (...) and the essential characteristics of scientific explanation. In the present essay, an attempt will be made to shed some light on these issues by means of an elementary survey of the basic pattern of scientific explanation and a subsequent more rigorous analysis of the concept of law and of the logical structure of explanatory arguments. (shrink)

The concept of mechanism is analyzed in terms of entities and activities, organized such that they are productive of regular changes. Examples show how mechanisms work in neurobiology and molecular biology. Thinking in terms of mechanisms provides a new framework for addressing many traditional philosophical issues: causality, laws, explanation, reduction, and scientific change.

We propose a formal framework for interpretable machine learning. Combining elements from statistical learning, causal interventionism, and decision theory, we design an idealisedexplanation gamein which players collaborate to find the best explanation(s) for a given algorithmic prediction. Through an iterative procedure of questions and answers, the players establish a three-dimensional Pareto frontier that describes the optimal trade-offs between explanatory accuracy, simplicity, and relevance. Multiple rounds are played at different levels of abstraction, allowing the players to explore overlapping causal patterns of (...) variable granularity and scope. We characterise the conditions under which such a game is almost surely guaranteed to converge on a (conditionally) optimal explanation surface in polynomial time, and highlight obstacles that will tend to prevent the players from advancing beyond certain explanatory thresholds. The game serves a descriptive and a normative function, establishing a conceptual space in which to analyse and compare existing proposals, as well as design new and improved solutions. (shrink)

From antiquity to the end of the twentieth century, philosophical discussions of understanding remained undeveloped, guided by a 'received view' that takes understanding to be nothing more than knowledge of an explanation. More recently, however, this received view has been criticized, and bold new philosophical proposals about understanding have emerged in its place. In this book, Kareem Khalifa argues that the received view should be revised but not abandoned. In doing so, he clarifies and answers the most central questions in (...) this burgeoning field of philosophical research: what kinds of cognitive abilities are involved in understanding? What is the relationship between the understanding that explanations provide and the understanding that experts have of broader subject matters? Can there be understanding without explanation? How can one understand something on the basis of falsehoods? Is understanding a species of knowledge? What is the value of understanding? (shrink)

The usefulness of machine learning algorithms has led to their widespread adoption prior to the development of a conceptual framework for making sense of them. One common response to this situation is to say that machine learning suffers from a “black box problem.” That is, machine learning algorithms are “opaque” to human users, failing to be “interpretable” or “explicable” in terms that would render categorization procedures “understandable.” The purpose of this paper is to challenge the widespread agreement about the existence (...) and importance of a black box problem. The first section argues that “interpretability” and cognates lack precise meanings when applied to algorithms. This makes the concepts difficult to use when trying to solve the problems that have motivated the call for interpretability. Furthermore, since there is no adequate account of the concepts themselves, it is not possible to assess whether particular technical features supply formal definitions of those concepts. The second section argues that there are ways of being a responsible user of these algorithms that do not require interpretability. In many cases in which a black box problem is cited, interpretability is a means to a further end such as justification or non-discrimination. Since addressing these problems need not involve something that looks like an “interpretation” of an algorithm, the focus on interpretability artificially constrains the solution space by characterizing one possible solution as the problem itself. Where possible, discussion should be reformulated in terms of the ends of interpretability. (shrink)

Achieving understanding of nature is one of the aims of science. In this paper we offer an analysis of the nature of scientific understanding that accords with actual scientific practice and accommodates the historical diversity of conceptions of understanding. Its core idea is a general criterion for the intelligibility of scientific theories that is essentially contextual: which theories conform to this criterion depends on contextual factors, and can change in the course of time. Our analysis provides a general account of (...) how understanding is provided by scientific explanations of diverse types. In this way, it reconciles conflicting views of explanatory understanding, such as the causal-mechanical and the unificationist conceptions. (shrink)

This article argues that the basic account of mechanism and mechanistic explanation, involving sequential execution of qualitatively characterized operations, is itself insufficient to explain biological phenomena such as the capacity of living organisms to maintain themselves as systems distinct from their environment. This capacity depends on cyclic organization, including positive and negative feedback loops, which can generate complex dynamics. Understanding cyclically organized mechanisms with complex dynamics requires coordinating research directed at decomposing mechanisms into parts and operations with research using computational (...) models to recompose mechanisms and determine their dynamic behavior. This coordinated endeavor yields dynamic mechanistic explanations. (shrink)

We propose a formal framework for interpretable machine learning. Combining elements from statistical learning, causal interventionism, and decision theory, we design an idealised explanation game in which players collaborate to find the best explanation for a given algorithmic prediction. Through an iterative procedure of questions and answers, the players establish a three-dimensional Pareto frontier that describes the optimal trade-offs between explanatory accuracy, simplicity, and relevance. Multiple rounds are played at different levels of abstraction, allowing the players to explore overlapping causal (...) patterns of variable granularity and scope. We characterise the conditions under which such a game is almost surely guaranteed to converge on a optimal explanation surface in polynomial time, and highlight obstacles that will tend to prevent the players from advancing beyond certain explanatory thresholds. The game serves a descriptive and a normative function, establishing a conceptual space in which to analyse and compare existing proposals, as well as design new and improved solutions. (shrink)

Biological processes are often explained by identifying the underlying mechanisms that generate a phenomenon of interest. I characterize a basic account of mechanistic explanation and then present three challenges to this account, illustrated with examples from molecular biology. The basic mechanistic account is insufficient for explaining nonsequential and nonlinear dynamic processes, is insufficient for explaining the inherently stochastic nature of many biological mechanisms, and fails to give a proper framework for analyzing organization. I suggest that biological processes are best approached (...) as a multidimensional gradient—with some processes being paradigmatic cases of mechanisms and some being marginal cases. (shrink)

In this paper I argue that the search for explainable models and interpretable decisions in AI must be reformulated in terms of the broader project of offering a pragmatic and naturalistic account of understanding in AI. Intuitively, the purpose of providing an explanation of a model or a decision is to make it understandable to its stakeholders. But without a previous grasp of what it means to say that an agent understands a model or a decision, the explanatory strategies will (...) lack a well-defined goal. Aside from providing a clearer objective for XAI, focusing on understanding also allows us to relax the factivity condition on explanation, which is impossible to fulfill in many machine learning models, and to focus instead on the pragmatic conditions that determine the best fit between a model and the methods and devices deployed to understand it. After an examination of the different types of understanding discussed in the philosophical and psychological literature, I conclude that interpretative or approximation models not only provide the best way to achieve the objectual understanding of a machine learning model, but are also a necessary condition to achieve post hoc interpretability. This conclusion is partly based on the shortcomings of the purely functionalist approach to post hoc interpretability that seems to be predominant in most recent literature. (shrink)

Although decision‐making algorithms are not new to medicine, the availability of vast stores of medical data, gains in computing power, and breakthroughs in machine learning are accelerating the pace of their development, expanding the range of questions they can address, and increasing their predictive power. In many cases, however, the most powerful machine learning techniques purchase diagnostic or predictive accuracy at the expense of our ability to access “the knowledge within the machine.” Without an explanation in terms of reasons or (...) a rationale for particular decisions in individual cases, some commentators regard ceding medical decision‐making to black box systems as contravening the profound moral responsibilities of clinicians. I argue, however, that opaque decisions are more common in medicine than critics realize. Moreover, as Aristotle noted over two millennia ago, when our knowledge of causal systems is incomplete and precarious—as it often is in medicine—the ability to explain how results are produced can be less important than the ability to produce such results and empirically verify their accuracy. (shrink)

Between 1940 and 1970 pioneers in the new field of cell biology discovered the operative parts of cells and their contributions to cell life. They offered mechanistic accounts that explained cellular phenomena by identifying the relevant parts of cells, the biochemical operations they performed, and the way in which these parts and operations were organised to accomplish important functions. Cell biology was a revolutionary science but in this book it also provides fuel for yet another revolution, one that focuses on (...) the very conception of science itself. Laws have traditionally been regarded as the primary vehicle of explanation, but in the emerging philosophy of science it is mechanisms that do the explanatory work. Bechtel emphasises how mechanisms were discovered, focusing especially on the way in which new instruments made these inquiries possible. He also describes how new journals and societies provided institutional structure to this new enterprise. (shrink)

Between 1940 and 1970 pioneers in the new field of cell biology discovered the operative parts of cells and their contributions to cell life. They offered mechanistic accounts that explained cellular phenomena by identifying the relevant parts of cells, the biochemical operations they performed, and the way in which these parts and operations were organised to accomplish important functions. Cell biology was a revolutionary science but in this book it also provides fuel for yet another revolution, one that focuses on (...) the very conception of science itself. Laws have traditionally been regarded as the primary vehicle of explanation, but in the emerging philosophy of science it is mechanisms that do the explanatory work. Bechtel emphasises how mechanisms were discovered, focusing especially on the way in which new instruments made these inquiries possible. He also describes how new journals and societies provided institutional structure to this new enterprise. (shrink)