Public policymakers and institutional decision-makers routinely face questions about whether interventions “work”: does universal basic income improve people’s welfare and stimulate entrepreneurial activity? Do gated alleyways reduce burglaries or merely shift the crime burden to neighbouring communities? What is the most cost-effective way to improve students’ reading abilities? These are empirical questions that seem best answered by looking at the world, rather than trusting speculations about what will be effective. Evidence-based policy (EBP) is a movement that concretizes this intuition. It (...) maintains that policy should be based on evidence of “what works.” Not any evidence will do, however. Following on the heels of its intellectual progenitor, evidence-based medicine (EBM), EBP insists on the use of high-quality evidence produced in accordance with rigorous methodological standards. Though intuitively compelling, EBP has attracted significant criticism from methodologists, political scientists, and philosophers. This chapter provides a critical overview of EBP through a philosophical lens, reviewing and discussing some of the most pressing challenges that EBP faces and outlining some proposals for improving it. I first provide a brief overview of EBP and distinguish a broader and narrower understanding of it. I then review existing lines of criticism of EBP, two of which are considered in detail. The first elaborates how EBP struggles with extrapolation, that is, the use of evidence from study populations to make inferences about the effects of policies on novel target populations. The second maintains that EBP’s methodological tenets are deeply entwined with moral and political values, which can threaten EBP’s promise to promote objectivity in policymaking. Finally, I make some proposals for how EBP could be improved going forward. (shrink)

I consider recent strategies proposed by econometricians for extrapolating causal effects from experimental to target populations. I argue that these strategies fall prey to the extrapolator’s circle: they require so much knowledge about the target population that the causal effects to be extrapolated can be identified from information about the target alone. I then consider comparative process tracing as a potential remedy. Although specifically designed to evade the extrapolator’s circle, I argue that CPT is unlikely to facilitate extrapolation in typical (...) econometrics and evidence-based policy applications. To argue this, I offer a distinction between two kinds of extrapolation, attributive and predictive, the latter being prevalent in econometrics and evidence-based policy. I argue that CPT is not helpful for predictive extrapolation when using the kinds of evidence that econometricians and evidence-based policy researchers prefer. I suggest that econometricians may need to consider qualitative evidence to overcome this problem. (shrink)

Francesco Guala once wrote that ‘The problem of extrapolation is a minor scandal in the philosophy of science’. This paper agrees with the statement, but for reasons different from Guala’s. The scandal is not, or not any longer, that the problem has been ignored in the philosophy of science. The scandal is that framing the problem as one of external validity encourages poor evidential reasoning. The aim of this paper is to propose an alternative—an alternative which constitutes much better evidential (...) reasoning about target systems of interest, and which makes do without consideration of external validity. (shrink)

Extrapolating causal effects is becoming an increasingly important kind of inference in Evidence-Based Policy, development economics, and microeconometrics more generally. While several strategies have been proposed to aid with extrapolation, the existing methodological literature has left our understanding of what extrapolation consists of and what constitutes successful extrapolation underdeveloped. This paper addresses this lack in understanding by offering a novel account of successful extrapolation. Building on existing contributions pertaining to the challenges involved in extrapolation, this more nuanced and comprehensive account (...) seeks to provide tools that facilitate the scrutiny of specific extrapolative inferences and general strategies for extrapolation. Offering such resources is important especially in view of the increasing amounts of real-world decision-making in policy, development, and beyond that involve extrapolation. (shrink)

This chapter provides an introduction to the study of the philosophical notions of mechanisms and causality in biology and economics. This chapter sets the stage for this volume, Mechanism and Causality in Biology and Economics, in three ways. First, it gives a broad review of the recent changes and current state of the study of mechanisms and causality in the philosophy of science. Second, consistent with a recent trend in the philosophy of science to focus on scientific practices, it in (...) turn implies the importance of studying the scientific methods employed by researchers. Finally, by way of providing an overview of each chapter in the volume, this chapter demonstrates that biology and economics are two fertile fields for the philosophy of science and shows how biological and economic mechanisms and causality can be synthesized. (shrink)