There are some philosophical questions that can be answered without attention to the social context in which evidence is produced and distributed.ing away from social context is an excellent way to ignore messy details and lay bare the underlying structure of the limits of inference. Idealization is entirely appropriate when one is essentially asking: In the best of all possible worlds, what am I entitled to infer? Yet, philosophers’ concerns often go beyond this domain. As an example I examine the (...) debate on mechanistic evidence and then reevaluate a canonical case study in this debate. I show that for the assessment of actual evidence, produced in a world that is far from ideal, omission of the social aspects of medical epistemology leads philosophers to draw the wrong lessons from cases they take as paradigmatic cases for their views. I close by arguing that social epistemology provides an avenue to incorporate these complications and provides the necessary framework to understand medical evidence. (shrink)

While practitioners think highly of randomized studies, some philosophers argue that there is no epistemic reason to randomize. Here I show that their arguments do not entail their conclusion. Moreover, I provide novel reasons for randomizing in the context of interventional studies. The overall discussion provides a unified framework for assessing baseline balance, one that holds for interventional and observational studies alike. The upshot: practitioners’ strong preference for randomized studies can be defended in some cases, while still offering a nuanced (...) approach to evidence-appraisal, one where not all non-randomized studies are treated equally. (shrink)

Public policymakers and institutional decision-makers routinely face questions about whether interventions “work”: does universal basic income improve people’s welfare and stimulate entrepreneurial activity? Do gated alleyways reduce burglaries or merely shift the crime burden to neighbouring communities? What is the most cost-effective way to improve students’ reading abilities? These are empirical questions that seem best answered by looking at the world, rather than trusting speculations about what will be effective. Evidence-based policy (EBP) is a movement that concretizes this intuition. It (...) maintains that policy should be based on evidence of “what works.” Not any evidence will do, however. Following on the heels of its intellectual progenitor, evidence-based medicine (EBM), EBP insists on the use of high-quality evidence produced in accordance with rigorous methodological standards. Though intuitively compelling, EBP has attracted significant criticism from methodologists, political scientists, and philosophers. This chapter provides a critical overview of EBP through a philosophical lens, reviewing and discussing some of the most pressing challenges that EBP faces and outlining some proposals for improving it. I first provide a brief overview of EBP and distinguish a broader and narrower understanding of it. I then review existing lines of criticism of EBP, two of which are considered in detail. The first elaborates how EBP struggles with extrapolation, that is, the use of evidence from study populations to make inferences about the effects of policies on novel target populations. The second maintains that EBP’s methodological tenets are deeply entwined with moral and political values, which can threaten EBP’s promise to promote objectivity in policymaking. Finally, I make some proposals for how EBP could be improved going forward. (shrink)

This paper develops an account of how the implementation of ML models into healthcare settings requires revising the methodological apparatus of philosophical bioethics. On this account, ML models are cognitive interventions that provide decision-support to physicians and patients. Due to reliability issues, opaque reasoning processes, and information asymmetries, ML models pose inferential problems for them. These inferential problems lay the grounds for many ethical problems that currently claim centre-stage in the bioethical debate. Accordingly, this paper argues that the best way (...) to make progress in remedying these ethical problems is to distil their epistemic core and to identify appropriate epistemic norms as guardrails. The viability of this approach will be highlighted based on four key issues: trust, responsibility, paternalism, and fairness. In that respect, the paper also contributes to a more pronounced view of the specific methodological challenges for ethical theorizing, when the point of reference are complex statistical models that make predictions, rather than individual agents, acting for reasons. (shrink)

Guido Löhr's recent article makes several insightful and productive suggestions about how to proceed with the empirical study of collective action. However, their critique of the conclusions drawn in Gomez-Lavin & Rachar (2022) is undermined by some issues with the interpretation of the debate and paper. This discussion article clears up those issues, presents new findings from experiments developed in response to Löhr's critiques, reflects on the role of experimental research in the development and refinement of philosophical theories, and adds (...) to Löhr's suggestions about the path forward. (shrink)

Experimental balance is usually understood as the control for the value of the conditions, other than the one under study, which are liable to affect the result of a test. We discuss three different approaches to balance. ‘Millean balance’ requires identifying and equalizing ex ante the value of these conditions in order to conduct solid causal inferences. ‘Fisherian balance’ measures ex post the influence of uncontrolled conditions through the analysis of variance. In ‘efficiency balance’ the value of the antecedent conditions (...) is decided ex ante according to the efficiency they yield in the estimation of the treatment outcome. Against some old arguments by Worrall, we show that in both Fisherian and efficiency balance there are good reasons to randomize the allocation of treatments, in particular when there is no agreement among experimenters as to the antecedent conditions to be controlled for. (shrink)

The new field of meta-research investigates industry bias, publication bias, contradictions between studies, and other trends in medical research. I argue that its findings should be used as meta-evidence for evaluating therapies. ‘Meta-evidence’ is evidence about the support that direct ‘first-order evidence’ provides the hypothesis. I consider three objections to my proposal: the irrelevance objection, the screening-off objection, and the underdetermination objection. I argue that meta-research evidence works by rationally revising our confidence in first-order evidence and, consequently, in the hypothesis—typically, (...) downward. (shrink)

Background: Evidence suggesting adverse drug reactions often emerges unsystematically and unpredictably in form of anecdotal reports, case series and survey data. Safety trials and observational studies also provide crucial information regarding the (un-)safety of drugs. Hence, integrating multiple types of pharmacovigilance evidence is key to minimising the risks of harm. Methods: In previous work, we began the development of a Bayesian framework for aggregating multiple types of evidence to assess the probability of a putative causal link between drugs and side (...) effects. This framework arose out of a philosophical analysis of the Bradford Hill Guidelines. In this article, we expand the Bayesian framework and add “evidential modulators,” which bear on the assessment of the reliability of incoming study results. The overall framework for evidence synthesis, “E-Synthesis”, is then applied to a case study. Results: Theoretically and computationally, E-Synthesis exploits coherence of partly or fully independent evidence converging towards the hypothesis of interest (or of conflicting evidence with respect to it), in order to update its posterior probability. With respect to other frameworks for evidence synthesis, our Bayesian model has the unique feature of grounding its inferential machinery on a consolidated theory of hypothesis confirmation (Bayesian epistemology), and in allowing any data from heterogeneous sources (cell-data, clinical trials, epidemiological studies), and methods (e.g., frequentist hypothesis testing, Bayesian adaptive trials, etc.) to be quantitatively integrated into the same inferential framework. Conclusions: E-Synthesis is highly flexible concerning the allowed input, while at the same time relying on a consistent computational system, that is philosophically and statistically grounded. Furthermore, by introducing evidential modulators, and thereby breaking up the different dimensions of evidence (strength, relevance, reliability), E-Synthesis allows them to be explicitly tracked in updating causal hypotheses. (shrink)

Causal pluralism can be defended not only in respect to causal concepts and methodological guidelines, but also at the finer-grained level of causal inference from a particular source of evidence for causation. An argument for this last variety of pluralism is made based on an analysis of causal inference from randomized experiments (RCTs). Here, the causal interpretation of a statistically significant association can be established via multiple paths of reasoning, each relying on different assumptions and providing distinct elements of information (...) in favour of a causal interpretation. (shrink)

This paper argues that an important type of experiment-target inference, extrapolating causal effects, requires models to be successful. Focusing on extrapolation in Evidence-Based Policy, it is ar...

In the biomedical, behavioural and social sciences, the leading method used to estimate causal effects is commonly randomised controlled trials (RCTs) that are generally viewed as both the source and justification of the most valid evidence. In studying the foundation and theory behind RCTs, the existing literature analyses important single issues and biases in isolation that influence causal outcomes in trials (such as randomisation, statistical probabilities and placebos). The common account of biased causal inference is described in a general way (...) in terms of probabilistic imbalances between trial groups. This paper expands the common account of causal bias by distinguishing between the range of biases arising between trial groups but also within one of the groups or across the entire sample during trial design, implementation and analysis. This is done by providing concrete examples from highly influential RCT studies. In going beyond the existing RCT literature, the paper provides a broader, practice-based account of causal bias that specifies the between-group, within-group and across-group biases that affect the estimated causal results of trials – impacting both the effect size and statistical significance. Within this expanded framework, we can better identify the range of different types of biases we face in practice and address the central question about the overall validity of the RCT method and its causal claims. A study can face several smaller biases (related simultaneously to a smaller sample, smaller estimated effect, greater unblinding etc.) that generally add up to greater aggregate bias. Though difficult to measure precisely, it is important to assess and provide information in studies on how much different sources of bias, combined, can explain the estimated causal effect. The RCT method is thereby often the best we have to inform our policy decisions – and the evidence is strengthened when combined with multiple studies and other methods. Yet there is room for continually improving trials and identifying ways to reduce biases they face and to increase their overall validity. Implications are discussed. (shrink)

There are two important traditions in the philosophy of induction. According to one tradition, which has dominated for the last couple of centuries, inductive arguments are warranted by rules. Bayesianism is the most popular view within this tradition. Rules of induction provide functional accounts of inductive support, but no rule is universal; hence, no rule is by itself an accurate model of inductive support. According to another tradition, inductive arguments are not warranted by rules but by matters of fact. Norton’s (...) material theory of induction (MTI) is an influential view within this tradition. Norton’s MTI fails to provide an account of inductive support, but it provides a good account of inductive warrant that can help us define the domain of validity of each rule of induction. Despite their limitations, both approaches to induction are illuminating important aspects of how relations of inductive support work. In this article I present a hybrid theory of induction (HTI), in which I acknowledge and articulate the role of both rules and matters of fact in our understanding of inductive support. According to the HTI, rules of induction accurately describe relations of inductive support when they are warranted, and a rule of induction is warranted if the right facts about the matter of the induction obtain. Crucially, the HTI allows us to address the main challenges that each tradition faces while retaining their strengths, thus obtaining a functional and accurate account of inductive support. The HTI also provides a useful general framework to examine and understand induction, to make sense of how different rules of induction can coexist and to tackle some problems in epistemology. Moreover, the HTI allows us to clarify and resolve some current debates on induction, like the apparent divide between Norton and rule-based theorists. (shrink)

In this thesis I motivate and develop a Hybrid Theory of Induction (HTI), and I explore some of its virtues and implications. The HTI is a hybrid second-order model of inductive support. It is a "hybrid" model of inductive support because it holds that two ingredients play a necessary role in understanding inductive support: rules and facts. It is a "second-order" model of inductive support because it is a model within which first-order models of inductive support (i.e. logics of induction) (...) can fit. In chapter 1 I argue that we need both rules and facts to play a role in a successful account of inductive support. Rules of induction accurately describe relations of inductive support when they are warranted; facts do the warranting work. I call this type of warrant "factual warrant". The resulting account is both functional and accurate, it helps us make sense of how different rules of induction can coexist and it allows us to resolve some current debates in induction. For the purposes of chapter 1 I adopt an existing binary account of factual warrant. In chapter 2 I develop a Graded account of Factual Warrant (GFW), according to which factual warrant comes in degrees. I integrate the GFW in the HTI. I then show that the GFW illuminates the connection between factual warrant and inductive support, and it can successfully account for the role of idealisations and theory in our understanding of inductive support. In chapter 3 I argue that the HTI is also useful for agents, since it can provide methodological guidance to ensure strong inferences and conceptual guidance to assess the strength of our inferences. Finally, in chapter 4, I explore Bayesian inductive logics from the perspective of the HTI. This analysis brings to light the central role that probability models play in Bayesian inductive logics, offering a logical underpinning for some recent suggestions in Bayesian epistemology. Furthermore, throughout this thesis I analyse in detail three rules of induction from the perspective of the HTI: enumerative induction in chapter 2, causal inference in chapter 3 and Bayesian inductive logics in chapter 4. These analyses illustrate how the HTI can help us think more clearly about rules of induction, offering new tools to tackle existing challenges. (shrink)