Extrapolating causal effects is becoming an increasingly important kind of inference in Evidence-Based Policy, development economics, and microeconometrics more generally. While several strategies have been proposed to aid with extrapolation, the existing methodological literature has left our understanding of what extrapolation consists of and what constitutes successful extrapolation underdeveloped. This paper addresses this lack in understanding by offering a novel account of successful extrapolation. Building on existing contributions pertaining to the challenges involved in extrapolation, this more nuanced and comprehensive account (...) seeks to provide tools that facilitate the scrutiny of specific extrapolative inferences and general strategies for extrapolation. Offering such resources is important especially in view of the increasing amounts of real-world decision-making in policy, development, and beyond that involve extrapolation. (shrink)

Over the last decade, philosophers of science have extensively criticized the epistemic superiority of randomized controlled trials for testing safety and effectiveness of new drugs, defending instead various forms of evidential pluralism. We argue that scientific methods in regulatory decision-making cannot be assessed in epistemic terms only: there are costs involved. Drawing on the legal distinction between rules and standards, we show that drug regulation based on evidential pluralism has much higher costs than our current RCT-based system. We analyze these (...) costs and advocate for evaluating any scheme for drug regulatory tests in terms of concrete empirical benchmarks, like the error rates of regulatory decisions. (shrink)

Recent work in social epistemology has shown that, in certain situations, less communication leads to better outcomes for epistemic groups. In this paper, we show that, ceteris paribus, a Bayesian agent may believe less strongly that a single agent is biased than that an entire group of independent agents is biased. We explain this initially surprising result and show that it is in fact a consequence one may conceive on the basis of commonsense reasoning.

The new field of meta-research investigates industry bias, publication bias, contradictions between studies, and other trends in medical research. I argue that its findings should be used as meta-evidence for evaluating therapies. ‘Meta-evidence’ is evidence about the support that direct ‘first-order evidence’ provides the hypothesis. I consider three objections to my proposal: the irrelevance objection, the screening-off objection, and the underdetermination objection. I argue that meta-research evidence works by rationally revising our confidence in first-order evidence and, consequently, in the hypothesis—typically, (...) downward. (shrink)

Varied evidence confirms more strongly than less varied evidence, ceteris paribus. This epistemological Variety of Evidence Thesis enjoys widespread intuitive support. We put forward a novel explication of one notion of varied evidence and the Variety of Evidence Thesis within Bayesian models of scientific inference by appealing to measures of entropy. Our explication of the Variety of Evidence Thesis holds in many of our models which also pronounce on disconfirmatory and discordant evidence. We argue that our models pronounce rightly. Against (...) a backdrop of failures of the Variety of Evidence Thesis, the intuitive case for the Variety of Evidence Thesis emerges strengthened. Our models do however not support the general case for the thesis since our explication of it fails to hold in certain cases. The parameter space of this failure is explored and an explanation for the failure is offered. (shrink)

If well-designed, the results of a Randomised Clinical Trial can justify a causal claim between treatment and effect in the study population; however, additional information might be needed to carry over this result to another population. RCTs have been criticized exactly on grounds of failing to provide this sort of information Evidence, inference and enquiry. Oxford University Press, New York, 2011), as well as to black-box important details regarding the mechanisms underpinning the causal law instantiated by the RCT result. On (...) the other side, so-called In Silico Clinical Trials face the same criticisms addressed against standard modelling and simulation techniques, and cannot be equated to experiments Philosophy of molecular medicine: foundational issues in research and practice, Routledge, New York, 2017; Parker in Synthese 169:483–496, 2009; Parke in Philos Sci 81:516–536, 2014; Diez Roux in Am J Epidemiol 181:100–102, 2015 and related discussions in Frigg and Reiss in Synthese 169:593–613, 2009; Winsberg in Synthese 169:575–592, 2009; Beisbart and Norton in Int Stud Philos Sci 26:403–422, 2012). We undertake a formal analysis of both methods in order to identify their distinct contribution to causal inference in the clinical setting. Britton et al.’s study :E2098–E2105, 2013) on the impact of ion current variability on cardiac electrophysiology is used for illustrative purposes. We deduce that, by predicting variability through interpolation, ISCTs aid with problems regarding extrapolation of RCTs results, and therefore in assessing their external validity. Furthermore, ISCTs can be said to encode “thick” causal knowledge —as opposed to “thin” difference-making information inferred from RCTs. Hence, ISCTs and RCTs cannot replace one another but rather, they are complementary in that the former provide information about the determinants of variability of causal effects, while the latter can, under certain conditions, establish causality in the first place. (shrink)

In this paper we consider the problem of how to measure the strength of statistical evidence from the perspective of evidence amalgamation operations. We begin with a fundamental measurement amalgamation principle : for any measurement, the inputs and outputs of an amalgamation procedure must be on the same scale, and this scale must have a meaningful interpretation vis a vis the object of measurement. Using the p value as a candidate evidence measure, we examine various commonly used approaches to amalgamation (...) of evidence across similar studies, including standard forms of meta-analysis. We show that none of these methods satisfies MAP. Thus an underlying measurement problem remains. We argue that a successful approach to evidence amalgamation necessitates a solution to the problem of evidence measurement, and we suggest some lines of reasoning that might guide further work towards this end. (shrink)

Establishing the effectiveness of medical treatments is one of the most important aspects of medical practice. Bradford Hill's viewpoints play an important role in inferring causality in medicine,...

The EBM+ programme is an attempt to improve the way in which present-day evidence-based medicine (EBM) assesses causal claims: according to EBM+, mechanistic studies should be scrutinised alongside association studies. This paper addresses two worries about EBM+: (i) that it is not feasible in practice, and (ii) that it is too malleable, i.e., its results depend on subjective choices that need to be made in order to implement the procedure. Several responses to these two worries are considered and evaluated. The (...) paper also discusses the question of whether we should have confidence in medical interventions, in the light of Stegenga's arguments for medical nihilism. (shrink)

In this paper we argue for a naturalistic solution to some of the methodological controversies in regulatory science, on the basis of two case studies: toxicology and health claim regulation. We analyze the debates related to the scientific evidence that is considered necessary for regulatory decision making in each of those two fields, with a particular attention to the interactions between scientific and regulatory aspects. This analysis allows us to identify two general stances in the debate: a) one that argues (...) for more permissive standards of evidence and for methodological pluralism, and b) an opposing one that not only defends strict evidence requirements but also stipulates the use of one particular scientific methodologies for data generation. We argue that the real-world outcomes produced by alternative regulatory options are a vital piece of information that allows for the empirical assessment of these two stances. In particular, this information on outcomes makes it possible to analyze which standards of evidence and scientific methods generate the most useful knowledge as input for regulatory decision making. Our conclusion is that instead of an a priori selection of methodologies and standards, such decisions ought to be based on empirical evidence related to real-world outcomes. (shrink)

Background: Evidence suggesting adverse drug reactions often emerges unsystematically and unpredictably in form of anecdotal reports, case series and survey data. Safety trials and observational studies also provide crucial information regarding the (un-)safety of drugs. Hence, integrating multiple types of pharmacovigilance evidence is key to minimising the risks of harm. Methods: In previous work, we began the development of a Bayesian framework for aggregating multiple types of evidence to assess the probability of a putative causal link between drugs and side (...) effects. This framework arose out of a philosophical analysis of the Bradford Hill Guidelines. In this article, we expand the Bayesian framework and add “evidential modulators,” which bear on the assessment of the reliability of incoming study results. The overall framework for evidence synthesis, “E-Synthesis”, is then applied to a case study. Results: Theoretically and computationally, E-Synthesis exploits coherence of partly or fully independent evidence converging towards the hypothesis of interest (or of conflicting evidence with respect to it), in order to update its posterior probability. With respect to other frameworks for evidence synthesis, our Bayesian model has the unique feature of grounding its inferential machinery on a consolidated theory of hypothesis confirmation (Bayesian epistemology), and in allowing any data from heterogeneous sources (cell-data, clinical trials, epidemiological studies), and methods (e.g., frequentist hypothesis testing, Bayesian adaptive trials, etc.) to be quantitatively integrated into the same inferential framework. Conclusions: E-Synthesis is highly flexible concerning the allowed input, while at the same time relying on a consistent computational system, that is philosophically and statistically grounded. Furthermore, by introducing evidential modulators, and thereby breaking up the different dimensions of evidence (strength, relevance, reliability), E-Synthesis allows them to be explicitly tracked in updating causal hypotheses. (shrink)

This article examines the relationship between the precautionary principle and the well-known Hill criteria of causation. Some have charged that the Hill criteria are anti-precautionary because the...

According to the variety of evidence thesis items of evidence from independent lines of investigation are more confirmatory, ceteris paribus, than, for example, replications of analogous studies. This thesis is known to fail (Bovens and Hartmann; Claveau). However, the results obtained by Bovens and Hartmann only concern instruments whose evidence is either fully random or perfectly reliable; instead, for Claveau, unreliability is modelled as deterministic bias. In both cases, the unreliable instrument delivers totally irrelevant information. We present a model that (...) formalizes both reliability and unreliability differently. Our instruments either are reliable, but affected by random error, or are biased but not deterministically so.Bovens and Hartmann’s results are counter-intuitive in that in their model a long series of consistent reports from the same instrument does not raise suspicion of ‘too-good-to-be-true’ evidence. This happens precisely because they contemplate neither the role of systematic bias, nor unavoidable random error of reliable instruments. In our model, the variety of evidence thesis fails as well, but the area of failure is considerably smaller than for Bovens and Hartmann and Claveau, and holds for (the majority of) realistic cases (that is, where biased instruments are very biased). The essential mechanism that triggers variety of evidence thesis failure is the rate of false to true positives for the two kinds of instruments. Our emphasis is on modelling beliefs about sources of knowledge and their role in hypothesis confirmation in interaction with dimensions of evidence, such as variety and consistency. (shrink)

Extrapolating causal effects from experiments to novel populations is a common practice in evidence-based-policy, development economics and other social science areas. Drawing on experimental evidence of policy effectiveness, analysts aim to predict the effects of policies in new populations, which might differ importantly from experimental populations. Existing approaches made progress in articulating the sorts of similarities one needs to assume to enable such inferences. It is also recognized, however, that many of these assumptions will remain surrounded by significant uncertainty in (...) practice. Unfortunately, the existing literature says little on how analysts may articulate and manage these uncertainties. This paper aims to make progress on these issues. First, it considers several existing ideas that bear on issues of uncertainty, elaborates the challenges they face, and extracts some useful rationales. Second, it outlines a novel approach, called the support graph approach, that builds on these rationales and allows analysts to articulate and manage uncertainty in extrapolation in a systematic and unified way. (shrink)

We review publications in automated scientific discovery using deep learning, with the aim of shedding light on problems with strong connections to philosophy of science, of physics in particular. We show that core issues of philosophy of science, related, notably, to the nature of scientific theories; the nature of unification; and of causation loom large in scientific deep learning. Therefore, advances in deep learning could, and ideally should, have impact on philosophy of science, and vice versa. We suggest lines of (...) further research, and highlight the role ‘theory-based’ AI could have in future developments of the field. (shrink)

Systems medicine is a promising new paradigm for discovering associations, causal relationships and mechanisms in medicine. But it faces some tough challenges that arise from the use of big data: in particular, the problem of how to integrate evidence and the problem of how to structure the development of models. I argue that objective Bayesian models offer one way of tackling the evidence integration problem. I also offer a general methodology for structuring the development of models, within which the objective (...) Bayesian approach fits rather naturally. (shrink)

The teratogenicity of the Zika virus was considered established in 2016, and is an interesting case because three different sets of causal criteria were used to assess teratogenicity. This paper appeals to the thesis of Russo and Williamson (2007) to devise an epistemological framework that can be used to compare and evaluate sets of causal criteria. The framework can also be used to decide when enough criteria are satisfied to establish causality. Arguably, the three sets of causal criteria considered here (...) offer only a rudimentary assessment of mechanistic studies, and some suggestions are made as to alternative ways to establish causality. (shrink)