The explanatory/pragmatic-trial distinction enjoys a burgeoning philosophical and medical literature and a significant contingent of support among philosophers and healthcare stakeholders as an important way to assess the design and results of randomized controlled trials. A major motivation has been the need to provide relevant, generalizable data to drive healthcare decisions. While talk of pragmatic and explanatory trials could be seen as convenient shorthand, the distinction can also be seen as harboring deeper issues related to inferential strategies used to evaluate (...) causal claims regarding medical treatments. A comprehensive, critical analysis of the distinction and underlying epistemological framework upon which the distinction is based, particularly with respect to treatment effectiveness, has yet to be forthcoming. I provide this, analyzing the distinction’s relationship to generalizability and cognate distinctions between ideal conditions and real-world practice, internal and external validity, and efficacy and effectiveness. I also analyze recent philosophical work that relies on the explanatory/pragmatic-trial distinction and that advocates for more pragmatic trials. I conclude that as an organizing principle for trial-design decisions and trial evaluation, the explanatory/pragmatic-trial distinction is conceptually problematic and not as useful as its proponents seem to think. Since some pragmatic-trial features can be inimical to establishing treatment effectiveness, pragmatic-trial features should not be conflated with pragmatic trials’ avowed goals. If the distinction is to be useful, it and some associated concepts, including generalizability, should be reformulated, lest they continue to underlie a medical epistemology that could contribute to methodologically flawed and potentially unethical advice for the design and interpretation of trials. (shrink)

I argue that non-epistemic values are necessarily embedded in the measure of evidential strength of contrastive evidence. When evidence is contrastive, evidence is stronger the more it favours a hypothesis over a set of plausible, mutually exclusive alternative hypotheses. In such a contrastive epistemic setting, evidence has an effect not only on a particular hypothesis, but on the whole probability distribution over the set of alternative hypotheses. A natural way of analysing the incremental impact of new evidence on a set (...) of alternative hypotheses is in terms of uncertainty or ‘entropy’ reduction. There is no unique single measure of uncertainty/entropy, however, and, consequently, no single unique measure of uncertainty reduction. Finally, I argue that the ‘right’ measure of entropy reduction, and hence of evidential strength, depends on the pragmatic context and non-epistemic values. (shrink)

Acceding to the demand that public policy should be based on “the best available evidence” can come at significant moral cost. Important policy questions cannot be addressed using “the best available evidence” as defined by the evidence-based policy paradigm; the paradigm can change the meaning of questions so that they can be addressed using the preferred kind of evidence; and important evidence that does not meet the standard defined by the paradigm can get ignored. We illustrate these problems in three (...) contexts of evidence-based policy. (shrink)

Philosophy of epidemiology has recently emerged as a distinct branch of philosophy. The field will surely benefit from pluralism, reflected in the broad range of topics and perspectives in this special issue. Here, I argue that a healthy pluralistic field of philosophy of epidemiology has social responsibilities that require the field as a whole to engage actively with research on social determinants of health and health disparities. Practicing epidemiologists and the broader community of public health scientists have gradually acknowledged that (...) much of their attention ought to be paid to these, i.e. inequitable between-population health variations that largely reflect the world’s inequitable distribution of resources and social conditions. The paper illustrates how and why health disparities and social determinants of health, through no ill will, become de facto secondary concerns in Alex Broadbent’s field-defining Philosophy of Epidemiology, showing the ease with which these topics can be incidentally sidelined. As means for philosophy of epidemiology to meet its social responsibility obligations, I suggest philosophy of epidemiology expand its attention to two particular lines research. First, the paper discusses Geoffrey Rose’s concept of “causes of incidence”—causes of disparities between populations—and argues for the importance of engaging with these causes. Second, the paper argues for the value of engaging with Bruce Link and Jo Phelan’s “fundamental causes” model of how flexible social resources serve as buffers from harms, generating a plethora of shifting and locally-contingent health effects. (shrink)

*PSA 2016, symposium on “Data in Time: Epistemology of Historical Data” organized by Sabina Leonelli, 5 November 2016* *See published version: "Radiocarbon Dating in Archaeology: Triangulation and Traceability" in Data Journeys in the Sciences (2020) - link below* Archaeologists put a premium on pressing “legacy data” into service, given the notoriously selective and destructive nature of their practices of data capture. Legacy data consist of material and records that been assembled over decades, sometimes centuries, often by means and for purposes (...) long since discredited or superseded. The primary strategies by which archaeologists put the data to work for new purposes are, I argue, secondary retrieval, recontextualization, and experimental modelling. I focus here on a particularly telling and complex example of secondary retrieval: the extraction of new data from old by means of radiocarbon dating. This is by no means a straightforward process of retrieving physical samples from legacy data to which 14C techniques can be applied that can, on their own, decisively settle chronological questions. When Libby’s post-war radiocarbon revolution got under way, it was expected to establish an absolute chronology that would render obsolete the local and relative chronologies on which archaeologists had long relied. Transformative though it has been, bringing these tools of physical dating to bear on archaeological problems has been a long, tortuous process, now described as proceeding through two subsequent radiocarbon revolutions. The second was an extended process of calibration by which 14C chronologies were corrected and refined, often against the very lines of evidence they were meant to displace. The most recent, a “pragmatic Bayesian” approach to archaeological dating, is motivated by concern that, no matter how much it is refined, radiocarbon dating cannot on its own resolve the chronological problems that archaeologists address; the challenge, its advocates argue, is to ‘fully integrate archaeological information with 14C dating in order to address archaeologically relevant timescales and episodes’. This is a genre of “robustness” reasoning that illustrates its epistemic risks as well as its appeal. As recent philosophical debate makes clear, it depends on appeals to the convergence of independent lines of evidence that may have more rhetorical than epistemic force and that may be spurious. Drawing on this philosophical literature I identify a set of conditions that must be met if these risks are to be avoided, all of which are an explicit focus of debate in cases of contestation about and reconciliation of chronologies based on legacy data. (shrink)

This chapter addresses the problem of ranking available drugs in guideline development to support clinicians in their work. Based on a pragmatic approach to the notion of evidence and a hierarchical view on different kinds of evidence this chapter introduces a decision aid, HiDAD, which draws on the multi criteria decision making literature. This decision aid implements the wide-spread intuition that there are different kinds of evidence with varying degrees of importance by relying on a strict ordinal ordering of kinds (...) of evidence. In order to construct a ranking every pair of drugs is first compared separately on all kinds of evidence. Next, these quantitative comparisons are then aggregated into an overall comparison between drugs based on all the available evidence in a way which avoids that evidence of less importance is trumped by evidence of the higher levels. Finally, these overall comparisons are used to determine the final ranking of drugs which then informs the process of guideline writing. Properties, modifications and applicability of the decision aid HiDAD are discussed and assessed. (shrink)

In 2015, Tomasetti and Vogelstein published a paper in Science containing the following provocative statement: “… only a third of the variation in cancer risk among tissues is attributable to environmental factors or inherited predispositions. The majority is due to “bad luck,” that is, random mutations arising during DNA replication in normal, noncancerous stem cells.” The paper—and perhaps especially this rather coy reference to “bad luck”—became a flash point for a series of letters and reviews, followed by replies and yet (...) further counterpoints. In this paper, I critically assess Tomasetti and Vogelstein's argument, discuss the meaning of “luck” (or, better: “chance”) in the context of the debate, and use this case study to address larger questions about methodological criteria for causal explanations of population level patterns in biomedicine. (shrink)

Establishing the effectiveness of medical treatments is one of the most important aspects of medical practice. Bradford Hill's viewpoints play an important role in inferring causality in medicine,...

Neuroimaging technology is the most widely used tool to study human cognition. While originally a promising tool for mapping the content of cognitive theories onto the structures of the brain, recently developed tools for the analysis, handling and sharing of data have changed the theoretical landscape of cognitive neuroscience. Even with these advancements philosophical analyses of evidence in neuroimaging remain skeptical of the promise of neuroimaging technology. These views often treat the analysis techniques used to make sense of data produced (...) in a neuroimaging experiment as one, attributing the inferential limitations of analysis pipelines to the technology as a whole. Situated against the neuroscientists own critical assessment of their methods and the limitations of those methods, this skepticism appears based on a misunderstanding of the role data analysis techniques play in neuroimaging research. My project picks up here, examining how data analysis techniques, such as pattern classification analysis, are used to assess the evidential value of neuroimaging data. The project takes the form of three papers. In the first I identify the use of multiple data analysis techniques as an important aspect of the data interpretation process that is overlooked by critics. In the second I develop an account of inferences in neuroimaging research that is sensitive to this use of data analysis techniques, arguing that interpreting neuroimaging data is a process of isolating and explaining a variety of data patterns. In the third I argue that the development and uptake of new techniques for analyzing data must be accompanied by changes in research practices and standards of evidence if they are to promote knowledge generation. My approach to this work is both traditionally philosophical, insofar as it involves reading and analyzing the work of philosophers and neuroscientists, and embedded insofar as most of the research was conducted while engaged in attending lab meetings and participating in the work of those scientists whose work is the object of my research. (shrink)

In this talk I present the main results from Anta (2021), namely, that the theoretical division between Boltzmannian and Gibbsian statistical mechanics should be understood as a separation in the epistemic capabilities of this physical discipline. In particular, while from the Boltzmannian framework one can generate powerful explanations of thermal processes by appealing to their microdynamics, from the Gibbsian framework one can predict observable values in a computationally effective way. Finally, I argue that this statistical mechanical schism contradicts the Hempelian (...) (1958) thesis that the predictive power of a scientific theory is directly proportional to its explanatory potential, and vice versa. (shrink)

Amalgamating evidence from heterogeneous sources and across levels of inquiry is becoming increasingly important in many pure and applied sciences. This special issue provides a forum for researchers from diverse scientific and philosophical perspectives to discuss evidence amalgamation, its methodologies, its history, its pitfalls, and its potential. We situate the contributions therein within six themes from the broad literature on this subject: the variety-of-evidence thesis, the philosophy of meta-analysis, the role of robustness/sensitivity analysis for evidence amalgamation, its bearing on questions (...) of extrapolation and external validity of experiments, its connection with theory development, and its interface with causal inference, especially regarding causal theories of cancer. (shrink)

This paper looks at an appeal to the authority of biomedical research that has recently been used by empirical economists to motivate and justify their methods. I argue that those who make this appeal mistake the nature of biomedical research. Randomised trials, which are said to have revolutionised biomedical research, are a central methodology, but according to only one paradigm. There is another paradigm at work in biomedical research, the inferentialist paradigm, in which randomised trials play no special role. I (...) outline the inferentialist alternative in broad strokes, apply it to a recent controversy in econometrics and draw some general conclusions concerning econometric methodology. (shrink)

This paper examines Patrick Suppes’ probabilistic theory of causality understood as a theory of causal inference, and draws some lessons for empirical economics and contemporary debates in the foundations of econometrics. It argues that a standard method of empirical economics, multiple regression, is inadequate for most but the simplest applications, that the Bayes’ nets approach, which can be understood as a generalisation of Suppes’ theory, constitutes a considerable improvement but is still subject to important limitations, and that the currently fashionable (...) ‘design-based approach’ suffers from the same flaws Suppes anticipated a long time ago. It then sketches an alternative in response, one that differs drastically from the formalisms Suppes endorsed but is consistent with his pragmatic general take on science. (shrink)

Francesco Guala once wrote that ‘The problem of extrapolation is a minor scandal in the philosophy of science’. This paper agrees with the statement, but for reasons different from Guala’s. The scandal is not, or not any longer, that the problem has been ignored in the philosophy of science. The scandal is that framing the problem as one of external validity encourages poor evidential reasoning. The aim of this paper is to propose an alternative—an alternative which constitutes much better evidential (...) reasoning about target systems of interest, and which makes do without consideration of external validity. (shrink)

This article discusses the concept of evidential reasoning in the context of qualitative research methods in the social sciences. A conceptualization of qualitative evidential reasoning is proposed...

In this paper we argue for a naturalistic solution to some of the methodological controversies in regulatory science, on the basis of two case studies: toxicology and health claim regulation. We analyze the debates related to the scientific evidence that is considered necessary for regulatory decision making in each of those two fields, with a particular attention to the interactions between scientific and regulatory aspects. This analysis allows us to identify two general stances in the debate: a) one that argues (...) for more permissive standards of evidence and for methodological pluralism, and b) an opposing one that not only defends strict evidence requirements but also stipulates the use of one particular scientific methodologies for data generation. We argue that the real-world outcomes produced by alternative regulatory options are a vital piece of information that allows for the empirical assessment of these two stances. In particular, this information on outcomes makes it possible to analyze which standards of evidence and scientific methods generate the most useful knowledge as input for regulatory decision making. Our conclusion is that instead of an a priori selection of methodologies and standards, such decisions ought to be based on empirical evidence related to real-world outcomes. (shrink)

Extrapolating causal effects from experiments to novel populations is a common practice in evidence-based-policy, development economics and other social science areas. Drawing on experimental evidence of policy effectiveness, analysts aim to predict the effects of policies in new populations, which might differ importantly from experimental populations. Existing approaches made progress in articulating the sorts of similarities one needs to assume to enable such inferences. It is also recognized, however, that many of these assumptions will remain surrounded by significant uncertainty in (...) practice. Unfortunately, the existing literature says little on how analysts may articulate and manage these uncertainties. This paper aims to make progress on these issues. First, it considers several existing ideas that bear on issues of uncertainty, elaborates the challenges they face, and extracts some useful rationales. Second, it outlines a novel approach, called the support graph approach, that builds on these rationales and allows analysts to articulate and manage uncertainty in extrapolation in a systematic and unified way. (shrink)

Philosophical theories of evidence have been on offer, but they are mostly evaluated in terms of all-or-none desiderata — if they fail to meet one of the desiderata, they are not a satisfactory theory. In this thesis, I aim to accomplish three missions. Firstly, I construct a new way of evaluating theories of evidence, which I call a tool-based view. Secondly, I analyse the nature of what I will call the various relevance-mediating vehicles that each theory of evidence employs. Thirdly, (...) I articulate the comparative core of evidential reasoning in the historical sciences, one which is overlooked in major theories of evidence. On the first mission, I endorse a meta-thesis of pluralism on theories of evidence, namely a tool-based view. I regard a theory of evidence as a purpose-specific and setting-sensitive tool which has its own strengths, difficulties and limitations. Among the major theories of evidence I have reviewed, I focus on Achinstein’s explanationist theory, Cartwright’s argument theory and Reiss’s inferentialist account, scrutinising and evaluating them against the purposes they set out and the scope of their applications. On the second mission, I note that there is no such thing as intrinsically ‘being evidence’. Rather, I hold that relevance-mediating vehicles configure data, materials or claims in such ways that some of them are labelled evidence. I identify the relevance-mediating vehicles that the theories of evidence employ. On the final mission, I argue that the likelihoodist account is an appropriate tool for explaining the evidential reasoning in poorly specified settings where likelihoods can be only imprecisely compared. Such settings, I believe, are typical in the historical sciences. Using the reconstruction of proto-sounds in historical linguistics as a case study, I formalise the rationale behind it by means of the law of likelihood. (shrink)

Policy-makers face an uncertain world. One way of getting a handle on decision-making in such an environment is to rely on evidence. Despite the recent increase in post-fact figures in politics, evidence-based policymaking takes centre stage in policy-setting institutions. Often, however, policy-makers face large volumes of evidence from different sources. Robustness analysis can, prima facie, handle this evidential diversity. Roughly, a hypothesis is supported by robust evidence if the different evidential sources are in agreement. In this thesis, I strengthen the (...) case for the use of robustness analysis in evidence-based policymaking by answering open research questions about this inference technique. First, I argue that existing taxonomies miss a fruitful category of robustness reasoning, that is predictive stability. Second, I claim that derivational robustness analysis – the investigation of whether the results of different models are in agreement – can yield interesting insights even if not the entire relevant model space is covered by available models or if the model results are only partially in agreement. Third, I claim that expert knowledge is necessary to address questions that arise when one applies measurement robustness analysis – the investigation into whether multiple means of measurement yield the same result. Finally, I argue that, in situations where evidence from different measurements is not in agreement, it can be advisable to no longer take all of the evidence into account. This can be done in a rationally defensible way by choosing the most adequate theory or model underlying parts of the evidence set. I discuss examples from climate, medical, and economic policy-making to establish my claims. (shrink)