Scientific consensus is widely deferred to in public debates as a social indicator of the existence of knowledge. However, it is far from clear that such deference to consensus is always justified. The existence of agreement in a community of researchers is a contingent fact, and researchers may reach a consensus for all kinds of reasons, such as fighting a common foe or sharing a common bias. Scientific consensus, by itself, does not necessarily indicate the existence of shared knowledge among (...) the members of the consensus community. I address the question of under what conditions it is likely that a consensus is in fact knowledge based. I argue that a consensus is likely to be knowledge based when knowledge is the best explanation of the consensus, and I identify three conditions—social calibration, apparent consilience of evidence, and social diversity, for knowledge being the best explanation of a consensus. (shrink)

Philosophical discussions on causal inference in medicine are stuck in dyadic camps, each defending one kind of evidence or method rather than another as best support for causal hypotheses. Whereas Evidence Based Medicine advocates the use of Randomised Controlled Trials and systematic reviews of RCTs as gold standard, philosophers of science emphasise the importance of mechanisms and their distinctive informational contribution to causal inference and assessment. Some have suggested the adoption of a pluralistic approach to causal inference, and an inductive (...) rather than hypothetico-deductive inferential paradigm. However, these proposals deliver no clear guidelines about how such plurality of evidence sources should jointly justify hypotheses of causal associations. We here develop such guidelines by first giving a philosophical analysis of the underpinnings of Hill’s viewpoints on causality. We then put forward an evidence-amalgamation framework adopting a Bayesian net approach to model causal inference in pharmacology for the assessment of harms. Our framework accommodates a number of intuitions already expressed in the literature concerning the EBM vs. pluralist debate on causal inference, evidence hierarchies, causal holism, relevance, and reliability. (shrink)

We claim that if a complete philosophy of evidence-based practice is intended, then attention to the nature of causation in health science is necessary. We identify how health science currently conceptualises causation by the way it prioritises some research methods over others. We then show how the current understanding of what causation is serves to constrain scientific progress. An alternative account of causation is offered. This is one of dispositionalism. We claim that by understanding causation from a dispositionalist stance, many (...) of the processes within an evidence-based practice framework are better accounted for. Further, some of the problems associated with the health research, e.g. external validity of causal findings, dissolve. (shrink)

Proponents of Evidence-based medicine (EBM) do not provide a clear role for basic science in therapeutic decision making. Of what they do say about basic science, most of it is negative. Basic science resides on the lower tiers of EBM's hierarchy of evidence. Therapeutic decisions, according to proponents of EBM, should be informed by evidence from randomised studies (and systematic reviews of randomised studies) rather than basic science. A framework of models explicates the links between the mechanisms of basic science, (...) experimental inquiry, and observed data. Relying on the framework of models I show that basic science often plays a role not only in specifying experiments, but also analysing and interpreting the data that is provided. Further, and contradicting what is implied in EBM's hierarchy of evidence, appeals to basic science are often required to apply clinical research to therapeutic questions. © 2010 Springer Science+Business Media B.V. (shrink)

Philosophical discussions have critically analysed the methodological pitfalls and epistemological implications of evidence assessment in medicine, however they have mainly focused on evidence of treatment efficacy. Most of this work is devoted to statistical methods of causal inference with a special attention to the privileged role assigned to randomized controlled trials (RCTs) in evidence based medicine. Regardless of whether the RCT’s privilege holds for efficacy assessment, it is nevertheless important to make a distinction between causal inference of intended and unintended (...) effects, in that the unknowns at stake are heterogonous in the two contexts. However, although “lower level” evidence is increasingly acknowledged to be a valid source of information contributory to assessing the risk profile of medications on theoretical or empirical grounds, current practices have difficulty in assigning a precise epistemic status to this kind of evidence because they are more or less implicitly parasitic on the (statistical) methods developed to test drug efficacy. My thesis is that (1) “lower level” evidence is justified on distinct grounds and at different conditions depending on the different epistemologies which one wishes to endorse, in that each impose different constraints on the methods we adopt to collect and evaluate evidence; (2) such constraints ought to be understood to be different in the case of evidence for risk versus benefit assessment for a series of reasons which I will illustrate on the basis of the recent debate on the causal association between acetaminophen (a.k.a. paracetamol) and asthma. (shrink)

The current strategy of searching for an effective treatment for COVID-19 relies mainly on repurposing existing therapies developed to target other diseases. Conflicting results have emerged in regard to the efficacy of several tested compounds but later results were negative. The number of conducted and ongoing trials and the urgent need for a treatment pose the risk that false-positive results will be incorrectly interpreted as evidence for treatments’ efficacy and a ground for drug approval. Our purpose is twofold. First, we (...) show that the number of drug-repurposing trials can explain the false-positive results. Second, we assess the evidence for treatments’ efficacy from the perspective of evidential pluralism and argue that considering mechanistic evidence is particularly needed in cases when the evidence from clinical trials is conflicting or of low quality. Our analysis is an application of the program of Evidence Based Medicine Plus to the drug repurposing trials for COVID. Our study shows that if decision-makers applied EBM+, authorizing the use of ineffective treatments would be less likely. We analyze the example of trials assessing the efficacy of hydroxychloroquine as a treatment for COVID-19 and mechanistic evidence in favor of and against its therapeutic power to draw a lesson for decision-makers and drug agencies on how excessive hypothesis testing can lead to spurious findings and how studying negative mechanistic evidence can be helpful in discriminating genuine from spurious results. (shrink)

Background: Evidence suggesting adverse drug reactions often emerges unsystematically and unpredictably in form of anecdotal reports, case series and survey data. Safety trials and observational studies also provide crucial information regarding the (un-)safety of drugs. Hence, integrating multiple types of pharmacovigilance evidence is key to minimising the risks of harm. Methods: In previous work, we began the development of a Bayesian framework for aggregating multiple types of evidence to assess the probability of a putative causal link between drugs and side (...) effects. This framework arose out of a philosophical analysis of the Bradford Hill Guidelines. In this article, we expand the Bayesian framework and add “evidential modulators,” which bear on the assessment of the reliability of incoming study results. The overall framework for evidence synthesis, “E-Synthesis”, is then applied to a case study. Results: Theoretically and computationally, E-Synthesis exploits coherence of partly or fully independent evidence converging towards the hypothesis of interest (or of conflicting evidence with respect to it), in order to update its posterior probability. With respect to other frameworks for evidence synthesis, our Bayesian model has the unique feature of grounding its inferential machinery on a consolidated theory of hypothesis confirmation (Bayesian epistemology), and in allowing any data from heterogeneous sources (cell-data, clinical trials, epidemiological studies), and methods (e.g., frequentist hypothesis testing, Bayesian adaptive trials, etc.) to be quantitatively integrated into the same inferential framework. Conclusions: E-Synthesis is highly flexible concerning the allowed input, while at the same time relying on a consistent computational system, that is philosophically and statistically grounded. Furthermore, by introducing evidential modulators, and thereby breaking up the different dimensions of evidence (strength, relevance, reliability), E-Synthesis allows them to be explicitly tracked in updating causal hypotheses. (shrink)

Evidence-based policy has support in many areas of government and in public affairs more generally. In this paper we outline what evidence-based policy is, then we discuss its strengths and weaknesses. In particular, we argue that it faces a serious challenge to provide a plausible, over-arching account of evidence. We contrast evidence-based policy with evidence-based medicine, especially the role of evidence in assessing the effectiveness of medicines. The evidence required for policy decisions does not easily lend itself to randomized controlled (...) trials, nor, for that matter, being listed in a single all-purpose hierarchy. (shrink)

The results of econometric modeling are fragile in the sense that minor changes in estimation techniques or sample can lead to statistical models that support inconsistent causal hypotheses. The fragility of econometric results undermines making conclusive inferences from the empirical literature. I argue that the program of evidential pluralism, which originated in the context of medicine and encapsulates to the normative reading of the Russo-Williamson Thesis that causal claims need the support of both difference-making and mechanistic evidence, offers a ground (...) for resolving empirical disagreements. I analyze a recent econometric controversy regarding the tax elasticity of cigarette consumption and smoking intensity. Both studies apply plausible estimation techniques but report inconsistent results. I show that mechanistic evidence allows for discriminating econometric models representing genuine causal relations from accidental dependencies in data. Furthermore, I discuss the differences between biological and social mechanisms and mechanistic evidence across the disciplines. I show that economists mainly rely on mathematical models to represent possible mechanisms. Still, claiming the actuality of the represented mechanisms requires establishing that crucial assumptions of these models are descriptively adequate. I exemplify my approach to assessing the quality of mechanistic evidence in economics with an analysis of two models of rational addiction. (shrink)

Agent-based models (ABMs) are one of the main sources of evidence for decisions regarding mitigation and suppression measures against the spread of SARS-CoV-2. These models have not been previously included in the hierarchy of evidence put forth by the evidence-based medicine movement, which prioritizes those research methods that deliver results less susceptible to the risk of confounding. We point out the need to assess the quality of evidence delivered by ABMs and ask the question of what is the risk that (...) assumptions entertained in ABMs do not include all the key factors and make model predictions susceptible to the problem of confounding. (shrink)

Heterogeneous treatment effects represent a major issue for medicine as they undermine reliable inference and clinical decision-making. To overcome the issue, the current vision of precision and personalized medicine acknowledges the need to control individual variability in response to treatment. In this paper, we argue that gene-treatment-environment interactions (G × T × E) undermine inferences about individual treatment effects from the results of both genomics-based methodologies—such as genome-wide association studies (GWAS) and genome-wide interaction studies (GWIS)—and randomized controlled trials (RCTs). Then, (...) we argue that N-of-1 trials can be a solution to overcome difficulties in handling individual variability in treatment response. Although this type of trial has been suggested as a promising strategy to assess individual treatment effects, it nonetheless has limitations that limit its use in everyday clinical practice. We analyze the existing variability within the designs of N-of-1 trials in terms of a continuum where each design prioritizes epistemic and pragmatic considerations. We then support wider use of the designs located at the pragmatic end of the explanatory-pragmatic continuum. (shrink)

Since its introduction just over two decades ago, evidence-based medicine (EBM) has come to dominate medical practice, teaching, and policy. There are a growing number of textbooks, journals, and websites dedicated to EBM research, teaching, and evidence dissemination. EBM was most recently defined as a method that integrates best research evidence with clinical expertise and patient values and circumstances in the treatment of patients. There have been debates throughout the early 21st century about what counts as good research evidence between (...) EBM proponents and philosophical critics and even within the EBM community itself. Similar controversy arises about the relative worth of patient values and clinical expertise (and how these can be integrated). EBM has also evolved in ways that have come under scrutiny. Specifically, policymakers have used EBM research methodology to increase the relative importance of clinical guidelines that some clinicians have argued are tyrannical. Philosophers have addressed all of these controversies, and with very few exceptions have been critical of EBM. In addition most philosophical attention has been on the epistemic role of Randomization and evidence hierarchies, with relatively little attention being paid to the role of Diagnosis, expertise, patient values, and Systematic Reviews within EBM. (shrink)

The existing philosophical views on what is the meaning of causality adequate to medicine are vastly divided. We approach this question and offer two arguments in favor of pluralism regarding concepts of causality. First, we analyze the three main types of research designs (randomized-controlled trials, observational epidemiology and laboratory research). We argue, using examples, that they allow for making causal conclusions that are best understood differently in each case (in agreement with a version of manipulationist, probabilistic and mechanistic definitions, respectively). (...) Second, we analyze clinical practice and argue that these manipulationist, probabilistic and mechanistic causal claims can be used as evidence for different therapeutic decisions. We differentiate among ‘predicting’ that does not change the relata of causal claims, (mechanistic) ‘interferences’, and ‘interventions’ in the strict sense that act on causes to change effects. The central conclusion is that causal claims agreeing with diverse concepts of causality can deliver evidence for different types of therapeutic decisions. (shrink)

Evidence-based policy is gaining support in many areas of government and in public affairs more generally. In this paper we outline what evidence—based policy is then discuss its strengths and weaknesses. In particular, we argue that it faces a serious challenge to provide a plausible account of evidence. This account needs to be at least in the spirit of the hierarchy of evidence subscribed to by evidence-based medicine (from which evidence—based policy derives its name and inspiration). Yet evidence-based policy’s hierarchy (...) needs to be tailored to the kinds of evidence relevant and available to the policy arena. The evidence required for policy decisions does not easily lend itself to randomised controlled trials (the "gold standard" in evidence-based medicine), nor, for that matter, being listed in a single all—purpose hierarchy. (shrink)

This chapter addresses the problem of ranking available drugs in guideline development to support clinicians in their work. Based on a pragmatic approach to the notion of evidence and a hierarchical view on different kinds of evidence this chapter introduces a decision aid, HiDAD, which draws on the multi criteria decision making literature. This decision aid implements the wide-spread intuition that there are different kinds of evidence with varying degrees of importance by relying on a strict ordinal ordering of kinds (...) of evidence. In order to construct a ranking every pair of drugs is first compared separately on all kinds of evidence. Next, these quantitative comparisons are then aggregated into an overall comparison between drugs based on all the available evidence in a way which avoids that evidence of less importance is trumped by evidence of the higher levels. Finally, these overall comparisons are used to determine the final ranking of drugs which then informs the process of guideline writing. Properties, modifications and applicability of the decision aid HiDAD are discussed and assessed. (shrink)

We revisit John Worrall’s old but still prominent argument against the view that randomization balances the impact of both known and unknown confounders across the treatment and control arms. We argue that his argument involving indefinitely many possible confounders is at odds with statistical theory as it (1) presumes that the purpose of randomized studies is obtaining perfect point estimates for which perfect balance is needed; (2) mistakes equalizing each confounder with the overall (average) impact of all confounders, and (3) (...) assumes that the joint effect of an infinite series of confounders cannot be bounded. We defend the role of randomization in balancing the impact of confounders across the treatment and control arms by putting forward the statistical sense of the balance claim. It involves the following three commitments: (1) randomization balances confounders in expectancy, (2) for RCTs to deliver unbiased estimates of the causal effect (true average treatment effect), the balance in the average effect of all confounders and not balancing each confounder is sufficient, and (3) randomization allows for calculating the probability of deviating from the balance. The paper includes a review of how the balance claim has been understood so far and discusses recent arguments supporting randomization balancing the impact of confounders in expectancy and the crucial role of the average impact of all actual confounders, and shows how statistical analysis of RCTs conducted both at the design and analysis stage makes possible estimating the probabilities of deviating from the balance. (shrink)

My doctoral project is a study of epistemological and ethical issues in Evidence-Based Medicine, a movement in medicine which emphasizes the use of randomized controlled trials. Much of the research on EBM suggests that, for a large part of the movement's history, EBM considered expertise, mechanisms, and values to be forces contrary to its goals and has sought to remove them, both from medical research and from the clinical encounter. I argue, however, that expertise, mechanisms and values have important epistemological (...) and ethical roles to play and can be incorporated into the current EBM movement. (shrink)

The problem of collecting, analyzing and evaluating evidence on adverse drug reactions (ADRs) is an example of the more general class of epistemological problems related to scientific inference and prediction, as well as a central problem of the health-care practice. Philosophical discussions have critically analysed the methodological pitfalls and epistemological implications of evidence assessment in medicine, however they have mainly focused on evidence of treatment efficacy. Most of this work is devoted to statistical methods of causal inference with a special (...) attention to the privileged role assigned to randomized controlled trials in Evidence Based Medicine. Regardless of whether the RCT’s privilege holds for efficacy assessment, it is nevertheless important to make a distinction between causal inference of intended and unintended effects, in that the unknowns at stake are heterogonous in the two contexts. This point has been emphasized by epidemiologists in the last decade. Their main focus is methodological, and regards the fact that bias and confounding do not affect studies on intended and unintended effects in the same way. However, deeper concerns ground the intuition for such a distinction; these are related to the constraints which we impose on evidence and their epistemological justification. My thesis is that such constraints ought to be understood to be different in the case of evidence for risk vs. benefit assessment. I present the recent debate on the causal association between acetaminophen and asthma in order to illustrate the point at issue. (shrink)

Hierarchies of evidence are an important and influential tool for appraising evidence in medicine. In recent years, hierarchies have been formally adopted by organizations including the Cochrane Collaboration [1], NICE [2,3], the WHO [4], the US Preventive Services Task Force [5], and the Australian NHMRC [6,7]. The development of such hierarchies has been regarded as a central part of Evidence-Based Medicine, a movement within healthcare which prioritises the use of epidemiological evidence such as that provided by Randomised Controlled Trials. Philosophical (...) work on the methodology of medicine has so far mostly focused on claims about the superiority of RCTs, and hence has largely neglected the questions of what hierarchies are, what assumptions they require, and how they affect clinical practice. This thesis shows that there is great variation in the hierarchies defended and in the interpretations they are, and can be, given. The interpretative assumptions made in using hierarchies are crucial to the content and defensibility of the underlying philosophical commitments concerning evidence and medical practice. Once this variation is been identified, it becomes clear that the little philosophical work that has been done so far affects only some hierarchies, under some interpretations. Modest interpretations offered by La Caze [11], conditional hierarchies like GRADE [12-14], and heuristic approaches such as that defended by Howick et al. [15,16] all survive previous philosophical criticism. This thesis extends previous criticisms by arguing that modest interpretations are so weak as to be unhelpful for clinical practice; that GRADE and similar conditional models omit clinically relevant information, such as information about variation in treatments’ effects and the causes of different responses to therapy; and that heuristic approaches lack the necessary empirical support. The conclusion is that hierarchies in general embed untenable philosophical assumptions: principally that information about average treatment effects backed by high-quality evidence can justify strong recommendations, and that the impact of evidence from individual studies can and should be appraised in isolation. Hierarchies are a poor basis for the application of evidence in clinical practice. The Evidence-Based Medicine movement should move beyond them and explore alternative tools for appraising the overall evidence for therapeutic claims. (shrink)