An astonishing volume and diversity of evidence is available for many hypotheses in the biomedical and social sciences. Some of this evidence—usually from randomized controlled trials (RCTs)—is amalgamated by meta-analysis. Despite the ongoing debate regarding whether or not RCTs are the ‘gold-standard’ of evidence, it is usually meta-analysis which is considered the best source of evidence: meta-analysis is thought by many to be the platinum standard of evidence. However, I argue that meta-analysis falls far short of that standard. Different meta-analyses (...) of the same evidence can reach contradictory conclusions. Meta-analysis fails to provide objective grounds for intersubjective assessments of hypotheses because numerous decisions must be made when performing a meta-analysis which allow wide latitude for subjective idiosyncrasies to influence its outcome. I end by suggesting that an older tradition of evidence in medicine—the plurality of reasoning strategies appealed to by the epidemiologist Sir Bradford Hill—is a superior strategy for assessing a large volume and diversity of evidence. (shrink)

Causal claims in biomedical contexts are ubiquitous albeit they are not always made explicit. This paper addresses the question of what causal claims mean in the context of disease. It is argued that in medical contexts causality ought to be interpreted according to the epistemic theory. The epistemic theory offers an alternative to traditional accounts that cash out causation either in terms of “difference-making” relations or in terms of mechanisms. According to the epistemic approach, causal claims tell us about which (...) inferences (e.g., diagnoses and prognoses) are appropriate, rather than about the presence of some physical causal relation analogous to distance or gravitational attraction. It is shown that the epistemic theory has important consequences for medical practice, in particular with regard to evidence-based causal assessment. (shrink)

Evidence-based medicine (EBM) makes use of explicit procedures for grading evidence for causal claims. Normally, these procedures categorise evidence of correlation produced by statistical trials as better evidence for a causal claim than evidence of mechanisms produced by other methods. We argue, in contrast, that evidence of mechanisms needs to be viewed as complementary to, rather than inferior to, evidence of correlation. In this paper we first set out the case for treating evidence of mechanisms alongside evidence of correlation in (...) explicit protocols for evaluating evidence. Next we provide case studies which exemplify the ways in which evidence of mechanisms complements evidence of correlation in practice. Finally, we put forward some general considerations as to how the two sorts of evidence can be more closely integrated by EBM. (shrink)

According to current hierarchies of evidence for EBM, evidence of correlation (e.g., from RCTs) is always more important than evidence of mechanisms when evaluating and establishing causal claims. We argue that evidence of mechanisms needs to be treated alongside evidence of correlation. This is for three reasons. First, correlation is always a fallible indicator of causation, subject in particular to the problem of confounding; evidence of mechanisms can in some cases be more important than evidence of correlation when assessing a (...) causal claim. Second, evidence of mechanisms is often required in order to obtain evidence of correlation (for example, in order to set up and evaluate RCTs). Third, evidence of mechanisms is often required in order to generalise and apply causal claims. While the EBM movement has been enormously successful in making explicit and critically examining one aspect of our evidential practice, i.e., evidence of correlation, we wish to extend this line of work to make explicit and critically examine a second aspect of our evidential practices: evidence of mechanisms. (shrink)

A number of arguments have shown that randomization is not essential in experimental design. Scientific conclusions can be drawn on data from experimental designs that do not involve randomization. John Worrall has recently taken proponents of randomized studies to task for suggesting otherwise. In doing so, however, Worrall makes an additional claim: randomized interventional studies are epistemologically equivalent to observational studies, providing the experimental groups are comparable according to background knowledge. I argue against this claim. In the context of testing (...) the efficacy of drug therapies, well-designed interventional studies are epistemologically superior to well-designed observational studies because they have the capacity to avoid a type of selection bias. Although arguments for interventional studies are present in the medical literature, these arguments are too often presented as an argument for randomization. Randomization in interventional studies is defended on Bayesian grounds. (shrink)

This chapter explores the idea that causal inference is warranted if and only if the mechanism underlying the inferred causal association is identified. This mechanistic stance is discernible in the epidemiological literature, and in the strategies adopted by epidemiologists seeking to establish causal hypotheses. But the exact opposite methodology is also discernible, the black box stance, which asserts that epidemiologists can and should make causal inferences on the basis of their evidence, without worrying about the mechanisms that might underlie their (...) hypotheses. I argue that the mechanistic stance is indeed a bad methodology for causal inference. However, I detach and defend a mechanistic interpretation of causal generalisations in epidemiology as existence claims about underlying mechanisms. (shrink)

The philosophy of evidence-based medicine -- What is EBM? -- What is good evidence for a clinical decision? -- Ruling out plausible rival hypotheses and confounding factors : a method -- Resolving the paradox of effectiveness : when do observational studies offer the same degree of evidential support as randomized trials? -- Questioning double blinding as a universal methodological virtue of clinical trials : resolving the Philip's paradox -- Placebo controls : problematic and misleading baseline measures of effectiveness -- Questioning (...) the methodological superiority of "placebo" over "active" controlled trials -- Examining the paradox that traditional roles for mechanistic reasoning and expert -- Judgment have been up-ended by EBM -- A qualified defence of the EBM stance on mechanistic reasoning -- Knowledge that versus knowledge how : situating the EBM position on expert clinical judgment -- Moving EBM forward. (shrink)

Proponents of evidence-based medicine (EBM) provide the “hierarchy of evidence” as a criterion for judging the reliability of therapeutic decisions. EBM's hierarchy places randomized interventional studies (and systematic reviews of such studies) higher in the hierarchy than observational studies, unsystematic clinical experience, and basic science. Recent philosophical work has questioned whether EBM's special emphasis on evidence from randomized interventional studies can be justified. Following the critical literature, and in particular the work of John Worrall, I agree that many of the (...) arguments put forward by advocates of EBM do not justify the ambitious claims that are often made on behalf of randomization. However, in contrast to the recent philosophical work, I argue that a justification for EBM's hierarchy of evidence can be provided. The hierarchy should be viewed as a hierarchy of comparative internal validity. Although this justification is defensible, the claims that EBM's hierarchy substantiates when viewed in this way are considerably more circumscribed than some claims found in the EBM literature. (shrink)

Scientific reasoning is—and ought to be—conducted in accordance with the axioms of probability. This Bayesian view—so called because of the central role it accords to a theorem first proved by Thomas Bayes in the late eighteenth ...

It is surely obvious that medicine, like any other rational activity, must be based on evidence. The interest is in the details: how exactly are the general principles of the logic of evidence to be applied in medicine? Focussing on the development, and current claims of the ‘Evidence-Based Medicine’ movement, this article raises a number of difficulties with the rationales that have been supplied in particular for the ‘evidence hierarchy’ and for the very special role within that hierarchy of randomized (...) controlled trials (and meta-analyses of the results of randomized controlled trials). The point is not at all to question the application of a scientific approach to evidence in medicine, but, on the contrary, to indicate a number of areas where philosophers of science can contribute to a proper implementation of exactly that scientific-evidential approach. (shrink)

The evidence from randomized controlled trials (RCTs) is widely regarded as supplying the ‘gold standard’ in medicine—we may sometimes have to settle for other forms of evidence, but this is always epistemically second-best. But how well justified is the epistemic claim about the superiority of RCTs? This paper adds to my earlier (predominantly negative) analyses of the claims produced in favour of the idea that randomization plays a uniquely privileged epistemic role, by closely inspecting three related arguments from leading contributors (...) to the burgeoning field of probabilistic causality—Papineau, Cartwright and Pearl. It concludes that none of these further arguments supplies any practical reason for thinking of randomization as having unique epistemic power. IntroductionWhy the issue is of great practical importance—the ECMO casePapineau on the ‘virtues of randomization’Cartwright on causality and the ‘ideal’ randomized experimentPearl on randomization, nets and causesConclusion. (shrink)

We argue that the health sciences make causal claims on the basis of evidence both of physical mechanisms, and of probabilistic dependencies. Consequently, an analysis of causality solely in terms of physical mechanisms or solely in terms of probabilistic relationships, does not do justice to the causal claims of these sciences. Yet there seems to be a single relation of cause in these sciences - pluralism about causality will not do either. Instead, we maintain, the health sciences require a theory (...) of causality that unifies its mechanistic and probabilistic aspects. We argue that the epistemic theory of causality provides the required unification. (shrink)

In this book, the author of "Language, Truth and Logic" tackles one of the central issues of philosophy - how we can know anything - by setting out all the sceptic's arguments and trying to counter them one by one.

The claims of randomized controlled trials to be the gold standard rest on the fact that the ideal RCT is a deductive method: if the assumptions of the test are met, a positive result implies the appropriate causal conclusion. This is a feature that RCTs share with a variety of other methods, which thus have equal claim to being a gold standard. This article describes some of these other deductive methods and also some useful non-deductive methods, including the hypothetico-deductive method. (...) It argues that with all deductive methods, the benefit that the conclusions follow deductively in the ideal case comes with a great cost: narrowness of scope. This is an instance of the familiar trade-off between internal and external validity. RCTs have high internal validity but the formal methodology puts severe constraints on the assumptions a target population must meet to justify exporting a conclusion from the test population to the target. The article reviews one such set of assumptions to show the kind of knowledge required. The overall conclusion is that to draw causal inferences about a target population, which method is best depends case-by-case on what background knowledge we have or can come to obtain. There is no gold standard. (shrink)

This paper critically examines a particular strategy for resolving the central ethical dilemma associated with randomized clinical trials — the “community equipoise” strategy . The dilemma is that RCTs appear to violate a physician's duty to choose that therapy which there is most reason to believe is in the patient's best interest, randomizing patients even once evidence begins to favor one treatment. The community equipoise strategy involves the suggestion that our judgment that neither treatment is to be preferred is to (...) be assessed according to a community rather than an individual standard. Thus, though a physician may personally believe that there is some reason to prefer one treatment, patients can legitimately be randomized if there remains disagreement in the community of medical professionals. Rationales in favor of this conception include the following: medical knowledge is best understood as residing in the community, the judgments of others count as evidence, and so should change one's own opinion, subjects would not be better off outside the trial, and the point of any trial is the resolution of dispute in the medical community. I critically examine these rationales and argue that they are insufficient. Amongst the problems are tensions between various of these underlying rationales, and important ambiguities in just what the CE criterion is to amount to. Finally, I argue that even if use of CE was justified, it would not justify carrying out RCTs anywhere near long enough to discharge our duty to gain reliable knowledge on which to base safe and effective medical practice. Hence, we need some different justification for carrying out RCTs. (shrink)

I will open the first part of this paper by trying to elucidate the frequentist foundations of RCTs. I will then present a number of methodological objections against the viability of these inferential principles in the conduct of actual clinical trials. In the following section, I will explore the main ethical issues in frequentist trials, namely those related to randomisation and the use of stopping rules. In the final section of the first part, I will analyse why RCTs were accepted (...) for regulatory purposes. I contend that their main virtue, from a regulatory viewpoint, is their impartiality, which is grounded in randomisation and fixed rules for the interpretation of the experiment. Thus the question will be whether Bayesian trials can match or exceed the achievements of frequentist RCTs in all these respects. In the second part of the paper, I will first present a quick glimpse of the introduction of Bayesianism in the field of medical experiments, followed by a summary presentation of the basic tenets of a Bayesian trial. The point here is to show that there is no such thing as “a” Bayesian trial. Bayesianism can ground many different approaches to medical experiments and we should assess their respective virtues separately. Thus I present two actual trials, planned with different goals in mind, and assess their respective epistemic, ethical and regulatory merits. In a tentative conclusion, I contend that, given the constraints imposed by our current regulatory framework, impartiality should preside over the design of clinical trials, even at the expense of many of their inferential and ethical virtues. (shrink)

The paper considers the legal tools that have been developed in German pharmaceutical regulation as a result of the precautionary attitude inaugurated by the Contergan decision. These tools are the notion of “well-founded suspicion”, which attenuates the requirements for safety intervention by relaxing the requirement of a proved causal connection between danger and source, and the introduction of the reversal of proof burden in liability norms. The paper focuses on the first and proposes seeing the precautionary principle as an instance (...) of the requirement that one should maximise expected utility. In order to maximise expected utility certain probabilities are required and it is argued that objective Bayesianism offers the most plausible means to determine the optimal decision in cases where evidence supports diverging choices. (shrink)

The claims of RCTs to be the gold standard rest on the fact that the ideal RCT is a deductive method: if the assumptions of the test are met, a positive result implies the appropriate causal conclusion. This is a feature that RCTs share with a variety of other methods, which thus have equal claim to being a gold standard. This paper describes some of these other deductive methods and also some useful non-deductive methods, including the hypothetico-deductive method. It argues (...) that with all deductive methods, the benefit that the conclusions follow deductively in the ideal case comes with a great cost: narrowness of scope. This is an instance of the familiar trade-off between internal and external validity. RCTs have high internal validity but the formal methodology puts severe constraints on the assumptions a target population must meet to justify exporting a conclusion from the test population to the target. The paper reviews one such set of assumptions to show the kind of knowledge required. The overall conclusion is that to draw causal inferences about a target population, which method is best depends case-by-case on what background knowledge we have or can come to obtain. There is no gold standard. (shrink)

Coping with uncertainty is a necessary part of ordinary life and is crucial to an understanding of how the mind works. For example, it is a vital element in developing artificial intelligence that will not be undermined by its own rigidities. There have been many approaches to the problem of uncertain inference, ranging from probability to inductive logic to nonmonotonic logic. Thisbook seeks to provide a clear exposition of these approaches within a unified framework. The principal market for the book (...) will be students and professionals in philosophy, computer science, and AI. Among the special features of the book are a chapter on evidential probability, which has not received a basic exposition before; chapters on nonmonotonic reasoning and theory replacement, matters rarely addressed in standard philosophical texts; and chapters on Mill's methods and statistical inference that cover material sorely lacking in the usual treatments of AI and computer science. (shrink)

The problem of collecting, analyzing and evaluating evidence on adverse drug reactions (ADRs) is an example of the more general class of epistemological problems related to scientific inference and prediction, as well as a central problem of the health-care practice. Philosophical discussions have critically analysed the methodological pitfalls and epistemological implications of evidence assessment in medicine, however they have mainly focused on evidence of treatment efficacy. Most of this work is devoted to statistical methods of causal inference with a special (...) attention to the privileged role assigned to randomized controlled trials in Evidence Based Medicine. Regardless of whether the RCT’s privilege holds for efficacy assessment, it is nevertheless important to make a distinction between causal inference of intended and unintended effects, in that the unknowns at stake are heterogonous in the two contexts. This point has been emphasized by epidemiologists in the last decade. Their main focus is methodological, and regards the fact that bias and confounding do not affect studies on intended and unintended effects in the same way. However, deeper concerns ground the intuition for such a distinction; these are related to the constraints which we impose on evidence and their epistemological justification. My thesis is that such constraints ought to be understood to be different in the case of evidence for risk vs. benefit assessment. I present the recent debate on the causal association between acetaminophen and asthma in order to illustrate the point at issue. (shrink)

In a randomized clinical trial (RCT), a group of patients, initially assembled through a mixture of deliberation (involving explicit inclusion and exclusion criteria) and serendipity (which patients happen to walk into which doctor’s clinic while the trial is in progress), are divided by some random process into an experimental group (members of which will receive the therapy under test) and a control group (members of which will receive some other treatment – perhaps placebo, perhaps the currently standard treatment for the (...) condition at issue). In a ‘double blind’ trial neither the patient nor the clinician knows to which of the groups a particular patient belongs. The results of double blind randomized controlled trials are almost universally regarded as providing the ‘gold standard’ for evidence in medicine. Fairly extreme claims to this effect can be found in the literature. For example the statistician Tukey wrote (1977, p. 679) “almost the only source of reliable evidence [in medicine] … is that obtained from … carefully conducted randomised trials”. And the clinician Victor Herbert claimed (1977, p. 690) “..the only source of reliable evidence rising to the level of proof about the usefulness of any new therapy is that obtained from wellplanned and carefully conducted randomized, and, where possible, coded (double blind) clinical trials. [Other] studies may point in a direction, but cannot be evidence as lawyers use the term evidence to mean something probative … [that is] tending to prove or actually proving”. Finally, the still very influential movement in favour of ‘Evidence Based Medicine’ (EBM) that began at McMaster University in the 1980s was initially often regarded as endorsing the claim that only RCTs provide real scientifically telling evidence. (shrink)

Peter Urbach has argued, on Bayesian grounds, that experimental randomization serves no useful purpose in testing causal hypothesis. I maintain that he fails to distinguish general issues of statistical inference from specific problems involved in identifying causes. I concede the general Bayesian thesis that random sampling is inessential to sound statistical inference. But experimental randomization is a different matter, and often plays an essential role in our route to causal conclusions.

Three main lines of arguments are presented as a defense of randomization in experimental design. The first concerns the computational advantages of randomizing when a well-defined underlying theoretical model is not available, as is often the case in much experimentation in the medical and social sciences. The high desirability, even for the most dedicated Bayesians, of physical randomization in some special cases is stressed. The second line of argument concerns communication of methodology and results, especially in terms of concerns about (...) bias. The third line of argument concerns the use of randomization to guarantee causal inferences, whether the inference consists of the identification of a prima facie or a genuine cause. In addition, the relation of randomization to measures of complexity and the possibility of accepting only random procedures that produce complex results are discussed. (shrink)

It is argued that randomization has no role to play in the design or analysis of an experiment. If a Bayesian approach is adopted this conclusion is easily demonstrated. Outside that approach two principles, of conditionality and similarity, lead, via the likelihood principle, to the same conclusion. In the case of design, however, it is important to avoid confounding the effect of interest with an unexpected factor and this consideration leads to a principle of haphazardness that is clearly related to, (...) but not identical with, randomization. The role of exchangeability is discussed. (shrink)