Randomized controlled trials (RCTs) are widely taken as the gold standard for establishing causal conclusions. Ideally conducted they ensure that the treatment ‘causes’ the outcome—in the experiment. But where else? This is the venerable question of external validity. I point out that the question comes in two importantly different forms: Is the specific causal conclusion warranted by the experiment true in a target situation? What will be the result of implementing the treatment there? This paper explains how the probabilistic theory (...) of causality implies that RCTs can establish causal conclusions and thereby provides an account of what exactly that causal conclusion is. Clarifying the exact form of the conclusion shows just what is necessary for it to hold in a new setting and also how much more is needed to see what the actual outcome would be there were the treatment implemented. (shrink)

Now in its fourth edition, Rational Diagnosis and Treatment: Evidence-Based Clinical Decision - Making is a unique book to look at evidence-based medicine and the difficulty of applying evidence from group studies to individual patients._ The book analyses the successive stages of the decision process and deals with topics such as the examination of the patient,_the reliability of clinical data, the logic of diagnosis, the fallacies of uncontrolled therapeutic experience and the need for randomised clinical trials and meta-analyses. It is (...) the main theme of the book that, whenever possible, clinical decisions must be based on the evidence from clinical research, but the authors also explain the pitfalls of such research and the problems involved in applying evidence from groups of patients to the individual patient._ For this new edition, the sections on placebo and meta-analysis and on alternative medicine have been thoroughly updated, and there is more focus on insufficient reporting of harms of interventions. The sections on different research designs describe advantages and limitations, and the increased medicalisation and the effects of cancer screening on health people are noted. A section on academic freedom when clinicians collaborate with industry and ghost authors is added._ This essential reference work integrates the science and statistical approach of evidence-based medicine with the art and humanism of medical practice; distinguishing between data, sets of data, knowledge and wisdom, and their application. Such an intellectually challenging book is ideal for both medical students and doctors who require theoretical and practical clinical skills to help ensure that they apply theory in practice. (shrink)

Ever since David Hume, empiricists have barred powers and capacities from nature. In this book Cartwright argues that capacities are essential in our scientific world, and, contrary to empiricist orthodoxy, that they can meet sufficiently strict demands for testability. Econometrics is one discipline where probabilities are used to measure causal capacities, and the technology of modern physics provides several examples of testing capacities (such as lasers). Cartwright concludes by applying the lessons of the book about capacities and probabilities to the (...) explanation of the role of causality in quantum mechanics. (shrink)

Evidence-Based Medicine is a relatively new movement that seeks to put clinical med- icine on a firmer scientific footing. I take it as uncontroversial that medical practice should be based on best evidence-the interesting questions concern the details. This paper tries to move towards a coherent and unified account of best evidence in medicine, by exploring in particular the EBM position on RCTs (randomized controlled trials).

The evidence from randomized controlled trials (RCTs) is widely regarded as supplying the ‘gold standard’ in medicine—we may sometimes have to settle for other forms of evidence, but this is always epistemically second-best. But how well justified is the epistemic claim about the superiority of RCTs? This paper adds to my earlier (predominantly negative) analyses of the claims produced in favour of the idea that randomization plays a uniquely privileged epistemic role, by closely inspecting three related arguments from leading contributors (...) to the burgeoning field of probabilistic causality—Papineau, Cartwright and Pearl. It concludes that none of these further arguments supplies any practical reason for thinking of randomization as having unique epistemic power. IntroductionWhy the issue is of great practical importance—the ECMO casePapineau on the ‘virtues of randomization’Cartwright on causality and the ‘ideal’ randomized experimentPearl on randomization, nets and causesConclusion. (shrink)

The claims of RCTs to be the gold standard rest on the fact that the ideal RCT is a deductive method: if the assumptions of the test are met, a positive result implies the appropriate causal conclusion. This is a feature that RCTs share with a variety of other methods, which thus have equal claim to being a gold standard. This paper describes some of these other deductive methods and also some useful non-deductive methods, including the hypothetico-deductive method. It argues (...) that with all deductive methods, the benefit that the conclusions follow deductively in the ideal case comes with a great cost: narrowness of scope. This is an instance of the familiar trade-off between internal and external validity. RCTs have high internal validity but the formal methodology puts severe constraints on the assumptions a target population must meet to justify exporting a conclusion from the test population to the target. The paper reviews one such set of assumptions to show the kind of knowledge required. The overall conclusion is that to draw causal inferences about a target population, which method is best depends case-by-case on what background knowledge we have or can come to obtain. There is no gold standard. (shrink)

In clinical and agricultural trials, there is the danger that an experimental outcome appears to arise from the causal process or treatment one is interested in when, in reality, it was produced by some extraneous variation in the experimental conditions. The remedy prescribed by classical statisticians involves the procedure of randomization, whose effectiveness and appropriateness is criticized. An alternative, Bayesian analysis of experimental design, is shown, on the other hand, to provide a coherent and intuitively satisfactory solution to the problem.

This is an introductory 2001 textbook on probability and induction written by one of the world's foremost philosophers of science. The book has been designed to offer maximal accessibility to the widest range of students and assumes no formal training in elementary symbolic logic. It offers a comprehensive course covering all basic definitions of induction and probability, and considers such topics as decision theory, Bayesianism, frequency ideas, and the philosophical problem of induction. The key features of this book are a (...) lively and vigorous prose style; lucid and systematic organization and presentation of ideas; many practical applications; a rich supply of exercises drawing on examples from such fields as psychology, ecology, economics, bioethics, engineering, and political science; numerous brief historical accounts of how fundamental ideas of probability and induction developed; and a full bibliography of further reading. (shrink)

What kinds of evidence reliably support predictions of effectiveness for health and social care interventions? There is increasing reliance, not only for health care policy and practice but also for more general social and economic policy deliberation, on evidence that comes from studies whose basic logic is that of JS Mill's method of difference. These include randomized controlled trials, case–control studies, cohort studies, and some uses of causal Bayes nets and counterfactual-licensing models like ones commonly developed in econometrics. The topic (...) of this paper is the 'external validity' of causal conclusions from these kinds of studies. We shall argue two claims. Claim, negative: external validity is the wrong idea; claim, positive: 'capacities' are almost always the right idea, if there is a right idea to be had. If we are right about these claims, it makes big problems for policy decisions. Many advice guides for grading policy predictions give top grades to a proposed policy if it has two good Mill's-method-of difference studies that support it. But if capacities are to serve as the conduit for support from a method-of-difference study to an effectiveness prediction, much more evidence, and much different in kind, is required. We will illustrate the complexities involved with the case of multisystemic therapy, an internationally adopted intervention to try to diminish antisocial behaviour in young people. (shrink)

Peter Urbach has argued, on Bayesian grounds, that experimental randomization serves no useful purpose in testing causal hypothesis. I maintain that he fails to distinguish general issues of statistical inference from specific problems involved in identifying causes. I concede the general Bayesian thesis that random sampling is inessential to sound statistical inference. But experimental randomization is a different matter, and often plays an essential role in our route to causal conclusions.

Evidence-Based Medicine is a relatively new movement that seeks to put clinical medicine on a firmer scientific footing. I take it as uncontroversial that medical practice should be based on best evidence—the interesting questions concern the details. This paper tries to move towards a coherent and unified account of best evidence in medicine, by exploring in particular the EBM position on RCTs.

The claims of randomized controlled trials to be the gold standard rest on the fact that the ideal RCT is a deductive method: if the assumptions of the test are met, a positive result implies the appropriate causal conclusion. This is a feature that RCTs share with a variety of other methods, which thus have equal claim to being a gold standard. This article describes some of these other deductive methods and also some useful non-deductive methods, including the hypothetico-deductive method. (...) It argues that with all deductive methods, the benefit that the conclusions follow deductively in the ideal case comes with a great cost: narrowness of scope. This is an instance of the familiar trade-off between internal and external validity. RCTs have high internal validity but the formal methodology puts severe constraints on the assumptions a target population must meet to justify exporting a conclusion from the test population to the target. The article reviews one such set of assumptions to show the kind of knowledge required. The overall conclusion is that to draw causal inferences about a target population, which method is best depends case-by-case on what background knowledge we have or can come to obtain. There is no gold standard. (shrink)

In a randomized clinical trial (RCT), a group of patients, initially assembled through a mixture of deliberation (involving explicit inclusion and exclusion criteria) and serendipity (which patients happen to walk into which doctor’s clinic while the trial is in progress), are divided by some random process into an experimental group (members of which will receive the therapy under test) and a control group (members of which will receive some other treatment – perhaps placebo, perhaps the currently standard treatment for the (...) condition at issue). In a ‘double blind’ trial neither the patient nor the clinician knows to which of the groups a particular patient belongs. The results of double blind randomized controlled trials are almost universally regarded as providing the ‘gold standard’ for evidence in medicine. Fairly extreme claims to this effect can be found in the literature. For example the statistician Tukey wrote (1977, p. 679) “almost the only source of reliable evidence [in medicine] … is that obtained from … carefully conducted randomised trials”. And the clinician Victor Herbert claimed (1977, p. 690) “..the only source of reliable evidence rising to the level of proof about the usefulness of any new therapy is that obtained from wellplanned and carefully conducted randomized, and, where possible, coded (double blind) clinical trials. [Other] studies may point in a direction, but cannot be evidence as lawyers use the term evidence to mean something probative … [that is] tending to prove or actually proving”. Finally, the still very influential movement in favour of ‘Evidence Based Medicine’ (EBM) that began at McMaster University in the 1980s was initially often regarded as endorsing the claim that only RCTs provide real scientifically telling evidence. (shrink)