This text provides a critique of the subjective Bayesian view of statistical inference, and proposes the author's own error-statistical approach as an alternative framework for the epistemology of experiment. It seeks to address the needs of researchers who work with statistical analysis.

The claims of RCTs to be the gold standard rest on the fact that the ideal RCT is a deductive method: if the assumptions of the test are met, a positive result implies the appropriate causal conclusion. This is a feature that RCTs share with a variety of other methods, which thus have equal claim to being a gold standard. This paper describes some of these other deductive methods and also some useful non-deductive methods, including the hypothetico-deductive method. It argues (...) that with all deductive methods, the benefit that the conclusions follow deductively in the ideal case comes with a great cost: narrowness of scope. This is an instance of the familiar trade-off between internal and external validity. RCTs have high internal validity but the formal methodology puts severe constraints on the assumptions a target population must meet to justify exporting a conclusion from the test population to the target. The paper reviews one such set of assumptions to show the kind of knowledge required. The overall conclusion is that to draw causal inferences about a target population, which method is best depends case-by-case on what background knowledge we have or can come to obtain. There is no gold standard. (shrink)

An astonishing volume and diversity of evidence is available for many hypotheses in the biomedical and social sciences. Some of this evidence—usually from randomized controlled trials (RCTs)—is amalgamated by meta-analysis. Despite the ongoing debate regarding whether or not RCTs are the ‘gold-standard’ of evidence, it is usually meta-analysis which is considered the best source of evidence: meta-analysis is thought by many to be the platinum standard of evidence. However, I argue that meta-analysis falls far short of that standard. Different meta-analyses (...) of the same evidence can reach contradictory conclusions. Meta-analysis fails to provide objective grounds for intersubjective assessments of hypotheses because numerous decisions must be made when performing a meta-analysis which allow wide latitude for subjective idiosyncrasies to influence its outcome. I end by suggesting that an older tradition of evidence in medicine—the plurality of reasoning strategies appealed to by the epidemiologist Sir Bradford Hill—is a superior strategy for assessing a large volume and diversity of evidence. (shrink)

Proponents of Evidence-based medicine (EBM) do not provide a clear role for basic science in therapeutic decision making. Of what they do say about basic science, most of it is negative. Basic science resides on the lower tiers of EBM's hierarchy of evidence. Therapeutic decisions, according to proponents of EBM, should be informed by evidence from randomised studies (and systematic reviews of randomised studies) rather than basic science. A framework of models explicates the links between the mechanisms of basic science, (...) experimental inquiry, and observed data. Relying on the framework of models I show that basic science often plays a role not only in specifying experiments, but also analysing and interpreting the data that is provided. Further, and contradicting what is implied in EBM's hierarchy of evidence, appeals to basic science are often required to apply clinical research to therapeutic questions. © 2010 Springer Science+Business Media B.V. (shrink)

The claims of randomized controlled trials to be the gold standard rest on the fact that the ideal RCT is a deductive method: if the assumptions of the test are met, a positive result implies the appropriate causal conclusion. This is a feature that RCTs share with a variety of other methods, which thus have equal claim to being a gold standard. This article describes some of these other deductive methods and also some useful non-deductive methods, including the hypothetico-deductive method. (...) It argues that with all deductive methods, the benefit that the conclusions follow deductively in the ideal case comes with a great cost: narrowness of scope. This is an instance of the familiar trade-off between internal and external validity. RCTs have high internal validity but the formal methodology puts severe constraints on the assumptions a target population must meet to justify exporting a conclusion from the test population to the target. The article reviews one such set of assumptions to show the kind of knowledge required. The overall conclusion is that to draw causal inferences about a target population, which method is best depends case-by-case on what background knowledge we have or can come to obtain. There is no gold standard. (shrink)

The philosophy of evidence-based medicine -- What is EBM? -- What is good evidence for a clinical decision? -- Ruling out plausible rival hypotheses and confounding factors : a method -- Resolving the paradox of effectiveness : when do observational studies offer the same degree of evidential support as randomized trials? -- Questioning double blinding as a universal methodological virtue of clinical trials : resolving the Philip's paradox -- Placebo controls : problematic and misleading baseline measures of effectiveness -- Questioning (...) the methodological superiority of "placebo" over "active" controlled trials -- Examining the paradox that traditional roles for mechanistic reasoning and expert -- Judgment have been up-ended by EBM -- A qualified defence of the EBM stance on mechanistic reasoning -- Knowledge that versus knowledge how : situating the EBM position on expert clinical judgment -- Moving EBM forward. (shrink)

Evidence-Based Medicine is a relatively new movement that seeks to put clinical medicine on a firmer scientific footing. I take it as uncontroversial that medical practice should be based on best evidence—the interesting questions concern the details. This paper tries to move towards a coherent and unified account of best evidence in medicine, by exploring in particular the EBM position on RCTs.

Scientifi c theories and hypotheses make claims that go well beyond what we can immediately observe. How can we come to know whether such claims are true? The obvious approach is to see what a hypothesis says about the observationally accessible parts of the world. If it gets that wrong, then it must be false; if it gets that right, then it may have some claim to being true. Any sensible a empt to construct a logic that captures how we (...) may come to reasonably believe the falsehood or truth of scientifi c hypotheses must be built on this idea. Philosophers refer to such logics as logics of confi rmation or as confi rmation theories. (shrink)