What kinds of evidence reliably support predictions of effectiveness for health and social care interventions? There is increasing reliance, not only for health care policy and practice but also for more general social and economic policy deliberation, on evidence that comes from studies whose basic logic is that of JS Mill's method of difference. These include randomized controlled trials, case–control studies, cohort studies, and some uses of causal Bayes nets and counterfactual-licensing models like ones commonly developed in econometrics. The topic (...) of this paper is the 'external validity' of causal conclusions from these kinds of studies. We shall argue two claims. Claim, negative: external validity is the wrong idea; claim, positive: 'capacities' are almost always the right idea, if there is a right idea to be had. If we are right about these claims, it makes big problems for policy decisions. Many advice guides for grading policy predictions give top grades to a proposed policy if it has two good Mill's-method-of difference studies that support it. But if capacities are to serve as the conduit for support from a method-of-difference study to an effectiveness prediction, much more evidence, and much different in kind, is required. We will illustrate the complexities involved with the case of multisystemic therapy, an internationally adopted intervention to try to diminish antisocial behaviour in young people. (shrink)

The DSM is the main classification of mental disorders used by psychiatrists in the United States and, increasingly, around the world. Although widely used, the DSM has come in for fierce criticism, with many commentators believing it to be conceptually flawed in a variety of ways. This paper assesses some of these philosophical worries. The first half of the paper asks whether the project of constructing a classification of mental disorders that ‘cuts nature at the joints’ makes sense. What is (...) mental disorder? Are types of mental disorder natural kinds (that is, are the distinctions between them objective and of fundamental theoretical importance, as are, say, the distinctions between the chemical elements)? The second half of the paper addresses epistemic worries. Even if types of mental disorder are natural kinds there may be reason to doubt that the DSM will come to reflect their natural structure. In particular, I examine the extent to which the DSM is theory-laden, and look at how it has been shaped by social and financial factors. Ultimately, I conclude that although the DSM is of immense practical importance it is not likely to become the best possible classification of mental disorders. (shrink)

The claims of randomized controlled trials to be the gold standard rest on the fact that the ideal RCT is a deductive method: if the assumptions of the test are met, a positive result implies the appropriate causal conclusion. This is a feature that RCTs share with a variety of other methods, which thus have equal claim to being a gold standard. This article describes some of these other deductive methods and also some useful non-deductive methods, including the hypothetico-deductive method. (...) It argues that with all deductive methods, the benefit that the conclusions follow deductively in the ideal case comes with a great cost: narrowness of scope. This is an instance of the familiar trade-off between internal and external validity. RCTs have high internal validity but the formal methodology puts severe constraints on the assumptions a target population must meet to justify exporting a conclusion from the test population to the target. The article reviews one such set of assumptions to show the kind of knowledge required. The overall conclusion is that to draw causal inferences about a target population, which method is best depends case-by-case on what background knowledge we have or can come to obtain. There is no gold standard. (shrink)

The evidence from randomized controlled trials (RCTs) is widely regarded as supplying the ‘gold standard’ in medicine—we may sometimes have to settle for other forms of evidence, but this is always epistemically second-best. But how well justified is the epistemic claim about the superiority of RCTs? This paper adds to my earlier (predominantly negative) analyses of the claims produced in favour of the idea that randomization plays a uniquely privileged epistemic role, by closely inspecting three related arguments from leading contributors (...) to the burgeoning field of probabilistic causality—Papineau, Cartwright and Pearl. It concludes that none of these further arguments supplies any practical reason for thinking of randomization as having unique epistemic power. IntroductionWhy the issue is of great practical importance—the ECMO casePapineau on the ‘virtues of randomization’Cartwright on causality and the ‘ideal’ randomized experimentPearl on randomization, nets and causesConclusion. (shrink)

For evidence-based practice and policy, randomised controlled trials (RCTs) are the current gold standard. But exactly why? We know that RCTs do not, without a series of strong assumptions, warrant predictions about what happens in practice. But just what are these assumptions? I maintain that, from a philosophical stance, answers to both questions are obscured because we don't attend to what causal claims say. Causal claims entering evidence-based medicine at different points say different things and, I would suggest, failure to (...) attend to these differences makes much current guidance about evidence for medical and social policy misleading. (shrink)