The evidence from randomized controlled trials (RCTs) is widely regarded as supplying the ‘gold standard’ in medicine—we may sometimes have to settle for other forms of evidence, but this is always epistemically second-best. But how well justified is the epistemic claim about the superiority of RCTs? This paper adds to my earlier (predominantly negative) analyses of the claims produced in favour of the idea that randomization plays a uniquely privileged epistemic role, by closely inspecting three related arguments from leading contributors (...) to the burgeoning field of probabilistic causality—Papineau, Cartwright and Pearl. It concludes that none of these further arguments supplies any practical reason for thinking of randomization as having unique epistemic power. IntroductionWhy the issue is of great practical importance—the ECMO casePapineau on the ‘virtues of randomization’Cartwright on causality and the ‘ideal’ randomized experimentPearl on randomization, nets and causesConclusion. (shrink)

Evidence-Based Medicine is a relatively new movement that seeks to put clinical med- icine on a firmer scientific footing. I take it as uncontroversial that medical practice should be based on best evidence-the interesting questions concern the details. This paper tries to move towards a coherent and unified account of best evidence in medicine, by exploring in particular the EBM position on RCTs (randomized controlled trials).

It is surely obvious that medicine, like any other rational activity, must be based on evidence. The interest is in the details: how exactly are the general principles of the logic of evidence to be applied in medicine? Focussing on the development, and current claims of the ‘Evidence-Based Medicine’ movement, this article raises a number of difficulties with the rationales that have been supplied in particular for the ‘evidence hierarchy’ and for the very special role within that hierarchy of randomized (...) controlled trials (and meta-analyses of the results of randomized controlled trials). The point is not at all to question the application of a scientific approach to evidence in medicine, but, on the contrary, to indicate a number of areas where philosophers of science can contribute to a proper implementation of exactly that scientific-evidential approach. (shrink)

Experimental balance is usually understood as the control for the value of the conditions, other than the one under study, which are liable to affect the result of a test. We will discuss three different approaches to balance. ‘Millean balance’ requires to identify and equalize ex ante the value of these conditions in order to conduct solid causal inferences. ‘Fisherian balance’ measures ex post the influence of uncontrolled conditions through the analysis of variance. In ‘efficiency balance’ the value of the (...) antecedent conditions is decided ex ante according to the efficiency they yield in the estimation of the treatment outcome. Against some old arguments by John Worrall, we will show that in both Fisherian and efficiency balance there are good reasons to randomize the allocation of treatments, in particular when there is no agreement among experimenters as to the antecedent conditions to be controlled for. (shrink)

It is sometimes thought that randomized study group allocation is uniquely proficient at producing comparison groups that are evenly balanced for all confounding causes. Philosophers have argued that in real randomized controlled trials this balance assumption typically fails. But is the balance assumption an important ideal? I run a thought experiment, the CONFOUND study, to answer this question. I then suggest a new account of causal inference in ideal and real comparative group studies that helps clarify the roles of confounding (...) variables and randomization. 1Confounders and Causes2The Balance Assumption3The CONFOUND Study 3.1CONFOUND 13.2CONFOUND 24Disjunction C and the Ideal Study 4.1The ultimate ‘other cause’: C4.2The ideal comparative group study4.3Required conditions for causal inference5Confounders as Causes, Confounders as Correlates6Summary. (shrink)

While practitioners think highly of randomized studies, some philosophers argue that there is no epistemic reason to randomize. Here I show that their arguments do not entail their conclusion. Moreover, I provide novel reasons for randomizing in the context of interventional studies. The overall discussion provides a unified framework for assessing baseline balance, one that holds for interventional and observational studies alike. The upshot: practitioners’ strong preference for randomized studies can be defended in some cases, while still offering a nuanced (...) approach to evidence-appraisal, one where not all non-randomized studies are treated equally. (shrink)

Peter Urbach has argued, on Bayesian grounds, that experimental randomization serves no useful purpose in testing causal hypothesis. I maintain that he fails to distinguish general issues of statistical inference from specific problems involved in identifying causes. I concede the general Bayesian thesis that random sampling is inessential to sound statistical inference. But experimental randomization is a different matter, and often plays an essential role in our route to causal conclusions.

Current debates surrounding the virtues and shortcomings of randomization are symptomatic of a lack of appreciation of the fact that causation can be inferred by two distinct inference methods, each requiring its own, specific experimental design. There is a non-statistical type of inference associated with controlled experiments in basic biomedical research; and a statistical variety associated with randomized controlled trials in clinical research. I argue that the main difference between the two hinges on the satisfaction of the comparability requirement, which (...) is in turn dictated by the nature of the objects of study, namely homogeneous or heterogeneous populations of biological systems. Among other things, this entails that the objection according to which randomized experiments fail to provide better evidence for causation because randomization cannot guarantee comparability is mistaken. (shrink)

"Correlation is not causation" is one of the mantras of the sciences—a cautionary warning especially to fields like epidemiology and pharmacology where the seduction of compelling correlations naturally leads to causal hypotheses. The standard view from the epistemology of causation is that to tell whether one correlated variable is causing the other, one needs to intervene on the system—the best sort of intervention being a trial that is both randomized and controlled. In this paper, we argue that some purely correlational (...) data contains information that allows us to draw causal inferences: statistical noise. Methods for extracting causal knowledge from noise provide us with an alternative to randomized controlled trials that allows us to reach causal conclusions from purely correlational data. (shrink)

In 1947 Donald Cary Williams claimed in The Ground of Induction to have solved the Humean problem of induction, by means of an adaptation of reasoning first advanced by Bernoulli in 1713. Later on David Stove defended and improved upon Williams’ argument in The Rational- ity of Induction (1986). We call this proposed solution of induction the ‘Williams-Stove sampling thesis’. There has been no lack of objections raised to the sampling thesis, and it has not been widely accepted. In our (...) opinion, though, none of these objections has the slightest force, and, moreover, the sampling thesis is undoubtedly true. What we will argue in this paper is that one particular objection that has been raised on numerous occasions is misguided. This concerns the randomness of the sample on which the inductive extrapolation is based. (shrink)

Experimental design is one aspect of a scientific method. A well-designed, properly conducted experiment aims to control variables in order to isolate and manipulate causal effects and thereby maximize internal validity, support causal inferences, and guarantee reliable results. Traditionally employed in the natural sciences, experimental design has become an important part of research in the social and behavioral sciences. Experimental methods are also endorsed as the most reliable guides to policy effectiveness. Through a discussion of some of the central concepts (...) associated with experimental design, including controlled variation and randomization, this chapter will provide a summary of key ethical issues that tend to arise in experimental contexts. In addition, by exploring assumptions about the nature of causation and by analyzing features of causal relationships, systems, and inferences in social contexts, this chapter will summarize the ways in which experimental design can undermine the integrity of not only social and behavioral research but policies implemented on the basis of such research. (shrink)

Debiasing procedures are experimental methods aimed at correcting errors arising from the cognitive biases of the experimenter. We discuss two of these methods, the predesignation rule and randomization, showing to what extent they are open to the experimenter’s regress: there is no metarule to prove that, after implementing the procedure, the experimental data are actually free from biases. We claim that, from a contractarian perspective, these procedures are nonetheless defensible since they provide a warrant of the impartiality of the experiment: (...) we only need proof that the result has not been intentionally manipulated for prima facie acceptance. (shrink)

Experimental balance is usually understood as the control for the value of the conditions, other than the one under study, which are liable to affect the result of a test. We discuss three different approaches to balance. ‘Millean balance’ requires identifying and equalizing ex ante the value of these conditions in order to conduct solid causal inferences. ‘Fisherian balance’ measures ex post the influence of uncontrolled conditions through the analysis of variance. In ‘efficiency balance’ the value of the antecedent conditions (...) is decided ex ante according to the efficiency they yield in the estimation of the treatment outcome. Against some old arguments by Worrall, we show that in both Fisherian and efficiency balance there are good reasons to randomize the allocation of treatments, in particular when there is no agreement among experimenters as to the antecedent conditions to be controlled for. (shrink)

Recently, certain philosophers of mathematics (Fallis [1997]; Womack and Farach [(1997]) have argued that there are no epistemic considerations that should stop mathematicians from using probabilistic methods to establish that mathematical propositions are true. However, mathematicians clearly should not use methods that are unreliable. Unfortunately, due to the fact that randomized algorithms are not really random in practice, there is reason to doubt their reliability. In this paper, I analyze the prospects for establishing that randomized algorithms are reliable. I end (...) by arguing that it would be inconsistent for mathematicians to suspend judgement on the truth of mathematical propositions on the basis of worries about the reliability of randomized algorithms. (shrink)

Evidence-based medicine (EBM) puts forward a hierarchy of evidence for informing therapeutic decisions. An unambiguous interpretation of how to apply EBM's hierarchy has not been provided in the clinical literature. However, as much as an interpretation is provided proponents suggest a categorical interpretation. The categorical interpretation holds that all the results of randomised trials always trump evidence from lower down the hierarchy when it comes to informing therapeutic decisions. Most of the critical replies to EBM react to this interpretation. While (...) proponents of EBM can avoid some of the problems raised by critics by suitably limited the claims made on behalf of the hierarchy, further problems arise. If EBM is to inform therapeutic decisions then a considerably more restricted, and context dependent interpretation of EBM's hierarchy is needed. (shrink)

Increasing philosophical attention is being directed to the rapidly growing discipline of evidence-based medicine. Philosophical discussions of EBM, however, remain narrowly focused on randomization, mechanisms, and the sociology of EBM. Other aspects of EBM have been all but ignored, including the nature of clinical reasoning and the question of whether it can be standardized; the application of EBM principles to the logic, value, and ethics of diagnosis and prognosis; evidence synthesis ; and the nature and ethics of placebo controls. Philosophical (...) analysis in each of these areas has the potential to enhance the discussion of EBM methodology and practice. (shrink)

In the recent philosophical literature, two questions have arisen concerning the status of natural selection: (1) Is it a population-level phenomenon, or is it an organism-level phenomenon? (2) Is it a causal process, or is it a purely statistical summary of lower-level processes? In an earlier work (Millstein, Br J Philos Sci, 57(4):627–653, 2006), I argue that natural selection should be understood as a population-level causal process, rather than a purely statistical population-level summation of lower-level processes or as an organism-level (...) causal process. In a 2009 essay entitled “Productivity, relevance, and natural selection,” Stuart Glennan argues in reply that natural selection is produced by causal pro- cesses operating at the level of individual organisms, but he maintains that there is no causal productivity at the population level. However, there are, he claims, many population-level properties that are causally relevant to the dynamics of evolution- ary processes. Glennan’s claims rely on a causal pluralism that holds that there are two types of causes: causal production and causal relevance. Without calling into question Glennan’s causal pluralism or his claims concerning the causal relevance of natural selection, I argue that natural selection does in fact exhibit causal production at the population level. It is true that natural selection does not fit with accounts of mechanisms that involve decomposition of wholes into parts, such as Glennan’s own. However, it does fit with causal production accounts that do not require decomposition, such as Salmon’s Mark Transmission account, given the extent to which populations act as interacting “objects” in the process of natural selection. (shrink)

Robert Truog describes the controversial randomized controlled trials (RCTs) of extracorporeal membrane oxygenation (ECMO) therapy in newborns. Because early results with ECMO indicated that it might be a great advance, saving many lives, Truog argues that ECMO should not have been tested using RCTs, but that a long-term, large-scale observational study of actual clinical practice should have been conducted instead. Central to Truog’s argument, however, is the idea that ECMO is an unusual case. Thus, it is an open question whether (...) Truog’s conclusions can be extended to other areas of medical research. In this paper, I look at epistemological and ethical issues arising in the care of patients with chronic diseases, using ECMO as a starting point. Both the similarities and the dissimilarities of these two cases highlight important issues in biomedical research and support a conclusion similar to Truog’s. Observational studies of clinical practice provide the best evidence to inform the treatment of patients with chronic disease. (shrink)

This chapter provides an introduction to the study of the philosophical notions of mechanisms and causality in biology and economics. This chapter sets the stage for this volume, Mechanism and Causality in Biology and Economics, in three ways. First, it gives a broad review of the recent changes and current state of the study of mechanisms and causality in the philosophy of science. Second, consistent with a recent trend in the philosophy of science to focus on scientific practices, it in (...) turn implies the importance of studying the scientific methods employed by researchers. Finally, by way of providing an overview of each chapter in the volume, this chapter demonstrates that biology and economics are two fertile fields for the philosophy of science and shows how biological and economic mechanisms and causality can be synthesized. (shrink)

En economía, la investigación se divide en dos grandes metodologías: los modelos teórico-matemáticos y los estudios empíricos. Estudiando modelos teóricos y métodos empíricos ) se da cuenta de las limitaciones de ambos métodos. Se concluye que ninguno de estos puede generar explicaciones de cómo en realidad suceden las cosas, sino que solo de cómo posiblemente suceden. La razón es que ambos necesitan un enlace interpretativo que permita extrapolar desde su propio sistema hacia un sistema objetivo. Los modelos tienen dominio general (...) y pueden dar cuenta de mecanismos. Los RCTs, al contrario, son válidos internamente y están conectados al mundo real, pero su dominio es muy específico. Aunque ninguno logre responder preguntas amplias de un fenómeno de interés, pueden complementarse para generar extrapolaciones más confiables sobre un sistema objetivo. Sin embargo, esto solo podrá hacerse si se conocen bien los mecanismos y el contexto en que ocurre una evidencia. Palabras clave: Causalidad, Mecanismos, Capacidades, Epistemología de la Economía, Generalización de Evidencia. (shrink)

The present paper deals with the problem of evaluating empirical evidence for therapeutic decisions in medicine. The article discusses the views of Nancy Cartwright and John Worrall on the function that randomization plays in ascertaining causal relations with reference to the therapies applied. The main purpose of the paper is to present a general idea of alternative method of evaluating empirical evidence. The method builds on data analysis that makes use of rough set theory. The first attempts to apply the (...) method show that it is an interesting alternative to randomized controlled trials. (shrink)

Urbach (1985) has concluded that the use of randomization in the design of clinical and agricultural trials is both inappropriate and ineffective. It is argued here that it is appropriate, as it eliminates the dependence of inference on the unknown precise physical model that underlies a set of observations, and effective, in that it is relatively simple to apply in practice compared with any competing method. Furthermore, it has been proven in practice.

Mayo (1987) sought to discredit Urbach's (1985) arguments against randomization as a universal requirement in clinical and agricultural trials. The present reply rebuts Mayo's criticisms.

Despite the consensus promoted by the evidence-based medicine framework, many authors continue to express doubts about the superiority of randomized controlled trials. This paper evaluates four objections targeting the legitimacy, feasibility, and extrapolation problems linked to the experimental practice of random allocation. I argue that random allocation is a methodologically sound and feasible practice contributing to the internal validity of controlled experiments dealing with heterogeneous populations. I emphasize, however, that random allocation is solely designed to ensure the validity of causal (...) inferences at the level of groups. By itself, random allocation cannot enhance test precision, doesn’t contribute to external validity, and limits the applicability of causal claims to individuals. (shrink)

We revisit John Worrall’s old but still prominent argument against the view that randomization balances the impact of both known and unknown confounders across the treatment and control arms. We argue that his argument involving indefinitely many possible confounders is at odds with statistical theory as it (1) presumes that the purpose of randomized studies is obtaining perfect point estimates for which perfect balance is needed; (2) mistakes equalizing each confounder with the overall (average) impact of all confounders, and (3) (...) assumes that the joint effect of an infinite series of confounders cannot be bounded. We defend the role of randomization in balancing the impact of confounders across the treatment and control arms by putting forward the statistical sense of the balance claim. It involves the following three commitments: (1) randomization balances confounders in expectancy, (2) for RCTs to deliver unbiased estimates of the causal effect (true average treatment effect), the balance in the average effect of all confounders and not balancing each confounder is sufficient, and (3) randomization allows for calculating the probability of deviating from the balance. The paper includes a review of how the balance claim has been understood so far and discusses recent arguments supporting randomization balancing the impact of confounders in expectancy and the crucial role of the average impact of all actual confounders, and shows how statistical analysis of RCTs conducted both at the design and analysis stage makes possible estimating the probabilities of deviating from the balance. (shrink)

In this paper I look at causality in the context of intervention research, and discuss some problems faced in the evaluation of causal hypotheses via interventions. I draw attention to a simple problem for evaluations that employ randomized controlled trials. The common alternative to randomized trials, the observational study, is shown to face problems of a similar nature. I then argue that these problems become especially acute in cases where the intervention is complex (i.e. that involves intervening in a complex (...) system). Finally, I consider and reject a possible resolution of the problem involving the simulation of complex interventions. The conclusion I draw from this is that we need to radically reframe the way we think about causal inference in complex intervention research. (shrink)

Why do classic biostatistical studies, alleged to provide causal explanations of effects, often fail? This article argues that in statistics-relevant areas of biology—such as epidemiology, population biology, toxicology, and vector ecology—scientists often misunderstand epistemic constraints on use of the statistical-significance rule (SSR). As a result, biologists often make faulty causal inferences. The paper (1) provides several examples of faulty causal inferences that rely on tests of statistical significance; (2) uncovers the flawed theoretical assumptions, especially those related to randomization, that likely (...) contribute to flawed biostatistics; (3) re-assesses the three classic (SSR-warrant, avoiding-selection-bias, and avoiding-confounders) arguments for using SSR only with randomization; and (4) offers five new reasons for biologists to use SSR only with randomized experiments. (shrink)

In their recent book, Is Inequality Bad for Our Health?, Daniels, Kennedy, and Kawachi claim that to “act justly in health policy, we must have knowledge about the causal pathways through which socioeconomic (and other) inequalities work to produce differential health outcomes.” One of the central problems with this approach is its dependency on “knowledge about the causal pathways.” A widely held belief is that the randomized clinical trial (RCT) is, and ought to be the “gold standard” of evaluating the (...) causal efficacy of interventions. However, often the only data available are non-experimental, observational data. For such data, the necessary randomization is missing. Because the randomization is missing, it seems to follow that it is not possible to make epistemically warranted claims about the causal pathways. Although we are not sanguine about the difficulty in using observational data to make warranted causal claims, we are not as pessimistic as those who believe that the only warranted causal claims are claims based on data from (idealized) RCTs. We argue that careful, thoughtful study design, informed by expert knowledge, that incorporates propensity score matching methods in conjunction with instrumental variable analyses, provides the possibility of warranted causal claims using observational data. (shrink)

Hierarchies of evidence are an important and influential tool for appraising evidence in medicine. In recent years, hierarchies have been formally adopted by organizations including the Cochrane Collaboration [1], NICE [2,3], the WHO [4], the US Preventive Services Task Force [5], and the Australian NHMRC [6,7]. The development of such hierarchies has been regarded as a central part of Evidence-Based Medicine, a movement within healthcare which prioritises the use of epidemiological evidence such as that provided by Randomised Controlled Trials. Philosophical (...) work on the methodology of medicine has so far mostly focused on claims about the superiority of RCTs, and hence has largely neglected the questions of what hierarchies are, what assumptions they require, and how they affect clinical practice. This thesis shows that there is great variation in the hierarchies defended and in the interpretations they are, and can be, given. The interpretative assumptions made in using hierarchies are crucial to the content and defensibility of the underlying philosophical commitments concerning evidence and medical practice. Once this variation is been identified, it becomes clear that the little philosophical work that has been done so far affects only some hierarchies, under some interpretations. Modest interpretations offered by La Caze [11], conditional hierarchies like GRADE [12-14], and heuristic approaches such as that defended by Howick et al. [15,16] all survive previous philosophical criticism. This thesis extends previous criticisms by arguing that modest interpretations are so weak as to be unhelpful for clinical practice; that GRADE and similar conditional models omit clinically relevant information, such as information about variation in treatments’ effects and the causes of different responses to therapy; and that heuristic approaches lack the necessary empirical support. The conclusion is that hierarchies in general embed untenable philosophical assumptions: principally that information about average treatment effects backed by high-quality evidence can justify strong recommendations, and that the impact of evidence from individual studies can and should be appraised in isolation. Hierarchies are a poor basis for the application of evidence in clinical practice. The Evidence-Based Medicine movement should move beyond them and explore alternative tools for appraising the overall evidence for therapeutic claims. (shrink)

Traditional Chinese Medicine has been criticized for being unscientific because the theory on which it is based involves entities like qi and ’meridians’ that appear ambiguous and because the internal ‘organs’ like the kidney and the spleen are very different from those of modern anatomy and physiology. Even more so, TCM methods of therapy based on the yin-yang principle, the model of the five elements, and the classification of illnesses according to standard constellations of symptoms are largely unproven by the (...) protocols of modern evidence-based medicine. This dissertation attempts to reconstruct TCM theory by: providing explanations of TCM entities as abstractions and constructs that relate to observable body functions and illness symptoms and interpreting TCM theory as comprising heuristic models that were constructed from clinical experience to fit empirical observations of illnesses and their treatments with herbal medications and acupuncture. It suggests that scientists should be less concerned with the ontological status of TCM entities and the epistemic credentials of TCM models than with the ability of these concepts and models to guide physicians in therapy. More importantly, it makes the argument that these models are testable using the methods of evidence-based medicine. There are methodological difficulties associated with randomized controlled trials partly because TCM treatments tend to be individualized and syndromes are dynamic in nature; observational trials may be more appropriate in some situations. It is also possible that, for patients who are more culturally attuned to TCM, the placebo effect is strongly at play and may render the real effects of TCM treatments harder to tease out in clinical trials. The dissertation concludes that the main postulates of TCM should be put to rigorous test. The result may be a leaner but more robust theory, with parts that do not stand up to the test being rejected or modified, and a possible acceptance of its more modest therapeutic claims for a limited range of pathological conditions like pain and chronic illnesses. (shrink)