Public policymakers and institutional decision-makers routinely face questions about whether interventions “work”: does universal basic income improve people’s welfare and stimulate entrepreneurial activity? Do gated alleyways reduce burglaries or merely shift the crime burden to neighbouring communities? What is the most cost-effective way to improve students’ reading abilities? These are empirical questions that seem best answered by looking at the world, rather than trusting speculations about what will be effective. Evidence-based policy (EBP) is a movement that concretizes this intuition. It (...) maintains that policy should be based on evidence of “what works.” Not any evidence will do, however. Following on the heels of its intellectual progenitor, evidence-based medicine (EBM), EBP insists on the use of high-quality evidence produced in accordance with rigorous methodological standards. Though intuitively compelling, EBP has attracted significant criticism from methodologists, political scientists, and philosophers. This chapter provides a critical overview of EBP through a philosophical lens, reviewing and discussing some of the most pressing challenges that EBP faces and outlining some proposals for improving it. I first provide a brief overview of EBP and distinguish a broader and narrower understanding of it. I then review existing lines of criticism of EBP, two of which are considered in detail. The first elaborates how EBP struggles with extrapolation, that is, the use of evidence from study populations to make inferences about the effects of policies on novel target populations. The second maintains that EBP’s methodological tenets are deeply entwined with moral and political values, which can threaten EBP’s promise to promote objectivity in policymaking. Finally, I make some proposals for how EBP could be improved going forward. (shrink)

There are two important traditions in the philosophy of induction. According to one tradition, which has dominated for the last couple of centuries, inductive arguments are warranted by rules. Bayesianism is the most popular view within this tradition. Rules of induction provide functional accounts of inductive support, but no rule is universal; hence, no rule is by itself an accurate model of inductive support. According to another tradition, inductive arguments are not warranted by rules but by matters of fact. Norton’s (...) material theory of induction (MTI) is an influential view within this tradition. Norton’s MTI fails to provide an account of inductive support, but it provides a good account of inductive warrant that can help us define the domain of validity of each rule of induction. Despite their limitations, both approaches to induction are illuminating important aspects of how relations of inductive support work. In this article I present a hybrid theory of induction (HTI), in which I acknowledge and articulate the role of both rules and matters of fact in our understanding of inductive support. According to the HTI, rules of induction accurately describe relations of inductive support when they are warranted, and a rule of induction is warranted if the right facts about the matter of the induction obtain. Crucially, the HTI allows us to address the main challenges that each tradition faces while retaining their strengths, thus obtaining a functional and accurate account of inductive support. The HTI also provides a useful general framework to examine and understand induction, to make sense of how different rules of induction can coexist and to tackle some problems in epistemology. Moreover, the HTI allows us to clarify and resolve some current debates on induction, like the apparent divide between Norton and rule-based theorists. (shrink)

In the biomedical, behavioural and social sciences, the leading method used to estimate causal effects is commonly randomised controlled trials (RCTs) that are generally viewed as both the source and justification of the most valid evidence. In studying the foundation and theory behind RCTs, the existing literature analyses important single issues and biases in isolation that influence causal outcomes in trials (such as randomisation, statistical probabilities and placebos). The common account of biased causal inference is described in a general way (...) in terms of probabilistic imbalances between trial groups. This paper expands the common account of causal bias by distinguishing between the range of biases arising between trial groups but also within one of the groups or across the entire sample during trial design, implementation and analysis. This is done by providing concrete examples from highly influential RCT studies. In going beyond the existing RCT literature, the paper provides a broader, practice-based account of causal bias that specifies the between-group, within-group and across-group biases that affect the estimated causal results of trials – impacting both the effect size and statistical significance. Within this expanded framework, we can better identify the range of different types of biases we face in practice and address the central question about the overall validity of the RCT method and its causal claims. A study can face several smaller biases (related simultaneously to a smaller sample, smaller estimated effect, greater unblinding etc.) that generally add up to greater aggregate bias. Though difficult to measure precisely, it is important to assess and provide information in studies on how much different sources of bias, combined, can explain the estimated causal effect. The RCT method is thereby often the best we have to inform our policy decisions – and the evidence is strengthened when combined with multiple studies and other methods. Yet there is room for continually improving trials and identifying ways to reduce biases they face and to increase their overall validity. Implications are discussed. (shrink)

Agent-based models (ABMs) are one type of simulation model used in the context of the COVID-19 pandemic. In contrast to equation-based models, ABMs are algorithms that use individual agents and attribute changing characteristics to each one, multiple times during multiple iterations over time. This paper focuses on three philosophical aspects of ABMs as models of causal mechanisms, as generators of emergent phenomena, and as providers of explanation. Based on my discussion, I conclude that while ABMs cannot help much with causal (...) inference, they can be viewed as etio-prognostic explanations of illness occurrence and outcome. (shrink)

Traditionally, social scientists perceived causality as regularity. As a consequence, qualitative comparative case study research was regarded as unsuitable for drawing causal inferences since a few cases cannot establish regularity. The dominant perception of causality has changed, however. Nowadays, social scientists define and identify causality through the counterfactual effect of a treatment. This brings causal inference in qualitative comparative research back on the agenda since comparative case studies can identify counterfactual treatment effects. We argue that the validity of causal inferences (...) from the comparative study of cases depends on the employed case-selection algorithm. We employ Monte Carlo techniques to demonstrate that different case-selection rules strongly differ in their ex ante reliability for making valid causal inferences and identify the most and the least reliable case selection rules. (shrink)

This paper defends the use of quasi-experiments for causal estimation in economics against the widespread objection that quasi-experimental estimates lack external validity. The defence is that quasi-experimental replication of estimates can yield defeasible evidence for external validity. The paper then develops a different objection. The stable unit treatment value assumption, on which quasi-experiments rely, is argued to be implausible due to the influence of social interaction effects on economic outcomes. A more plausible stable marginal unit treatment value assumption is proposed, (...) but it is demonstrated to severely limit the usefulness of quasi-experiments for economic policy evaluation. (shrink)

Experimental design is one aspect of a scientific method. A well-designed, properly conducted experiment aims to control variables in order to isolate and manipulate causal effects and thereby maximize internal validity, support causal inferences, and guarantee reliable results. Traditionally employed in the natural sciences, experimental design has become an important part of research in the social and behavioral sciences. Experimental methods are also endorsed as the most reliable guides to policy effectiveness. Through a discussion of some of the central concepts (...) associated with experimental design, including controlled variation and randomization, this chapter will provide a summary of key ethical issues that tend to arise in experimental contexts. In addition, by exploring assumptions about the nature of causation and by analyzing features of causal relationships, systems, and inferences in social contexts, this chapter will summarize the ways in which experimental design can undermine the integrity of not only social and behavioral research but policies implemented on the basis of such research. (shrink)

Medical research makes intensive use of statistics in order to support its claims. In this paper we make explicit an epistemological tension between the conduct of clinical trials and their interpretation: statistical evidence is sometimes discarded on the basis of an underlined Bayesian reasoning. We suggest that acknowledging the potentiality of Bayesian statistics might contribute to clarify and improve comprehension of medical research. Nevertheless, despite Bayesianism may provide a better account for scientific inference with respect to the standard frequentist approach, (...) Bayesian statistics is rarely adopted in clinical research. The main reason lies in the supposed subjective elements characterizing this perspective. Hence, we discuss this objection presenting the so-called Reference analysis, a formal method which has been developed in the context of objective Bayesian statistics in order to define priors which have a minimal or null impact on posterior probabilities. Furthermore, according to this method only available data are relevant sources of information, so that it resists the most common criticisms against Bayesianism. (shrink)

ABSTRACT“Evidence-based” methods, which most prominently include randomized controlled trials, have gained increasing purchase as the “gold standard” for assessing the effect of public policies. But the enthusiasm for evidence-based research overlooks questions about the reliability and applicability of experimental findings to diverse real-world settings. Perhaps surprisingly, a qualitative study of British educators suggests that they are aware of these limitations and therefore take evidence-based findings with a much larger grain of salt than do policy makers. Their experience suggests that the (...) real world is more heterogeneous than the world imagined by evidence-based policy enthusiasts. (shrink)

After describing the disorder of psychopathy, I examine the theories and the evidence concerning the psychopaths’ deficient moral capacities. I first examine whether or not psychopaths can pass tests of moral knowledge. Most of the evidence suggests that they can. If there is a lack of moral understanding, then it has to be due to an incapacity that affects not their declarative knowledge of moral norms, but their deeper understanding of them. I then examine two suggestions: it is their deficient (...) practical reason or their stunted emotions that are at fault. The evidence supports both explanations. I conclude with an overview of the debate concerning whether they are morally or legally responsible for their actions. (shrink)

Philosophical discussions on causal inference in medicine are stuck in dyadic camps, each defending one kind of evidence or method rather than another as best support for causal hypotheses. Whereas Evidence Based Medicine advocates the use of Randomised Controlled Trials and systematic reviews of RCTs as gold standard, philosophers of science emphasise the importance of mechanisms and their distinctive informational contribution to causal inference and assessment. Some have suggested the adoption of a pluralistic approach to causal inference, and an inductive (...) rather than hypothetico-deductive inferential paradigm. However, these proposals deliver no clear guidelines about how such plurality of evidence sources should jointly justify hypotheses of causal associations. We here develop such guidelines by first giving a philosophical analysis of the underpinnings of Hill’s viewpoints on causality. We then put forward an evidence-amalgamation framework adopting a Bayesian net approach to model causal inference in pharmacology for the assessment of harms. Our framework accommodates a number of intuitions already expressed in the literature concerning the EBM vs. pluralist debate on causal inference, evidence hierarchies, causal holism, relevance, and reliability. (shrink)

The disconnect problem arises wherever there is ongoing and severe discordance between the scientific assessment of a politically relevant issue, and the politics and legislation of said issue. Here, we focus on the disconnect problem as it arises in the case of climate change, diagnosing a failure to respect the necessary tradeoff between authority and autonomy within a public institution like science. After assessing the problematic deployment of scientific authority in this arena, we offer suggestions for how to mitigate climate (...) change’s particular disconnect problem, as well as more general proposals for reforming science advising. (shrink)

With the ascent of modern epidemiology in the Twentieth Century came a new standard model of prediction in public health and clinical medicine. In this article, we describe the structure of the model. The standard model uses epidemiological measures-most commonly, risk measures-to predict outcomes (prognosis) and effect sizes (treatment) in a patient population that can then be transformed into probabilities for individual patients. In the first step, a risk measure in a study population is generalized or extrapolated to a target (...) population. In the second step, the risk measure is particularized or transformed to yield probabilistic information relevant to a patient from the target population. Hence, we call the approach the Risk Generalization-Particularization (Risk GP) Model. There are serious problems at both stages, especially with the extent to which the required assumptions will hold and the extent to which we have evidence for the assumptions. Given that there are other models of prediction that use different assumptions, we should not inflexibly commit ourselves to one standard model. Instead, model pluralism should be standard in medical prediction. (shrink)

Harms of medical interventions are systematically underestimated in clinical research. Numerous factors—conceptual, methodological, and social—contribute to this underestimation. I articulate the depth of such underestimation by describing these factors at the various stages of clinical research. Before any evidence is gathered, the ways harms are operationalized in clinical research contributes to their underestimation. Medical interventions are first tested in phase 1 ‘first in human’ trials, but evidence from these trials is rarely published, despite the fact that such trials provide the (...) foundation for assessing the harm profile of medical interventions. If a medical intervention is deemed safe in a phase 1 trial, it is tested in larger phase 2 and 3 clinical trials. One way to think about the problem of underestimating harms is in terms of the statistical ‘power’ of a clinical trial—the ability of a trial to detect a difference of a certain effect size between the experimental group and the control group. Power is normally thought to be pertinent to detecting benefits of medical interventions. It is important, though, to distinguish between the ability of a trial to detect benefits and the ability of a trial to detect harms. I refer to the former as power-B and the latter as power-H. I identify several factors that maximize power-B by sacrificing powerH in phase 3 clinical trials. If a medical intervention is approved for general use, it is evaluated by phase 4 post-market surveillance. Phase 4 surveillance of harms further contributes to underestimating the harm profile of medical interventions. At every stage of clinical research the hunt for harms is shrouded in secrecy, which further contributes to the underestimation of the harm profiles of medical interventions. (shrink)

John Worrall and Nancy Cartwright have both argued that randomized controlled trials are “testing the wrong theory.” They claim that RCTs are designed to test inferences about the causal relationships in the study population, but this does not guarantee a justified inference about the causal relationships in the more diverse population in clinical practice. In this article I argue that the epistemology of theory testing in trials is more complicated than either Worrall’s or Cartwright’s accounts suggest. I illustrate this more (...) complex theoretical structure with case studies in medical theory testing from Alzheimer’s research and anticancer drugs in personalized medicine. (shrink)

This paper outlines the methodological and empirical limitations of analysing the potential relationship between complex social phenomena such as democracy and inequality. It shows that the means to assess how they may be related is much more limited than recognised in the existing literature that is laden with contradictory hypotheses and findings. Better understanding our scientific limitations in studying this potential relationship is important for research and policy because many leading economists and other social scientists such as Acemoglu and Robinson (...) mistakenly claim to identify causal linkages between inequality and democracy but at times still inform policy. In contrast to the existing literature, the paper argues that ‘structural’ or ‘causal’ mechanisms that may potentially link the distribution of economic wealth and different political regimes will remain unknown given reasons such as their highly complex and idiosyncratic characteristics, fundamental econometric constraints and analysis at the macro-level. Neither new data sources, different analysed time periods nor new data analysis techniques can resolve this question and provide robust, general conclusions about this potential relationship across countries. Researchers are thus restricted to exploring rough correlations over specific time periods and geographic contexts with imperfect data that are very limited for cross-country comparisons. (shrink)

This thesis examines philosophical controversies surrounding the evaluation of medical treatments, with a focus on the evidential roles of randomised trials and mechanisms in Evidence-Based Medicine. Current 'best practice' usually involves excluding non-randomised trial evidence from systematic reviews in cases where randomised trials are available for inclusion in the reviews. The first paper challenges this practice and evaluates whether adding of evidence from non-randomised trials might improve the quality and precision of some systematic reviews. The second paper compares the alleged (...) methodological benefits of randomised trials over observational studies for investigating treatment benefits. It suggests that claims about the superiority of well-conducted randomised controlled trials over well-conducted observational studies are justified, especially when results from the two methods are contradictory. The third paper argues that postulating the unpredictability paradox in systematic reviews when no detectable empirical differences can be found requires further justification. The fourth paper examines the problem of absence causation in the context of explaining causal mechanisms and argues that a recent solution is incomplete and requires further justification. Solving the problem by describing absences as causes of 'mechanism failure' fails to take into account the effects of absences that lead to vacillating levels of mechanism functionality. The fifth paper criticises literature that has emphasised functioning versus 'broken' or 'non-functioning' mechanisms emphasising that many diseases result from increased or decreased mechanism function, rather than complete loss of function. Mechanistic explanations must account for differences in the effectiveness of performed functions, yet current philosophical mechanistic explanations do not achieve this. The last paper argues that the standard of evidence embodied in the ICE theory of technological function is too permissive for evaluating whether the proposed functions of medical technologies have been adequately assessed and correctly ascribed. It argues that high-quality evidence from clinical studies is necessary to justify functional ascriptions to health care technologies. (shrink)

Consensus conferences are social techniques which involve bringing together a group of scientific experts, and sometimes also non-experts, in order to increase the public role in science and related policy, to amalgamate diverse and often contradictory evidence for a hypothesis of interest, and to achieve scientific consensus or at least the appearance of consensus among scientists. For consensus conferences that set out to amalgamate evidence, I propose three desiderata: Inclusivity, Constraint, and Evidential Complexity. Two examples suggest that consensus conferences can (...) readily satisfy Inclusivity and Evidential Complexity, but consensus conferences do not as easily satisfy Constraint. I end by discussing the relation between social inclusivity and the three desiderata. (shrink)

Proponents of evidence-based medicine have argued convincingly for applying this scientific method to medicine. However, the current methodological framework of the EBM movement has recently been called into question, especially in epidemiology and the philosophy of science. The debate has focused on whether the methodology of randomized controlled trials provides the best evidence available. This paper attempts to shift the focus of the debate by arguing that clinical reasoning involves a patchwork of evidential approaches and that the emphasis on evidence (...) hierarchies of methodology fails to lend credence to the common practice of corroboration in medicine. I argue that the strength of evidence lies in the evidence itself, and not the methodology used to obtain that evidence. Ultimately, when it comes to evaluating the effectiveness of medical interventions, it is the evidence obtained from the methodology rather than the methodology that should establish the strength of the evidence. (shrink)

The present paper deals with the problem of evaluating empirical evidence for therapeutic decisions in medicine. The article discusses the views of Nancy Cartwright and John Worrall on the function that randomization plays in ascertaining causal relations with reference to the therapies applied. The main purpose of the paper is to present a general idea of alternative method of evaluating empirical evidence. The method builds on data analysis that makes use of rough set theory. The first attempts to apply the (...) method show that it is an interesting alternative to randomized controlled trials. (shrink)

Over the past several decades, we devoted much energy to generating, reviewing and summarizing evidence. We have given far less attention to the issue of how to thoughtfully apply the evidence once we have it. That’s fine if all we care about is that our clinical decisions are evidence-based, but not so good if we also want them to be well-reasoned. Let us not forget that evidence based medicine (EBM) grew out of an interest in making medicine ‘rational’, with the (...) idea that rational clinical evaluations should be evidence-based. I agree with the uncontroversial statement that the best decision is supported, at least in part, by the best available evidence. Rationality, however, is constituted by reasoning, not evidence. Complete arguments are necessary for rational evaluations, arguments that begin with general evidence and end in a conclusion about a particular patient. In order to traverse these inferential gaps, medicine must address the issue of how to establish, as an intermediate premise, what the evidence has to say about the efficacy of an intervention for particular patients in a particular practice setting. (shrink)

Evidence hierarchies are widely used to assess evidence in systematic reviews of medical studies. I give several arguments against the use of evidence hierarchies. The problems with evidence hierarchies are numerous, and include methodological shortcomings, philosophical problems, and formal constraints. I argue that medical science should not employ evidence hierarchies, including even the latest and most-sophisticated of such hierarchies.

Medical scientists employ ‘quality assessment tools’ (QATs) to measure the quality of evidence from clinical studies, especially randomized controlled trials (RCTs). These tools are designed to take into account various methodological details of clinical studies, including randomization, blinding, and other features of studies deemed relevant to minimizing bias and error. There are now dozens available. The various QATs on offer differ widely from each other, and second-order empirical studies show that QATs have low inter-rater reliability and low inter-tool reliability. This (...) is an instance of a more general problem I call the underdetermination of evidential significance. Disagreements about the strength of a particular piece of evidence can be due to different—but in principle equally good—weightings of the fine-grained methodological features which constitute QATs. (shrink)

The issue of how to incorporate the individual's first‐hand experience of illness into broader medical understanding is a major question in medical theory and practice. In a philosophical context, phenomenology, with its emphasis on the subject's perception of phenomena as the basis for knowledge and its questioning of naturalism, seems an obvious candidate for addressing these issues. This is a review of current phenomenological approaches to medicine, looking at what has motivated this philosophical approach, the main problems it faces and (...) suggesting how it might become a useful philosophical tool within medicine, with its own individual, but interrelated, contribution to make to current medical debates. After the general background, there is a brief summary of phenomenological ideas and their current usage in a medical context. Next is a critique of four key claims within current phenomenological medical works, concerning both the role phenomenology plays and the supposedly clear divide between phenomenology and other approaches. There are significant problems within these claims, largely because they overlook the complexity of the questions they consider. Finally, there is some more in‐depth examination of phenomenology itself and the true complexity of phenomenological debate concerning subjectivity. The aim is to show that it will be both more productive and truer to phenomenology itself, if we use phenomenology as a philosophical method for explicating and gaining deeper understanding of complex and fundamental problems, which are central to medicine, rather than as providing simple, but flawed solutions. (shrink)

I describe two traditions of philosophical accounts of evidence: one characterizes the notion in terms of signs of success, the other characterizes the notion in terms of conditions of success. The best examples of the former rely on the probability calculus, and have the virtues of generality and theoretical simplicity. The best examples of the latter describe the features of evidence which scientists appeal to in practice, which include general features of methods, such as quality and relevance, and general features (...) of evidence, such as patterns in data, concordance with other evidence, and believability of the evidence. Two infamous episodes from biomedical research help to illustrate these features. Philosophical characterization of these latter features—conditions of success—has the virtue of potential relevance to, and descriptive accuracy of, practices of experimental scientists. (shrink)

As the quality movement in health care now enters its fourth decade, the language of quality is ubiquitous. Practitioners, organizations, and government agencies alike vociferously testify their commitments to quality and accept numerous forms of governance aimed at improving quality of care. Remarkably, the powerful phrase ‘‘quality of care’’ is rarely defined in the health care literature. Instead it operates as an accepted and assumed goal worth pursuing. The status of evidence-based medicine, for instance, hinges on its ability to improve (...) quality of care, and efforts are made by both proponents and detractors to unpack the contents and outcomes of evidence-based practice while the contents of ‘‘quality of care’’ are presumed to be understood. Because the goals of medicine are far from obvious, this paper investigates the neglected term, ‘‘quality of care,’’ in an effort to understand what it is that health care practices are so uncritically assumed to be striving for. Finding lack of consensus on the terminology in the quality literature, I propose that the term operates rhetorically by way of persuasive appeal (and lack of descriptive meaning). Unsatisfied that ‘‘quality of care’’ operates as a mere buzzword in morally contentious debates over resource allocation and duties of care, I implore health care communities to go beyond mere commitments to quality and, instead, to focus attention on the difficult task of specifying what counts as quality care within an economically constrained health care system. (shrink)

Because few would object to evidence-based medicine’s (EBM) principal task of basing medical decisionmaking on the most judicious and up-to-date evidence, the debate over this prolific movement may seem puzzling. Who, one may ask, could be against evidence (Carr-Hill, 2006)? Yet this question belies the sophistication of the evidence-based movement. This chapter presents the evidence-based approach as a socio-medical phenomenon and seeks to explain and negotiate the points of disagreement between supporters and detractors. This is done by casting EBM as (...) more than the simple application of research findings to clinical care and improved health outcomes, but rather an umbrella term that harnesses a specific set of pedagogical objectives (some rather radical) under a name that makes it difficult to argue against. (shrink)

An astonishing volume and diversity of evidence is available for many hypotheses in the biomedical and social sciences. Some of this evidence—usually from randomized controlled trials (RCTs)—is amalgamated by meta-analysis. Despite the ongoing debate regarding whether or not RCTs are the ‘gold-standard’ of evidence, it is usually meta-analysis which is considered the best source of evidence: meta-analysis is thought by many to be the platinum standard of evidence. However, I argue that meta-analysis falls far short of that standard. Different meta-analyses (...) of the same evidence can reach contradictory conclusions. Meta-analysis fails to provide objective grounds for intersubjective assessments of hypotheses because numerous decisions must be made when performing a meta-analysis which allow wide latitude for subjective idiosyncrasies to influence its outcome. I end by suggesting that an older tradition of evidence in medicine—the plurality of reasoning strategies appealed to by the epidemiologist Sir Bradford Hill—is a superior strategy for assessing a large volume and diversity of evidence. (shrink)

My dissertation combines philosophy of science and political philosophy. Drawing directly on the work of Alasdair MacIntyre and inspired by John Dewey, I develop two rival conceptions of scientific practice. I show that these rivals are closely linked to the two basic sides in the science and values debate -- the debate over the extent to which ethical and political values may legitimately influence scientific inquiry. Finally, I start to develop an account of justice that is sensitive to these legitimate (...) and illegitimate influences. (shrink)

Evidence-based policy is gaining support in many areas of government and in public affairs more generally. In this paper we outline what evidence—based policy is then discuss its strengths and weaknesses. In particular, we argue that it faces a serious challenge to provide a plausible account of evidence. This account needs to be at least in the spirit of the hierarchy of evidence subscribed to by evidence-based medicine (from which evidence—based policy derives its name and inspiration). Yet evidence-based policy’s hierarchy (...) needs to be tailored to the kinds of evidence relevant and available to the policy arena. The evidence required for policy decisions does not easily lend itself to randomised controlled trials (the "gold standard" in evidence-based medicine), nor, for that matter, being listed in a single all—purpose hierarchy. (shrink)

Robustness is a common platitude: hypotheses are better supported with evidence generated by multiple techniques that rely on different background assumptions. Robustness has been put to numerous epistemic tasks, including the demarcation of artifacts from real entities, countering the “experimenter’s regress,” and resolving evidential discordance. Despite the frequency of appeals to robustness, the notion itself has received scant critique. Arguments based on robustness can give incorrect conclusions. More worrying is that although robustness may be valuable in ideal evidential circumstances (i.e., (...) when evidence is concordant), often when a variety of evidence is available from multiple techniques, the evidence is discordant. †To contact the author, please write to: Jacob Stegenga, Department of Philosophy, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093; e‐mail: [email protected]. (shrink)

Evidence-Based Medicine (EBM) developed from the work of clinical epidemiologists at McMaster University and Oxford University in the 1970s and 1980s and self-consciously presented itself as a "new paradigm" called "evidence-based medicine" in the early 1990s. The techniques of the randomized controlled trial, systematic review and meta-analysis have produced an extensive and powerful body of research. They have also generated a critical literature that raises general concerns about its methods. This paper is a systematic review of the critical literature. It (...) finds the description of EBM as a Kuhnian paradigm helpful and worth taking further. Three kinds of criticism are evaluated in detail: criticisms of procedural aspects of EBM (especially from Cartwright, Worrall and Howick), data showing the greater than expected fallibility of EBM (Ioaanidis and others), and concerns that EBM is incomplete as a philosophy of science (Ashcroft and others). The paper recommends a more instrumental or pragmatic approach to EBM, in which any ranking of evidence is done by reference to the actual, rather than the theoretically expected, reliability of results. Emphasis on EBM has eclipsed other necessary research methods in medicine. With the recent emphasis on translational medicine, we are seeing a restoration of the recognition that clinical research requires an engagement with basic theory (e.g. physiological, genetic, biochemical) and a range of empirical techniques such as bedside observation, laboratory and animal studies. EBM works best when used in this context. (shrink)

Exploratory experiments are widely characterized as experiments that do not test hypotheses. Experiments that do test hypotheses are characterized as confirmatory experiments. Philosophers have pointed out that research programmes can be both confirmatory and exploratory. However, these definitions preclude single experiments being characterized as both exploratory and confirmatory; how can an experiment test and not test a hypothesis? Given the intuition that some experiments are exploratory, some are confirmatory, and some are both, a recharacterization of the relationship between exploratory and (...) confirmatory experimentation is needed. I discuss ‘phase IV’ trials to show what this recharacterization could look like. Phase IV trials can be exploratory and confirmatory insofar as they concurrently test hypotheses and explore for unforeseen phenomena. Even if it is uncontroversial that a single experiment can have multiple aims, the recharacterization of the relationship between exploratory and confirmatory experimentation is still required for these aims to be held together coherently. I offer an alternative characterization of the relationship between exploratory and confirmatory experimentation where the former remains a distinct kind of experimentation but is not characterized as non-hypothesis-testing. (shrink)

Despite wide recognition that speculation is critical for successful science, philosophers have attended little to it. When they have, speculation has been characterized in narrowly epistemic terms: a hypothesis is speculative due to its (lack of) evidential support. These ‘evidence-first’ accounts provide little guidance for what makes speculation productive or egregious, nor how to foster the former while avoiding the latter. I examine how scientists discuss speculation and identify various functions speculations play. On this basis, I develop a ‘function-first’ account (...) of speculation. This analysis grounds a richer discussion of when speculation is egregious and when it is productive, based in both fine-grained analysis of the speculation’s purpose, and what I call the ‘epistemic situation’ scientists face. (shrink)

In the human sciences, experimental research is used to establish causal relationships. However, the extrapolation of these results to the target population can be problematic. To facilitate extrapolation, we propose to use the statistical technique Latent Class Regression Analysis in combination with the analogical reasoning theory for extrapolation. This statistical technique can identify latent classes that differ in the effect of X on Y. In order to extrapolate by means of analogical reasoning, one can characterize the latent classes by a (...) combination of features, and then compare these features to features of the target. (shrink)

Depuis le début des années 2000, les essais randomisés contrôlés ont connu un fulgurant retour sur le devant de la scène académique dans le domaine de l’économie du développement et plus généralement dans celui de l’évaluation des politiques publiques. Alors que ses partisans revendiquent une filiation avec les essais cliniques randomisés en médecine, ces derniers demeurent mal connus par les économistes. Cet article vise à combler ce manque en proposant une comparaison de l’usage et des limites des essais contrôlés randomisés (...) entre la médecine et l’économie. Nous montrons ainsi que si cette méthode est bien souvent placée au sommet de la hiérarchie des preuves en médecine, des voix critiques existent et méritent d’être prises en compte pour penser son application en économie. Nous soulignons également que le traitement du problème de la généralisation des résultats, très bien documenté en médecine, permet de nuancer l’intérêt de la méthode en économie. (shrink)

The context for this paper is evidence-based practice. EBP is about production of desirable change. The evidence should come from randomized controlled trials. To make sense of RCT evidence it must be placed in an argument structure. I compare two different models, Toulmin and Cartwright, and investigate whether the two models can be merged into one. I shall argue that such merging is not feasible.

Psychotherapy research is known for its pursuit of evidence-based treatment. Psychotherapeutic efficacy is assessed by calculation of aggregated differences between pre treatment- and post treatment symptom levels. As this ‘gold standard methodology’ is regarded as ‘procedurally objective’, the efficacy number that results from the procedure is taken as a valid indicator of treatment efficacy. However, I argue that the assumption of procedural objectivity is not justified, as the methodology is build upon a problematic numerical basis. I use an empirical case (...) study to show how measurement problems practically occur in the first step of data collection, i.e. in individual symptom measurement. These problems have been discussed and acknowledged for decades, but still measurement is regarded as the best epistemic means to gain evidence on psychotherapeutic efficacy. Therefore, I show how initial measurement problems are overlooked in the remainder of the methodological procedure, which harms the ‘evidence-base’ of psychotherapeutic EBT. Via this applied analysis, I exhibit concerns that are increasingly raised in the literature in an empirical way, to emphasize the need for a non-idealized consideration of the ‘gold standard methodology’ as a means towards its clinical end. (shrink)

The most conspicuous form of agricultural experiment is the field trial, and within the history of such trials, the arrival of the randomised control trial is considered revolutionary. Originating with R.A. Fisher within British agricultural science in the 1920s and 30s, the RCT has since become one of the most prodigiously used experimental techniques throughout the natural and social sciences. Philosophers of science have already scrutinised the epistemological uniqueness of RCTs, undermining their status as the ‘gold standard’ in experimental design. (...) The present paper introduces a historical case study from the origins of the RCT, uncovering the initially cool reception given to this method by agricultural scientists at the University of Cambridge and the National Institute of Agricultural Botany. Rather than giving further attention to the RCT, the paper focuses instead on a competitor method – the half-drill strip – which both predated the RCT and remained in wide use for at least a decade beyond the latter’s arrival. In telling this history, John Pickstone’s Ways of Knowing is adopted, as the most flexible, and productive way to write the history of science, particularly when sciences and scientists have to work across a number of different kinds of place. It is shown that those who resisted the RCT did so in order to preserve epistemic and social goals that randomisation would have otherwise run a tractor through. (shrink)

This paper looks at an appeal to the authority of biomedical research that has recently been used by empirical economists to motivate and justify their methods. I argue that those who make this appeal mistake the nature of biomedical research. Randomised trials, which are said to have revolutionised biomedical research, are a central methodology, but according to only one paradigm. There is another paradigm at work in biomedical research, the inferentialist paradigm, in which randomised trials play no special role. I (...) outline the inferentialist alternative in broad strokes, apply it to a recent controversy in econometrics and draw some general conclusions concerning econometric methodology. (shrink)

This paper examines the scientific controversy over the yields of genetically modified [GM] crops as a case study in epistemologically deep disagreements. Appeals to “the evidence” are inadequate to resolve such disagreements; not because the interlocutors have radically different metaphysical views (as in cases of incommensurability), but instead because they assume rival epistemological frameworks and so have incompatible views about what kinds of research methods and claims count as evidence. Specifically, I show that, in the yield debate, proponents and opponents (...) of GM crops cite two different sets of claims as evidence, which correspond to two rival epistemological frameworks, classical experimental epistemology and Nancy Cartwright's evidence for use. I go on to argue that, even if both sides of the debate accepted Cartwright's view, they might still disagree over what counts as evidence, because evidence for use ties standards of evidence to what is sometimes called the “context of application.”. (shrink)

This is a response to the response by Robyn Bluhm to my paper, and I am again arguing for a limited role of capacities in psychiatry, given the current scientific uncertainties.

We claim that if a complete philosophy of evidence-based practice is intended, then attention to the nature of causation in health science is necessary. We identify how health science currently conceptualises causation by the way it prioritises some research methods over others. We then show how the current understanding of what causation is serves to constrain scientific progress. An alternative account of causation is offered. This is one of dispositionalism. We claim that by understanding causation from a dispositionalist stance, many (...) of the processes within an evidence-based practice framework are better accounted for. Further, some of the problems associated with the health research, e.g. external validity of causal findings, dissolve. (shrink)