Eyal Shahar’s essay review [1] of James Penston’s remarkable book [2] seems more inspired playful academic provocation than review or essay, expressing dramatic views of impossible validity. The account given of modern biostatistical causation reveals the slide from science into the intellectual confusion and non-science RCTs have created: “…. the purpose of medical research is to estimate the magnitude of the effect of a causal contrast, for example the probability ratio of a binary outcome …” But Shahar’s world is simultaneously (...) not probabilistic, but of absolute uncertainty: “We should have no confidence in any type of evidence ….. We should have no confidence at all”. Shahar’s "Causal contrast" is attractive. It seems to make sense, but bypasses in two words the means of establishing causation by the scientific method. This phrase assumes a numeric statistically significant “contrast” is causal rather than a potential correlation requiring further investigation. The concept of “causal contrast” is a slippery slope from sense into biostatistical non-science. This can be illustrated with an hypothetical RCT where 100% of interventions exhibit a posited treatment effect and 0% of placebo controls. Internal validity is seemingly quite reasonably assumed satisfied (common-sense dictating the likelihood of an awesome magnificent fraud, bias or plain error of the magnitude required is infinitesimal). Scientific method appears satisfied. The RCT demonstrates: (1) strict regularity of outcome in the presence of posited cause; (2) the absence of outcome in its absence and (3) an intervention (experiment) showing the direction of causation is from posited cause to posited effect. Now travel further down the slope from science. Assume 50% of interventions and 0% of controls are positive. We compromise scientific method, but justify this by assuming a large subgroup which we say surely must on these figures be exhibiting the posited treatment effect. But what of 10% of interventions and 9% of placebo controls exhibiting the posited treatment effect? Our biostatistician says the 1% “causal contrast” is statistically significant. But we have: (1) minimal evidence of regularity; (2) the posited outcome irrespective of presence of posited cause and (3) our intervention is at the highest equivocal in demonstrating any form of causation. This is not science. It is, however, where biostatistics has unthinkingly taken us, as Penston has shown comprehensively [2]. We, the audience of published medical research, are now for the 10% / 9% example well down the slope from science. An unattractive hypothesis results requiring numerous assumptions similar to these:- "There is a 'contrast' which is ‘causal’, albeit the method employed is not scientific. An effect of the intervention has been observed in a very small subgroup. This subgroup is susceptible to treatment. The similar number of placebo controls exhibiting the outcome sought is irrelevant, because the 1% difference between intervention and controls is statistically significant. The statistical analysis is valid and reliable. The RCT’s internal validity is sufficiently satisfied. No funding or bias or fraud has affected the results or their analysis.” As Penston notes: “Confirming and refuting the results of research is crucial to science …. But … there’s no way of testing the results of any particular large-scale RCT or epidemiological study. Each study … is left hanging in the air, unsupported.” It gets worse. To identify a rare serious adverse reaction of a frequency of 1:10,000 can require a trial of 200,000 or larger split between controls and interventions. This is not done. But for every 100 who prospectively benefit from the intervention, 9,900 also receive it. And for every 100 benefiting one person (who likely gains no benefit) will suffer a serious unidentified adverse reaction. This is also without taking account of more common adverse reactions whether serious or otherwise. References [1] Shahar, E. (2011) Research and medicine: human conjectures at every turn. International Journal of Person Centered Medicine 1 (2), 250-253. [2] Penston, J. (2010). stats.con: How we’ve been fooled by statistics-based research in medicine. London: The London Press, UK. (shrink)

In this article, I ask whether a principle analogous to the principle of clinical equipoise should govern the design and conduct of RCTs evaluating the effectiveness of policy interventions. I answer this question affirmatively, and introduce and defend the principle of policy equipoise. According to this principle, all arms of a policy RCT must be, at minimum, in a state of equipoise with the best proven policy that is also morally and practically attainable and sustainable. For all arms of a (...) policy RCT, policy experts must either reasonably disagree about whether the trial arms are more effective than this policy, or know that they are. (shrink)

An astonishing volume and diversity of evidence is available for many hypotheses in the biomedical and social sciences. Some of this evidence—usually from randomized controlled trials (RCTs)—is amalgamated by meta-analysis. Despite the ongoing debate regarding whether or not RCTs are the ‘gold-standard’ of evidence, it is usually meta-analysis which is considered the best source of evidence: meta-analysis is thought by many to be the platinum standard of evidence. However, I argue that meta-analysis falls far short of that standard. Different meta-analyses (...) of the same evidence can reach contradictory conclusions. Meta-analysis fails to provide objective grounds for intersubjective assessments of hypotheses because numerous decisions must be made when performing a meta-analysis which allow wide latitude for subjective idiosyncrasies to influence its outcome. I end by suggesting that an older tradition of evidence in medicine—the plurality of reasoning strategies appealed to by the epidemiologist Sir Bradford Hill—is a superior strategy for assessing a large volume and diversity of evidence. (shrink)

Medical scientists employ ‘quality assessment tools’ (QATs) to measure the quality of evidence from clinical studies, especially randomized controlled trials (RCTs). These tools are designed to take into account various methodological details of clinical studies, including randomization, blinding, and other features of studies deemed relevant to minimizing bias and error. There are now dozens available. The various QATs on offer differ widely from each other, and second-order empirical studies show that QATs have low inter-rater reliability and low inter-tool reliability. This (...) is an instance of a more general problem I call the underdetermination of evidential significance. Disagreements about the strength of a particular piece of evidence can be due to different—but in principle equally good—weightings of the fine-grained methodological features which constitute QATs. (shrink)

Governments are increasingly using randomized controlled trials (RCTs) to evaluate policy interventions. RCTs are often understood to provide the highest quality evidence regarding the causal efficacy of an intervention. While randomization plays an essential epistemic role in the context of policy RCTs however, it also plays an important distributive role. By randomly assigning participants to either the intervention or control arm of an RCT, people are subject to different policies and so, often, to different types and levels of benefits. In (...) this paper, I identify one necessary condition as well as a set of sufficient conditions for the permissible use of random assignment by government agencies. I argue first that random assignment is permissible only if it is consistent with governments’ duty to realize morally important outcomes. I argue second that random assignment is permissible in cases where investigators are in a state of genuine equipoise regarding all arms of the experiment and the policy to which people have a claim of justice. Finally, I defend a set of conditions under which random assignment is permissible in cases where one or more arms of a policy RCT are reasonably expected to be either superior or inferior to this policy. (shrink)

The transferability problem—whether the results of an experiment will transfer to a treatment population—affects not only Randomized Controlled Trials but any type of study. The problem for any given type of study can also, potentially, be addressed to some degree through many different types of study. The transferability problem for a given RCT can be investigated further through another RCT, but the variables to use in the further experiment must be discovered. This suggests we could do better on the epistemological (...) problem of transferability by promoting, in the repeated process of formulating public health guidelines, feedback loops of information from the implementation setting back to researchers who are defining new studies. (shrink)

We raise a problem of applicability of RCTs to validate nuclear diagnostic imaging tests. In spite of the wide application of PET and other similar techniques that use radiopharmaceuticals for diagnostic purposes, RCT-based evidence on their validity is sparse. We claim that this is due to a general conceptual problem that we call Prevalence of Treatment, which arises in connection with designing RCTs for testing any diagnostic procedure in the present context of medical research, and is particularly apparent in this (...) case. We also identify three practical reasons why RCTs do not qualify as the best option for PET validation, which have to do with specific characteristics of nuclear diagnostic imaging, and of radiopharmaceuticals. The paper is meant to contribute both to the philosophical discussion on the EBM hierarchy of evidence, and on the specific debate on radiopharmaceuticals in nuclear medicine. (shrink)

I will open the first part of this paper by trying to elucidate the frequentist foundations of RCTs. I will then present a number of methodological objections against the viability of these inferential principles in the conduct of actual clinical trials. In the following section, I will explore the main ethical issues in frequentist trials, namely those related to randomisation and the use of stopping rules. In the final section of the first part, I will analyse why RCTs were accepted (...) for regulatory purposes. I contend that their main virtue, from a regulatory viewpoint, is their impartiality, which is grounded in randomisation and fixed rules for the interpretation of the experiment. Thus the question will be whether Bayesian trials can match or exceed the achievements of frequentist RCTs in all these respects. In the second part of the paper, I will first present a quick glimpse of the introduction of Bayesianism in the field of medical experiments, followed by a summary presentation of the basic tenets of a Bayesian trial. The point here is to show that there is no such thing as “a” Bayesian trial. Bayesianism can ground many different approaches to medical experiments and we should assess their respective virtues separately. Thus I present two actual trials, planned with different goals in mind, and assess their respective epistemic, ethical and regulatory merits. In a tentative conclusion, I contend that, given the constraints imposed by our current regulatory framework, impartiality should preside over the design of clinical trials, even at the expense of many of their inferential and ethical virtues. (shrink)

Introduction: Clinical practice guidelines (CPGs) are an important source of justification for clinical decisions in modern evidence-based practice. Yet, we have given little attention to how they argue their evidence. In particular, how do CPGs argue for treatment with long-term medications that are increasingly prescribed to older patients? Approach and rationale: I selected six disease-specific guidelines recommending treatment with five of the medication classes most commonly prescribed for seniors in Ontario, Canada. I considered the stated aims of these CPGs and (...) the techniques employed towards those aims. Finally, I reconstructed and logically analysed the arguments supporting recommendations for pharmacotherapy. Analysis: The primary function of CPGs is rhetorical, or persuasive, and their means of persuasion include both a display of their credibility and their argumentation. Arguments supporting pharmacotherapy recommendations for the target population follow a common inductive pattern: statistical generalization from randomized controlled trial (RCT) and meta-analysis evidence. Two of the CPGs also argue their treatment recommendations for older patients in this style, while three fail to justify pharmacotherapy specifically for the older population. Discussion: The arguments analysed lack the auxiliary assumptions that would warrant making a generalization about the clinical effectiveness of medications for the older population. Guidelines reason using simple induction, while ignoring important inferential gaps. Future guidelines should aspire to be well-reasoned rather than simply evidence-based; argue from a plurality of evidence; be wary of hasty inductions; appropriately limit the scope of their recommendations; and avoid making law-like, prescriptive generalizations. (shrink)

Heterogeneous treatment effects represent a major issue for medicine as they undermine reliable inference and clinical decision-making. To overcome the issue, the current vision of precision and personalized medicine acknowledges the need to control individual variability in response to treatment. In this paper, we argue that gene-treatment-environment interactions (G × T × E) undermine inferences about individual treatment effects from the results of both genomics-based methodologies—such as genome-wide association studies (GWAS) and genome-wide interaction studies (GWIS)—and randomized controlled trials (RCTs). Then, (...) we argue that N-of-1 trials can be a solution to overcome difficulties in handling individual variability in treatment response. Although this type of trial has been suggested as a promising strategy to assess individual treatment effects, it nonetheless has limitations that limit its use in everyday clinical practice. We analyze the existing variability within the designs of N-of-1 trials in terms of a continuum where each design prioritizes epistemic and pragmatic considerations. We then support wider use of the designs located at the pragmatic end of the explanatory-pragmatic continuum. (shrink)

Over the last decade, philosophers of science have extensively criticized the epistemic superiority of randomized controlled trials for testing safety and effectiveness of new drugs, defending instead various forms of evidential pluralism. We argue that scientific methods in regulatory decision-making cannot be assessed in epistemic terms only: there are costs involved. Drawing on the legal distinction between rules and standards, we show that drug regulation based on evidential pluralism has much higher costs than our current RCT-based system. We analyze these (...) costs and advocate for evaluating any scheme for drug regulatory tests in terms of concrete empirical benchmarks, like the error rates of regulatory decisions. (shrink)

The volume of clinical research is increasing exponentially—far beyond our ability to process and absorb the results. Given this situation, it may be beneficial to consider reducing the flow at its source. In what follows, I will motivate and critically evaluate the following proposal: researchers should conduct fewer clinical trials. More specifically, I c onsider whether researchers should be permitted to conduct only clinical research of very high quality and, in turn, whether research ethics committees should prohibit all other, lower-quality (...) research, even when it might appear to meet some minimal ethical standard. Following a close analysis of the social-value requirement of ethical clinical research, I argue that this proposal is defensible. The problem identified in this paper has two parts, quantity and quality, and some clarification is needed about the latter because “quality” is a highly contested term in the medical literature. When some scholars advocate for high-quality trials, they mean large-scale, simple, explanatory randomized controlled trials. Others, including myself, have defended a different characterization of high-quality research that tends more toward pragmatic trial design and the use of methods other than RCTs. Pragmatic trials aim to provide evidence that directly supports clinical decision-making in “usual” care settings. Unlike explanatory trials, which aim to abstract away from particular settings and patients, in the hopes of creating ideal conditions for the success of an intervention, pragmatic trials deliberately pursue knowledge of high applicability, through the use of representative subjects, clinically important questions, flexible treatment protocols, patient-oriented outcome measures, and so on. I see applicability as a marker of high-quality research. The context in which research is meant to be applied should be the context in which new interventions are evaluated. (shrink)

Health anxiety is, among other things, a response to a universal epistemological problem about whether changes in one’s body indicate serious illness, a problem that grows more challenging to the individual with age and with every advance in medical science, detection, and treatment. There is growing evidence that dysfunctional metacognitive beliefs – beliefs about thinking – are the driving factor, with dysfunctional substantive beliefs about the probability of illness a side‐effect, and that Metacognitive Therapy (MCT) is more effective than Cognitive (...) Behavioral Therapy (CBT). However, hypochondria is distinct from other forms of anxiety, I argue, in ways that make some reality‐checking techniques of CBT and MCT of limited usefulness. I propose a Re‐Calibration Technique (RCT) that complements these therapies by focusing on a metacognitive belief that has not been studied: the patient’s presumption of his own personal reliability in judging symptoms, an assumption exposed every time he disagrees with a doctor. I propose a technique whereby a patient keeps a long‐term register of every episode of alarm about symptoms and its resolution, possibly years later. When healthcare‐seeking impulses arise the patient then uses his own track record to re‐calibrate his confidence that medical attention is needed. The new technique allows one to improve self‐judgment about whether one has an illness or not by improving self‐knowledge of one’s own reliability. (shrink)

According to the norm of policy equipoise, it is permissible to randomly assign participants to two or more interventions in a public policy randomized controlled trial (RCT) when there is meaningful uncertainty among the relevant expert community regarding which intervention is superior. While this norm is gaining traction in the research ethics literature, the idea of interventional superiority remains unclear. Is one intervention superior to another if it is reasonably expected to realize one outcome of interest more effectively, even though (...) there is uncertainty regarding other outcomes of interest? Or must an intervention be reasonably expected to realize all outcomes of interest more effectively? I address this question in this paper. My aim is to develop and defend an account of interventional superiority for policy RCTs that are authorized, funded, or conducted by government institutions. I defend the greatest value view, according to which one intervention is superior to another if and only if it is reasonably expected to more effectively realize a set of outcomes with greater value. (shrink)

A central feature of experimental development economics is the use of randomized controlled trials (RCTs) to evaluate the effectiveness of prospective socioeconomic interventions. The use of RCTs in development economics raises a host of ethical issues which are just beginning to be explored. In this article, I address one ethical issue in particular: the routine use of the status quo as a control when designing and conducting a development RCT. Drawing on the literature on the principle of standard care in (...) medical research ethics, as well as considerations of distributive justice in non-ideal circumstances, I argue that the practice of using the status quo as a control is ethically justifiable. However, I add an important qualification based on the natural duty of rescue to address the concern that my account is overly permissive. (shrink)

Evidence based-policy (EBP) is a popular research paradigm in the applied social sciences and within government agencies. Informally, EBP represents an explicit commitment to applying scientific methods to public affairs, in contrast to ideologically-driven or merely intuitive “common-sense” approaches to public policy. More specifically, the EBP paradigm places great weight on the results of experimental research designs, especially randomised controlled trials (RCTs), and systematic literature reviews that place evidential weight on experimental results. One hope is that such research designs and (...) approaches to analysing the scientific literature are sufficiently robust that they can settle what really ‘works’ in public policy. Can EBP succeed in displacing reliance on domain-specific expertise? On our account, this is seldom, if ever, the case. The key reason for this is that underlying this approach is generally an appeal to argument by induction, which always requires further assumptions to underwrite its validity, and if not induction, some other argument form that also requires assumptions that are very often not validated for the case at hand. (shrink)

Mill (1872, 1874) is an early proponent of the thesis that economics has a special domain in which it can operate relatively independently of findings from other sciences. Contra Mill, I argue that this so-called separateness-thesis is best defendedunder an externalist interpretation of Rational Choice Theory (RCT). Mill’s defence is consistent with an internalist interpretation of RCT. Internalism holds that RCT depicts psychological mechanisms operating in economic agents. I argue that such a defence fails to establish separateness, because it makes (...) economics highly depended on psychological findings. However, externalism understands RCT as an adequate description of how agents react to incentives if certain environmental structures are present. Under this interpretation, we can defend separateness by investigating which features of an environmental structure will lead agents to behave in accordance with RCT. Thereby, we can derive indicators that enable us to demarcate a separate domain for economics. (shrink)

Government agencies and nonprofit organizations have increasingly turned to randomized controlled trials (RCTs) to evaluate public policy interventions. Random assignment is widely understood to be fair when there is equipoise; however, some scholars and practitioners argue that random assignment is also permissible when an intervention is reasonably expected to be superior to other trial arms. For example, some argue that random assignment to such an intervention is fair when the intervention is scarce, for it is sometimes fair to use a (...) lottery to allocate scarce goods. We investigate the permissibility of randomization in public policy RCTs when there is no equipoise, identifying two sets of conditions under which it is fair to allocate access to a superior intervention via random assignment. We also reject oft-made claims that alternative study designs, including stepped-wedge designs and uneven randomization, offer fair ways to allocate beneficial interventions. (shrink)

En economía, la investigación se divide en dos grandes metodologías: los modelos teórico-matemáticos y los estudios empíricos. Estudiando modelos teóricos y métodos empíricos ) se da cuenta de las limitaciones de ambos métodos. Se concluye que ninguno de estos puede generar explicaciones de cómo en realidad suceden las cosas, sino que solo de cómo posiblemente suceden. La razón es que ambos necesitan un enlace interpretativo que permita extrapolar desde su propio sistema hacia un sistema objetivo. Los modelos tienen dominio general (...) y pueden dar cuenta de mecanismos. Los RCTs, al contrario, son válidos internamente y están conectados al mundo real, pero su dominio es muy específico. Aunque ninguno logre responder preguntas amplias de un fenómeno de interés, pueden complementarse para generar extrapolaciones más confiables sobre un sistema objetivo. Sin embargo, esto solo podrá hacerse si se conocen bien los mecanismos y el contexto en que ocurre una evidencia. Palabras clave: Causalidad, Mecanismos, Capacidades, Epistemología de la Economía, Generalización de Evidencia. (shrink)

Recently the role of ideology and hegemony has received increased attention to explain varying dynamics of diffusion and autocratic cooperation. As a result, patterns of interaction in clusters from regions without hegemony or ideology have been overlooked because their autocracy-toautocracy transitions are no threat to the global status of democracy, even when active regime promotion is very common. This article will apply insights from economic cluster theory to political regimes and introduce a typology to differentiate among clusters. Regime Cluster Theory (...) is the first framework that presents three ideal-types of ideological, hegemonic and biotopical regime clusters. With a new concept of ‘biotopical clusters’ the paper explains the dynamics of clusters in often omitted regions, like in Sub Saharan Africa, Latin America during the Cold War, or Central Asia during the 1990s. RCT offers a dynamic approach to recognize and assess patterns of forcible regime promotion per cluster as well as distinguish between their different diffusion patterns (coercive, voluntary, bounded learning, contagion) in four arenas: institutions, ideas, policy and administrative practices. RCT advances the comparative study of regime promotion and diffusion in various regions of the world and hopes to shed new light on related theories of alliance formation, regional institutionalization, and (conflict) spill-over effects. (shrink)

Objectives. The purpose of this study was to conduct the first randomized controlled trial (RCT) to evaluate the effectiveness of a second-generation mindfulness-based intervention (SG-MBI) for treating fibromyalgia syndrome (FMS). Compared to first generation mindfulness-based interventions, SG-MBIs are more acknowledging of the spiritual aspect of mindfulness. Design. A RCT employing intent-to-treat analysis. Methods. Adults with FMS received an 8-week SG-MBI known as meditation awareness training (MAT; n = 74) or an active control intervention known as cognitive behaviour theory for groups (...) (n = 74). Assessments were performed at pre-, post-, and 6-month follow-up phases. Results. Meditation awareness training participants demonstrated significant and sustained improvements over control group participants in FMS symptomatology, pain perception, sleep quality, psychological distress, non-attachment (to self, symptoms, and environment), and civic engagement. A mediation analysis found that (1) civic engagement partially mediated treatment effects for all outcome variables, (2) non-attachment partially mediated treatment effects for psychological distress and sleep quality, and (3) non-attachment almost fully mediated treatment effects for FMS symptomatology and pain perception. Average daily time spent in meditation was found to be a significant predictor of changes in all outcome variables. Conclusions. Meditation awareness training may be a suitable treatment for adults with FMS and appears to ameliorate FMS symptomatology and pain perception by reducing attachment to self. (shrink)

Introduction: Liposomal bupivacaine (LB) is a long-acting analgesic that, due to its liposomal formulation, purportedly extends its analgesic effect up to 72 hours. However, the clinical efficacy of LB appears mixed. This systematic review seeks to evaluate the effectiveness of liposomal bupivacaine in improving postoperative outcomes compared to ropivacaine (ROPI), another commonly used long-acting analgesic. -/- Materials and Methods: Prospective and randomized controlled trials (RCTs) evaluating the efficacy of LB compared to ROPI were selected for review. Primary outcomes included hospital (...) length of stay (LOS) and postoperative opioid consumption measured in oral morphine equivalents (OME). Secondary outcomes included analgesic cost. -/- Results: 14 studies met the review criteria. We found that LB and ROPI are equivalent in managing postoperative pain. 8 of the 14 trials reported equal efficacy between LB and ROPI as determined by OME post-procedure and 10 of the 14 trials reported similar LOS after surgery. These findings remained consistent across multiple surgical procedures and multiple drug administrative modalities. -/- Conclusion: Our systematic review found that LB was not superior to ROPI in reducing postoperative OME use and hospital LOS. The only consistent finding was the significantly increased cost of LB compared to that of ROPI. Therefore, the use of LB over ROPI cannot be justified. (shrink)

Interpersonal comparisons of well-being (ICWs) confront the longstanding unsolved epistemic problem of other minds (EPOM): the problem of how to achieve objective knowledge of people's subjective mental states. The intractability of the EPOM may lead to the hope that Rational Choice Theory (RCT) can show that information about how people would choose over goods and gambles is sufficient--and information about subjective mental states therefore unnecessary--for interpersonal comparisons of levels and changes in well-being, thereby bypassing the EPOM. I argue that this (...) hope cannot be fulfilled. Our most plausible theories of value--whether anti-realist or realist--and theories of what makes a life go best--whether preference hedonism, success theory, or objective list theory--tie well-being to our evaluative attitudes towards our lives. These are distinct from and only contingently related to motivational attitudes to choose or behave in certain ways and therefore to choices and behaviors themselves. Interpersonal comparisons of the evaluative attitudes are therefore necessary, though perhaps insufficient, for ICWs. Preference theory's zero-one rule ignores these attitudes and is therefore implausible. Its extended preference approach assumes that our preferences are perfectly sympathetic and therefore begs the question of the EPOM. I argue that a principled solution to the EPOM, and to interpersonal comparisons of the evaluative attitudes, is provided by type correspondence between these attitudes and brain states. It remains an open and difficult question whether there exists a summary evaluative attitude whose intensity can serve as an index of an individual's over-all well-being, and which is the appropriate target of all efforts aimed at promoting the personal good, or whether the self and therefore well-being are too fragmented for this. (shrink)

Evidence hierarchies are widely used to assess evidence in systematic reviews of medical studies. I give several arguments against the use of evidence hierarchies. The problems with evidence hierarchies are numerous, and include methodological shortcomings, philosophical problems, and formal constraints. I argue that medical science should not employ evidence hierarchies, including even the latest and most-sophisticated of such hierarchies.

As a way to make medical decisions, Evidence-Based Medicine (EBM) has failed. EBM's failure arises from not being founded on real-world decision-making. EBM aspires to a scientific standard for the best way to treat a disease and determine its cause, but it fails to recognise that the scientific method is inapplicable to medical and other real-world decision-making. EBM also wrongly assumes that evidence can be marshaled and applied according to an hierarchy that is determined in an argument by authority to (...) the method by which it has been obtained. If EBM had valid theoretical, practical or empirical foundations, there would be no hierarchy of evidence. In all real-world decision-making, evidence stands or falls on its inherent reliability. This has to be and can only be assessed on a case-by-case basis applying understanding and wisdom against the background of all available facts—the "factual matrix." EBM's failure is structural and was inevitable from its inception. EBM confuses the inherent reliability and probative value of evidence with the means by which it is obtained. -/- EBM is therefore an ad hoc construct and is not a valid basis for medical decision-making. This is further demonstrated by its exclusion of relevant scientific and probative real-world decision-making evidence and processes. It draws upon a narrow evidence base that is itself inherently unreliable. It fails to take adequate account of the nature of causation, the full range of evidence relevant to its determination, and differing approaches to determining cause and effect in real-world decision-making. EBM also makes a muddled attempt to emulate the scientific method and it does not acknowledge the role of experience, understanding and wisdom in making medical decisions. (shrink)

Public policymakers and institutional decision-makers routinely face questions about whether interventions “work”: does universal basic income improve people’s welfare and stimulate entrepreneurial activity? Do gated alleyways reduce burglaries or merely shift the crime burden to neighbouring communities? What is the most cost-effective way to improve students’ reading abilities? These are empirical questions that seem best answered by looking at the world, rather than trusting speculations about what will be effective. Evidence-based policy (EBP) is a movement that concretizes this intuition. It (...) maintains that policy should be based on evidence of “what works.” Not any evidence will do, however. Following on the heels of its intellectual progenitor, evidence-based medicine (EBM), EBP insists on the use of high-quality evidence produced in accordance with rigorous methodological standards. Though intuitively compelling, EBP has attracted significant criticism from methodologists, political scientists, and philosophers. This chapter provides a critical overview of EBP through a philosophical lens, reviewing and discussing some of the most pressing challenges that EBP faces and outlining some proposals for improving it. I first provide a brief overview of EBP and distinguish a broader and narrower understanding of it. I then review existing lines of criticism of EBP, two of which are considered in detail. The first elaborates how EBP struggles with extrapolation, that is, the use of evidence from study populations to make inferences about the effects of policies on novel target populations. The second maintains that EBP’s methodological tenets are deeply entwined with moral and political values, which can threaten EBP’s promise to promote objectivity in policymaking. Finally, I make some proposals for how EBP could be improved going forward. (shrink)