There are some philosophical questions that can be answered without attention to the social context in which evidence is produced and distributed.ing away from social context is an excellent way to ignore messy details and lay bare the underlying structure of the limits of inference. Idealization is entirely appropriate when one is essentially asking: In the best of all possible worlds, what am I entitled to infer? Yet, philosophers’ concerns often go beyond this domain. As an example I examine the (...) debate on mechanistic evidence and then reevaluate a canonical case study in this debate. I show that for the assessment of actual evidence, produced in a world that is far from ideal, omission of the social aspects of medical epistemology leads philosophers to draw the wrong lessons from cases they take as paradigmatic cases for their views. I close by arguing that social epistemology provides an avenue to incorporate these complications and provides the necessary framework to understand medical evidence. (shrink)

Francesco Guala once wrote that ‘The problem of extrapolation is a minor scandal in the philosophy of science’. This paper agrees with the statement, but for reasons different from Guala’s. The scandal is not, or not any longer, that the problem has been ignored in the philosophy of science. The scandal is that framing the problem as one of external validity encourages poor evidential reasoning. The aim of this paper is to propose an alternative—an alternative which constitutes much better evidential (...) reasoning about target systems of interest, and which makes do without consideration of external validity. (shrink)

Pragmatic comparative effectiveness randomized controlled trials evaluate the effectiveness of one interventions under real-world clinical conditions. The results of ceRCTs are often directly generalizable to everyday clinical practice, providing information critical to decision-making by patients, clinicians, and healthcare policymakers. The PRECIS-2 framework identifies nine domains that serve to score a trial on a continuum between very explanatory to very pragmatic. According to the framework, pragmatic trials may have one or more of the following features: there are fewer eligibility criteria for (...) participants, in an... (shrink)

Medical nihilism is the view that we should have little confidence in the effectiveness of medical interventions. Jacob Stegenga argues persuasively that this is how we should see modern medicine, and suggests that medical research must be modified, clinical practice should be less aggressive, and regulatory standards should be enhanced.

Clinical trials and other forms of evaluation of medical treatment are held to give an objective assessment of the ‘clinical effectiveness’ of the medical treatments under evaluation. This kind of evaluation is central to the evidence-based medicine movement, as it provides a basis for the rational selection of treatment. The ethical status of randomised clinical trials is widely agreed to depend crucially upon the state of equipoise regarding which of two (or more) treatments is more (or most) effective in a (...) defined population. However, the meaning and nature of ‘clinical effectiveness’ are unclear. in this paper, I discuss the proposals to define clinical effectiveness as a relational property and as an intrinsic property, and the way effectiveness may supervene upon more fundamental physical properties of treatments. I discuss whether effectiveness is a single property or a family of properties; the types of outcome which can be explained by effectiveness properties; and the relationship between ‘objective’ and ‘preference’ outcomes. This paper suggests that while it may be possible to put clinical effectiveness on a proper metaphysical footing, in practice the language of clinical effectiveness is more properly a topic of the human sciences than of the natural sciences. (shrink)

Measuring the effectiveness of medical interventions faces three epistemological challenges: the choice of good measuring instruments, the use of appropriate analytic measures, and the use of a reliable method of extrapolating measures from an experimental context to a more general context. In practice each of these challenges contributes to overestimating the effectiveness of medical interventions. These challenges suggest the need for corrective normative principles. The instruments employed in clinical research should measure patient-relevant and disease-specific parameters, and should not be sensitive (...) to parameters that are only indirectly relevant. Effectiveness always should be measured and reported in absolute terms (using measures such as 'absolute risk reduction'), and only sometimes should effectiveness also be measured and reported in relative terms (using measures such as 'relative risk reduction')-employment of relative measures promotes an informal fallacy akin to the base-rate fallacy, which can be exploited to exaggerate claims of effectiveness. Finally, extrapolating from research settings to clinical settings should more rigorously take into account possible ways in which the intervention in question can fail to be effective in a target population. (shrink)

Two approaches to evidential reasoning compete in the biomedical and social sciences: the experimental and the pragmatist. Whereas experimentalism has received considerable philosophical analysis and support since the times of Bacon and Mill, pragmatism about evidence has been neither articulated nor defended. The overall aim is to fill this gap and develop a theory that articulates the latter. The main ideas of the theory will be illustrated and supported by a case study on the smoking/lung cancer controversy in the 1950s.

Over the past several decades, we devoted much energy to generating, reviewing and summarizing evidence. We have given far less attention to the issue of how to thoughtfully apply the evidence once we have it. That’s fine if all we care about is that our clinical decisions are evidence-based, but not so good if we also want them to be well-reasoned. Let us not forget that evidence based medicine (EBM) grew out of an interest in making medicine ‘rational’, with the (...) idea that rational clinical evaluations should be evidence-based. I agree with the uncontroversial statement that the best decision is supported, at least in part, by the best available evidence. Rationality, however, is constituted by reasoning, not evidence. Complete arguments are necessary for rational evaluations, arguments that begin with general evidence and end in a conclusion about a particular patient. In order to traverse these inferential gaps, medicine must address the issue of how to establish, as an intermediate premise, what the evidence has to say about the efficacy of an intervention for particular patients in a particular practice setting. (shrink)

Most phase III clinical trials today are explanatory. Because explanatory, or efficacy, trials test hypotheses under “ideal” conditions, they are not well suited to providing guidance on decisions made in most clinical care contexts. Pragmatic trials, which test hypotheses under “usual” conditions, are often better suited to this task. Yet, pragmatic, or effectiveness, trials are infrequently carried out. This mismatch between the design of clinical trials and the needs of health care professionals is frustrating for everyone involved, and explains some (...) of the challenges inherent in attempts to enhance knowledge translation and encourage evidence-based practice. The situation is more than simply frustrating, however; it is potentially unethical. Clinical trials must be socially valuable in order to (1) warrant the risks they impose on human research subjects and (2) fairly and efficiently assess new clinical interventions. Most bioethicists would agree that trials that have no social value, for instance, because their results do not have the potential to advance clinical care, should not be performed. What is less widely appreciated is that given limited research resources, trials that are more socially valuable should be preferred to trials that are less socially valuable when all else is equal. With respect to clinical trial design, I argue that while explanatory trials often have some social value, many have less social value than their pragmatic counterparts. On the basis of this general ethical assessment, I provide a preliminary defense of the position that clinical researchers should aim to conduct pragmatic trials, that is, that researchers face a burden of justification related to any idealizing elements added to trial designs. (shrink)

WE AIM HERE to outline a theory of evidence for use. More specifically we lay foundations for a guide for the use of evidence in predicting policy effectiveness in situ, a more comprehensive guide than current standard offerings, such as the Maryland rules in criminology, the weight of evidence scheme of the International Agency for Research on Cancer (IARC), or the US ‘What Works Clearinghouse’. The guide itself is meant to be well-grounded but at the same time to give practicable (...) advice, that is, advice that can be used by policy-makers not expert in the natural and social sciences, assuming they are well-intentioned and have a reasonable but limited amount of time and resources available for searching out evidence and deliberating. (shrink)

What kinds of evidence reliably support predictions of effectiveness for health and social care interventions? There is increasing reliance, not only for health care policy and practice but also for more general social and economic policy deliberation, on evidence that comes from studies whose basic logic is that of JS Mill's method of difference. These include randomized controlled trials, case–control studies, cohort studies, and some uses of causal Bayes nets and counterfactual-licensing models like ones commonly developed in econometrics. The topic (...) of this paper is the 'external validity' of causal conclusions from these kinds of studies. We shall argue two claims. Claim, negative: external validity is the wrong idea; claim, positive: 'capacities' are almost always the right idea, if there is a right idea to be had. If we are right about these claims, it makes big problems for policy decisions. Many advice guides for grading policy predictions give top grades to a proposed policy if it has two good Mill's-method-of difference studies that support it. But if capacities are to serve as the conduit for support from a method-of-difference study to an effectiveness prediction, much more evidence, and much different in kind, is required. We will illustrate the complexities involved with the case of multisystemic therapy, an internationally adopted intervention to try to diminish antisocial behaviour in young people. (shrink)

Randomized controlled trials (RCTs) are widely taken as the gold standard for establishing causal conclusions. Ideally conducted they ensure that the treatment ‘causes’ the outcome—in the experiment. But where else? This is the venerable question of external validity. I point out that the question comes in two importantly different forms: Is the specific causal conclusion warranted by the experiment true in a target situation? What will be the result of implementing the treatment there? This paper explains how the probabilistic theory (...) of causality implies that RCTs can establish causal conclusions and thereby provides an account of what exactly that causal conclusion is. Clarifying the exact form of the conclusion shows just what is necessary for it to hold in a new setting and also how much more is needed to see what the actual outcome would be there were the treatment implemented. (shrink)

Sections include: experiments and generalised causal inference; statistical conclusion validity and internal validity; construct validity and external validity; quasi-experimental designs that either lack a control group or lack pretest observations on the outcome; quasi-experimental designs that use both control groups and pretests; quasi-experiments: interrupted time-series designs; regresssion discontinuity designs; randomised experiments: rationale, designs, and conditions conducive to doing them; practical problems 1: ethics, participation recruitment and random assignment; practical problems 2: treatment implementation and attrition; generalised causal inference: a grounded theory; (...) generalised causal inference: methods for single studies; generalised causal inference: methods for multiple studies; a critical assessment of our assumptions. (shrink)

The principle of clinical equipoise requires that, aside from certain exceptional cases, second generation treatments ought to be tested against standard therapy. In violation of this principle, placebo-controlled trials (PCTs) continue to be used extensively in the development and licensure of second-generation treatments. This practice is typically justified by appeal to methodological arguments that purport to demonstrate that active-controlled trials (ACTs) are methodologically flawed. Foremost among these arguments is the so called assay sensitivity argument. In this paper, I take a (...) closer look at this argument. Following Duhem, I argue that all trials, placebo-controlled or not, rely on external information for their meaningful interpretation. Pending non-circular empirical evidence that we can trust the findings of PCTs to a greater degree than the findings of ACTs, I conclude that the assay sensitivity argument fails to demonstrate that placebo-controlled trials are preferable, methodologically or otherwise, to active-controlled trials. Contrary to the intentions of its authors, the fundamental lesson taught by the assay sensitivity argument is Duhemian: the validity of all clinical trials depends on external information. (shrink)

Much of the methodological discussion around experiments in economics and other social sciences is framed in terms of the notions of internal and external validity. The standard view is that internal validity and external validity stand in a relationship best described as a _trade-off_. However, it is also commonly held that internal validity is a _prerequisite_ to external validity. This article addresses the problem of the compatibility of these two ideas and analyzes critically the standard arguments about the conditions under (...) which a trade-off between internal and external validity arises. Our argument stands against common associations of internal validity and external validity with the distinction between field and laboratory experiments and assesses critically the arguments that link the artificiality of experimental settings done in the laboratory with the purported trade-off between internal and external validity. We conclude that the idea of a trade-off or tension between internal and external validity seems, upon analysis, far less cogent than its intuitive attractiveness may lead us to think at first sight. (shrink)

Randomized controlled trials test new drugs using various debiasing devices to prevent participants from manipulating the trials. But participants often dislike controls, arguing that they impose a paternalist constraint on their legitimate preferences. The 21st Century Cures Act, passed by US Congress in 2016, encourages the Food and Drug Administration to use alternative testing methods, incorporating participants’ preferences, for regulatory purposes. We discuss, from a historical perspective, the trade-off between trial impartiality and participants’ freedom. We argue that the only way (...) out is considering which methods improve upon the performance of conventional trials in keeping dangerous or inefficacious compounds out of pharmaceutical markets. (shrink)

Simple extrapolation is the orthodox approach to extrapolating from clinical trials in evidence-based medicine: extrapolate the relative effect size from the trial unless there is a compelling reason not to do so. I argue that this method relies on a myth and a fallacy. The myth of simple extrapolation is the idea that the relative risk is a ‘golden ratio’ that is usually transportable due to some special mathematical or theoretical property. The fallacy of simple extrapolation is an unjustified argument (...) from ignorance: we conclude that the relative effect size is transportable in the absence of evidence to the contrary. In short, simple extrapolation is a deeply problematic solution to the problem of extrapolation. (shrink)

The concept of clinical equipoise restricts the use of placebo controls in clinical trials when there already exists a proven effective treatment. Several critics of clinical equipoise have put forward alleged counter-examples to this restriction—describing instances of ethical placebo-controlled trials that apparently violate clinical equipoise. In this essay, we respond to these examples and show that clinical equipoise is not as restrictive of placebos as these authors assume. We argue that a subtler appreciation for clinical equipoise—in particular the distinction between (...) de facto and de jure interpretations of the concept—allows the concept to explain when and why a placebo control may be necessary to answer a question of clinical importance. (shrink)

There is a takeover movement fast gaining influence in development economics, a movement that demands that predictions about development outcomes be based on randomized controlled trials. The problem it takes up—of using evidence of efficacy from good studies to predict whether a policy will be effective if we implement it—is a general one, and affects us all. My discussion is the result of a long struggle to develop the right concepts to deal with the problem of warranting effectiveness predictions. Whether (...) I have it right or not, these are questions of vast social importance that philosophers of science can, and should, help answer. (shrink)

The use of evidence in medicine is something we should continuously seek to improve. This book seeks to develop our understanding of evidence of mechanism in evaluating evidence in medicine, public health, and social care; and also offers tools to help implement improved assessment of evidence of mechanism in practice. In this way, the book offers a bridge between more theoretical and conceptual insights and worries about evidence of mechanism and practical means to fit the results into evidence assessment procedures.

In February 2010, the World Medical Association hosted an international symposium on the ethics of placebo controls in clinical trials (WMA 2010). Despite years of debate, ethicists, clinical trialists, and policy makers remain divided over the ethical acceptability of using placebos in research when a proven, effective treatment is available. The protracted nature of this problem is due, at least in part, to a perceived conflict between the opposing demands placed on clinical research by science and ethics. A good, scientifically (...) valid trial, it is argued, must be “assay sensitive,” and without using a placebo control, there can be no guarantee that it is. In this article, we revisit some of the claims made about .. (shrink)

Historically, "Healing Methods" (HMS) have not been based on rational theories. Of the thousands of HMs that have arisen over the ages, only a small number survive today, drawing their power and longevity mostly from their superior ability to act as a placebo within the context of modern-day culture, rather than through any other mode of action.When it comes to HMs, Western scientific culture has not yet evolved beyond a pre-scientific stage (Fancher 1995). A scientific analysis of the part played (...) by practitioner/subject empathy and other emotional influences within the healing situation is needed. Furthermore, the question "Does 'it'work?" should be reformulated as "Is 'it' an effective placebo?"Once one accepts .. (shrink)

A resilient issue in research ethics is whether and when a placebo-controlled trial is justified if it deprives research subjects of a recognized treatment. The clinicians' moral duty to provide the best available care seems to require the use of ‘active’ controlled trials that use an established treatment as a control whenever such a therapy is available. In another regard, ACTs are supposedly methodologically inferior to PCTs. Hence, the moral duty of the clinical researcher to use the best methods will (...) favor PCTs. In this target article, I analyze the three reasons for believing that ACTs are inferior to PCTs namely: 1) ACTs lack ‘assay sensitivity’; 2) ACTs do not measure absolute effect size; and 3) ACTs require more participants; and I contend that none are acceptable. Consequently the tension between clinical and research ethics dissolves: the moral duty of the clinician to avoid PCTs is unopposed by methodological considerations. (shrink)

John Worrall and Nancy Cartwright have both argued that randomized controlled trials are “testing the wrong theory.” They claim that RCTs are designed to test inferences about the causal relationships in the study population, but this does not guarantee a justified inference about the causal relationships in the more diverse population in clinical practice. In this article I argue that the epistemology of theory testing in trials is more complicated than either Worrall’s or Cartwright’s accounts suggest. I illustrate this more (...) complex theoretical structure with case studies in medical theory testing from Alzheimer’s research and anticancer drugs in personalized medicine. (shrink)

Arguments in favor of greater research-practice integration in medicine have tended to be ethical, political, or pragmatic. There are good epistemic reasons to pursue greater integration, and it is important to think through these reasons in order to avoid inadvertently designing new systems in ways that replicate the epistemic elitism common within current systems. Meaningful transformation within health care is possible with close attention to all reasons in favor of greater research-practice integration, including epistemic reasons.

In this article we explore the role evidence ought to play in complementary and alternative medicine. First, we consider the claim that evidence in the form of randomized controlled trials cannot be obtained for CAMs. Second, we consider various claims to the effect that there are ways of obtaining evidence that do not make use of RCTs. We argue that there is no good reason why CAM should be exempted from the general requirement that treatments undergo evaluation by RCT. Third, (...) we consider two implications for health care policy. First, many activities in conventional medicine have never been rigorously evaluated and are widely in use nonetheless. We argue that this fails to provide a reason for exempting CAM from a demand for evidence. Second, CAM use may be compared to a choice of lifestyle, and this has a significant impact on which requirements of evidence can reasonably be imposed. (shrink)

Pragmatic trials aim to directly inform health care decision-making through the collection of so-called ‘real world data’ from observations of comparative treatment effects in clinical practice. In order to ensure the applicability and feasibility of a pragmatic trial, design features may be necessary that deviate from standard research ethics requirements. Examples are traditional requirements to seek written informed consent and to perform extensive data and safety monitoring. Proposals for deviations from standard research ethics practice have resulted in controversy about their (...) ethical acceptability. One of the justifications for altered procedures is the allegedly high social value of pragmatic trials. In order to properly operationalize the concept in the ethical assessment of pragmatic trial designs, specification is warranted. We identified three determinants from common claims about a pragmatic trial's social value: the extent to which the research question has real world relevance, the trial design's ability to generate a real world answer and the probability of direct uptake of the results by decision-makers in practice. Subsequently, we discuss how these determinants should be applied to the practice of pragmatic trials, and to what extent they might be applicable to explanatory trials. (shrink)

I first propose to distinguish between two kinds of external validity inferences, predictive and explanatory. I then argue that we have a satisfactory answer to the question of the conditions under which predictive external validity inferences are good. If this claim is correct, then it has two immediate consequences: First, some external validity inferences are deductive, contrary to what is commonly assumed. Second, Steel’s requirement that an account of external validity inference break what he calls the ‘Extrapolator’s Circle’ is misplaced, (...) at least when it comes to predictive external validity inferences. (shrink)

There is a takeover movement fast gaining influence in development economics, a movement that demands that predictions about development outcomes be based on randomized controlled trials. The problem it takes up—of using evidence of efficacy from good studies to predict whether a policy will be effective if we implement it—is a general one, and affects us all. My discussion is the result of a long struggle to develop the right concepts to deal with the problem of warranting effectiveness predictions. Whether (...) I have it right or not, these are questions of vast social importance that philosophers of science can, and should, help answer. (shrink)