Proponents of evidence-based medicine and some philosophers of science seem to agree that knowledge of mechanisms can help solve the problem of applying results of controlled studies to target populations (‘the problem of extrapolation’). We describe the problem of extrapolation, characterize mechanisms, and outline how mechanistic knowledge might be used to solve the problem. Our main thesis is that there are four often overlooked problems with using mechanistic knowledge to solve the problem of extrapolation. First, our understanding of mechanisms is (...) often (and arguably, likely to remain) incomplete. Secondly, knowledge of mechanisms is not always applicable outside the tightly controlled laboratory conditions in which it is gained. Thirdly, mechanisms can behave paradoxically. Fourthly, as Daniel Steel points out, using mechanistic knowledge faces the problem of the ‘extrapolator’s circle’. At the same time, when the problems with mechanistic knowledge have been addressed, such knowledge can and should be used to mitigate (nothing can entirely solve) the problem of extrapolation. (shrink)

Arguments that medical decision making should rely on a variety of evidence often begin from the claim that meta-analysis has been shown to be problematic. In this paper, I first examine Stegenga’s argument that meta-analysis requires multiple decisions and thus fails to provide an objective ground for medical decision making. Next, I examine three arguments from social epistemologists that contend that meta-analyses are systematically biased in ways not appreciated by standard epistemology. In most cases I show that critiques of meta-analysis (...) fail to account for the full range of meta-analytic procedures. In the remainder of cases, I argue that the critiques identify problems that do not uniquely cut against meta-analysis. I close by suggesting one reason why it may be pragmatically rational to violate the principle of total evidence and by outlining the criteria for a successful argument against meta-analysis. A set of criteria I contend remain unmet. (shrink)

This paper critically analyzes Sherrilyn Roush's (Tracking truth: knowledge, evidence and science, 2005) definition of evidence and especially her powerful defence that in the ideal, a claim should be probable to be evidence for anything. We suggest that Roush treats not one sense of 'evidence' but three: relevance, leveraging and grounds for knowledge; and that different parts of her argument fare differently with respect to different senses. For relevance, we argue that probable evidence is sufficient but not necessary for Roush's (...) own two criteria of evidence to be met. With respect to grounds for knowledge, we agree that high probability evidence is indeed ideal for the central reason Roush gives: When believing a hypothesis on the basis of e it is desirable that e be probable. But we maintain that her further argument that Bayesians need probable evidence to warrant the method they recommend for belief revision rests on a mistaken interpretation of Bayesian conditionalization. Moreover, we argue that attempts to reconcile Roush's arguments with Bayesianism fail. For leveraging, which we agree is a matter of great importance, the requirement that evidence be probable suffices for leveraging to the probability of the hypothesis if either one of Roush's two criteria for evidence are met. Insisting on both then seems excessive. To finish, we show how evidence, as Roush defines it, can fail to track the hypothesis. This can remedied by adding a requirement that evidence be probable, suggesting another rationale for taking probable evidence as ideal—but only for a grounds-for-knowledge sense of evidence. (shrink)

I aim to show that the failure of the California Class Size Reduction initiative highlights an important class of situations in the application of evidence to policy. There are some circumstances in which the implementation of a policy will be self-defeating. The introduction of the factor assumed to have causal efficacy into the target population can lead to changes in the conditions of the target population that amount to interfering factors. Because these interfering factors are a direct result of the (...) policy implementation they should be relatively easy to predict, and so part of the tricky issue of judging where evidence is relevant should in some circumstances be relatively straightforward. -/- The failure of the California Class Size Reduction initiative also shows how important it is to identify the correct causal factor. The more accurate the attribution of causality, the less susceptible it will be to interfering factors and breaks in the causal chain. (shrink)