I consider recent strategies proposed by econometricians for extrapolating causal effects from experimental to target populations. I argue that these strategies fall prey to the extrapolator’s circle: they require so much knowledge about the target population that the causal effects to be extrapolated can be identified from information about the target alone. I then consider comparative process tracing as a potential remedy. Although specifically designed to evade the extrapolator’s circle, I argue that CPT is unlikely to facilitate extrapolation in typical (...) econometrics and evidence-based policy applications. To argue this, I offer a distinction between two kinds of extrapolation, attributive and predictive, the latter being prevalent in econometrics and evidence-based policy. I argue that CPT is not helpful for predictive extrapolation when using the kinds of evidence that econometricians and evidence-based policy researchers prefer. I suggest that econometricians may need to consider qualitative evidence to overcome this problem. (shrink)

I argue that non-epistemic values are necessarily embedded in the measure of evidential strength of contrastive evidence. When evidence is contrastive, evidence is stronger the more it favours a hypothesis over a set of plausible, mutually exclusive alternative hypotheses. In such a contrastive epistemic setting, evidence has an effect not only on a particular hypothesis, but on the whole probability distribution over the set of alternative hypotheses. A natural way of analysing the incremental impact of new evidence on a set (...) of alternative hypotheses is in terms of uncertainty or ‘entropy’ reduction. There is no unique single measure of uncertainty/entropy, however, and, consequently, no single unique measure of uncertainty reduction. Finally, I argue that the ‘right’ measure of entropy reduction, and hence of evidential strength, depends on the pragmatic context and non-epistemic values. (shrink)

This paper argues that an important type of experiment-target inference, extrapolating causal effects, requires models to be successful. Focusing on extrapolation in Evidence-Based Policy, it is ar...

Evidence-based practice in education involves basing decisions about what to do on evidence about the relative effectiveness of available interventions (e.g. programmes, products, practices). This article considers two influential critiques of evidence-based education (EBE) pertaining to its treatment of values. The ‘general critique’ condemns EBE for excluding values from decisions about what to do in education. The ‘specific critique’ condemns EBE for relying on a deterministic view of causality in education which disregards the complex, value-laden nature of educational contexts. I (...) argue that virtually all versions of EBE escape the general critique, including the dominant intervention-centred approach that relies on experimental research to discover ‘what works’, because the predictions EBE aims to support are only one premise in a broader normative argument. Further, intervention-centred EBE can avoid much of the specific values-based critique because it is consistent with a probabilistic, rather than deterministic, understanding of causality. However, I argue that only a context-centred approach to EBE that relies on evidence about the specific target setting from local sources in addition to evidence from theory and mixed methods research can fully address the specific critique by accommodating critics’ descriptive claims about the nature of educational contexts. (shrink)

In evaluating science, philosophers tends to focus on general laws and on their truth. I urge a shift in focus to the reliability of the panoply of outputs science produces and in tandem, from the general to the particular. Here I give five arguments to support this, including many, many scientific outputs, which must be supposed reliable if we are to warrant our general principles, aren’t truth-apt; and reliability invites the crucial question, ‘Reliable for what?’ Getting clear the particular purpose (...) is essential for effective action and just evaluation. (shrink)