External validity is typically regarded as the downside of case study research by methodologists and social scientists; case studies, however, are often aimed at drawing lessons that are generalizable to new contexts. The gap between the generalizability potential of case studies and the research goals demands closer scrutiny. I suggest that the conclusion that case study research is weak in external validity follows from a set of assumptions that I term the "traditional view," which are disputable at best. In this (...) view, external validity is treated as a matter of mere representativeness. I argue that it is best understood instead as a problem of inference and that the emphasis should be placed on the comparability of the study rather than on the typicality of the case. By making case studies highly comparable, their external validity can be reliably and efficiently assessed and, in this way, their generalizability potential enhanced. (shrink)

An important question for the causal modeling approach is how to integrate non‐causal dependence relations such as asymmetric supervenience into the approach. The most prominent proposal to that effect (due to Gebharter) is to treat those dependence relationships as formally analogous to causal relationships. We argue that this proposal neglects some crucial differences between causal and non‐causal dependencies, and that in the context of causal modeling non‐causal dependence relationships should be represented as mutual dependence relationships. We develop a new kind (...) of model – “hybrid models” ‐ based on this suggestion, and formulate a set of axioms for those models. Our formalism has important implications for Kim's exclusion problem: whereas Gebharter's framework vindicates Kim's causal exclusion objection against nonreductive physicalism, our framework has no such implication, and can help non‐reductive physicalists vindicate the efficacy of high‐level properties. A further benefit of our formalism is that it yields a natural and plausible way of thinking about interventions in multi‐level contexts. (shrink)

Extrapolating causal effects is becoming an increasingly important kind of inference in Evidence-Based Policy, development economics, and microeconometrics more generally. While several strategies have been proposed to aid with extrapolation, the existing methodological literature has left our understanding of what extrapolation consists of and what constitutes successful extrapolation underdeveloped. This paper addresses this lack in understanding by offering a novel account of successful extrapolation. Building on existing contributions pertaining to the challenges involved in extrapolation, this more nuanced and comprehensive account (...) seeks to provide tools that facilitate the scrutiny of specific extrapolative inferences and general strategies for extrapolation. Offering such resources is important especially in view of the increasing amounts of real-world decision-making in policy, development, and beyond that involve extrapolation. (shrink)

The experimental revolution in the social sciences is one of the most significant methodological shifts undergone by the field since the ‘quantitative revolution’ in the nineteenth century. One of the often valued features of social science experimentation is precisely the fact that there are clear methodological rules regarding hypothesis testing that come from the methods of the natural sciences and from the methodology of RCTs in the biomedical sciences, and that allow for the adjudication among contentious causal claims. We examine (...) critically this claim and argue that some current understandings of the practices that surround social science experimentation overestimate the degree to which experiments can actually fulfil this role as “objective” adjudicators, by neglecting the importance of shared background knowledge or assumptions and of consensus regarding the validity of the constructs involved in an experiment. We take issue with the way the distinction between internal and external validity is often used to comment on the inferential import of experiments, used both among practitioners and among philosophers of science. We describe the ways in which the more common use of the internal/external distinction differs from Cook and Campbell’s original methodological project, in which construct validity and the four-fold validity typology were all important in assessing the inferential import of experiments. We argue that the current uses of the labels internal and external, as applied to experimental validity, help to encroach a simplistic view on the inferential import of experiments that, in turn, misrepresents their capacity to provide objective knowledge about the causal relations between variables. (shrink)

This paper argues that an important type of experiment-target inference, extrapolating causal effects, requires models to be successful. Focusing on extrapolation in Evidence-Based Policy, it is ar...

Sustainability science seeks to extend scientific investigation into domains characterized by a distinct problem-solving agenda, physical and social complexity, and complex moral and ethical landscapes. In this endeavor it arguably pushes scientific investigation beyond its usual comfort zones, raising fundamental issues about how best to structure such investigation. Philosophers of science have long scrutinized the structure of science and scientific practices, and the conditions under which they operate effectively. We propose a critical engagement between sustainability scientists and philosophers of science (...) with respect to how to engage in scientific activity in these complex domains. We identify specific issues philosophers of science raise concerning current sustainability science and the contributions philosophers can make to resolving them. In conclusion we reflect on the steps philosophers of science could take to advance sustainability science. (shrink)

A shared problem across the sciences is to make sense of correlational data coming from observations and/or from experiments. Arguably, this means establishing when correlations are causal and when they are not. This is an old problem in philosophy. This paper, narrowing down the scope to quantitative causal analysis in social science, reformulates the problem in terms of the validity of statistical models. Two strategies to make sense of correlational data are presented: first, a 'structural strategy', the goal of which (...) is to model and test causal structures that explain correlational data; second, a 'manipulationist or interventionist strategy', that hinges upon the notion of invariance under intervention. It is argued that while the former can offer a solution the latter cannot. (shrink)

Although researchers agree on the importance of validity, they have not yet reached a consensus on what validity consists of. This article traces the historic trajectories of validity theory development in three research traditions: psychometrics, experiment in social settings, and animal models of psychiatric disorders, showing that the multiplicity in the concept of validity is shaped by its multiple historic trajectories and reflects the diversity of practices and concerns in different research traditions. I argue that specifying validity of what target (...) practice and for what purpose in discussions helps to connect validity to its rich context that gives rise to its specific meaning and relevance. (shrink)

Martin Jones has criticized my account of the methodology of experimental economics on three points: the impossibility of testing external validity claims in the laboratory, my reconstruction of external validity inferences as analogical arguments, and the distinction between laboratory and non-laboratory sciences. I defend my account here and try to eliminate some misunderstandings that may have prompted Jones’s criticism.

This article highlights a use of analogies in science that so far has received relatively little systematic discussion: providing reasons for pursuing a model or theory. Using the development of the liquid drop model as a test case, I critically assess two extant pursuit worthiness accounts: that analogies justify pursuit by supporting plausibility arguments and that analogies can serve as a guide to potential theoretical unification. Neither of these fit the liquid drop model case. Instead, I develop an alternative account, (...) based on the idea that analogies facilitate the transfer of a well-understood modelling strategy to a new domain. 1Introduction2Case Study: The Development of the Liquid Drop Model3Plausibility Accounts 3.1Bartha on plausibility and analogical inference3.2Plausibility and the drop analogy4Analogies as a Guide to Unification5Generative Accounts 5.1Analogy-based modelling strategies5.2Did analogies play a merely generative role?6A New Pursuit Worthiness Account of Analogies 6.1Transferring understanding-with through analogies6.2Understanding-with and the liquid drop model7Conclusion. (shrink)

Animal models have received particular attention as key examples of material models. In this paper, we argue that the specificities of establishing animal models—acknowledging their status as living beings and as epistemological tools—necessitate a more complex account of animal models as materialised models. This becomes particularly evident in animal-based models of diseases that only occur in humans: in these cases, the representational relation between animal model and human patient needs to be generated and validated. The first part of this paper (...) presents an account of how disease-specific animal models are established by drawing on the example of transgenic mice models for Alzheimer’s disease. We will introduce an account of validation that involves a three-fold process including from human being to experimental organism; from experimental organism to animal model; and from animal model to human patient. This process draws upon clinical relevance as much as scientific practices and results in disease-specific, yet incomplete, animal models. The second part of this paper argues that the incompleteness of models can be described in terms of multi-level abstractions. We qualify this notion by pointing to different experimental techniques and targets of modelling, which give rise to a plurality of models for a specific disease. (shrink)

While the history and methodology of laboratory experiments in economics have been extensively studied by philosophers, those of field experiments have not attracted much attention until recently. What is the historical context in which field experiments have been advocated? And what are the methodological rationales for conducting experiments in the field as opposed to in the lab? This article addresses these questions by combining historical and methodological perspectives. In terms of history, we show that the movement toward field experiments in (...) economics has two distinct roots. One is the general orientation of medical and social sciences to evidence-based policy evaluation, which gave rise to randomized field experiments in economics (e.g., behavioral public policy, poverty alleviation policy). The other is an awareness of several methodological limitations of lab experiments in economics, which required practitioners to get out of the lab and into the field. In these senses, the movement is a consequence of influences from both outside and inside economics: the general evidence-based trend in policy science and an internal methodological development of experimental economics. In terms of methodology, we show that these two roots resulted in two somewhat different notions of “external validity” as methodological rationales of field experiment. Finally, we suggest that analysis of experiments as exhibits highlights a methodological strategy in which both strands complement each other. (shrink)

We argue that behavioural public policies (BPP) should be categorized by the kind of mechanism through which they operate, not by the kind of treatment they implement. Reviewing the energy consumption BPP literature, we argue (i) that BPPs are currently categorized by treatment; (ii) that treatment-based categories are subject to mechanistic heterogeneity: there is substantial variation of mechanisms within each treatment type; and (iii) that they also display mechanistic overlap: there is substantial overlap between mechanisms across treatment types. Consequently, current (...) categorizations of BPPs do not reveal the conditions of their efficacy and should be revised to better reflect mechanistic information. (shrink)