Mitchell & Gronenborn propose that we account for the presence of multiple models of protein structure, each produced in different contexts, through the framework of integrative pluralism. I argue that two interpretations of this framework are available, neither of which captures the relationship between a model and the protein structure it represents or between multiple models of protein structure. Further, it inclines us toward concluding prematurely that models of protein structure are right in their contexts and makes extrapolation of findings (...) from one context to another seem unwarranted. Instead, protein structure determination ought to be understood as modestly monistic. There is one model for every protein in each physicochemical context, and models of the same protein produced in different contexts are compatible with one another. ‘Integrating’ multiple models amounts to extrapolating from one context to another; this is possible because the effect of context on protein folding is relatively weak and predictable. Modest monism better describes the practice of protein structure determination than integrative pluralism and enables greater attention to how context affects protein folding. (shrink)

While the term “reactivity” has come to be associated with specific phenomena in the social sciences, having to do with subjects’ awareness of being studied, this paper takes a broader stance on this concept. I argue that reactivity is a ubiquitous feature of the psychological subject matter and that this fact is a precondition of experimental research, while also posing potential problems for the experimenter. The latter are connected to the worry about distorted data and experimental artifacts. But what are (...) experimental artifacts and what is the most productive way of dealing with them? In this paper, I approach these questions by exploring the ways in which experimenters in psychology simultaneously exploit and suppress the reactivity of their subject matter in order to produce experimental data that speak to the question or subject matter at hand. Highlighting the artificiality of experimental data, I raise the question of what distinguishes a genuine experimental result from an experimental artifact. My analysis construes experimental results as the outcomes of inferences from the data that take material background assumptions as auxiliary premises. Artifacts occur when one or more of these background assumptions are false, such that the data do not reliably serve the purposes they were generated for. I conclude by laying out the ways in which my analysis of data quality is relevant to, and informed by, recent debates about the replicability of experimental results. (shrink)

There is an ongoing and robust tradition of science and technology studies scholars conducting ethnographic laboratory studies. These laboratory studies—like all ethnographies—are each conducted at a particular time, are situated in a particular place, and are about a particular culture. Presumably, this contextual specificity means that such ethnographies have limited applicability beyond the narrow slice of time, place, and culture that they each subject to examination. But we do not always or even often treat them that way. It is beyond (...) common for us to speak about what one or another laboratory study reveals about the laboratory, or “science” much more broadly. Given the contextual specificity of our ethnographic laboratory studies, what justifies this presumed generalizability? Initially, this manuscript surveys typical responses to this question, but then it pursues an unusual one: the potential replicability of ethnographic results. This potential is hereby explored, via an ethnographic replication attempt—one designed and conducted in order to test the generalizability of a particular laboratory study, that of Latour and Woolgar’s classic Laboratory Life. The results of the ethnographic replication attempt are reported, and a remarkable degree of replicability is established. (shrink)

In the human sciences, experimental research is used to establish causal relationships. However, the extrapolation of these results to the target population can be problematic. To facilitate extrapolation, we propose to use the statistical technique Latent Class Regression Analysis in combination with the analogical reasoning theory for extrapolation. This statistical technique can identify latent classes that differ in the effect of X on Y. In order to extrapolate by means of analogical reasoning, one can characterize the latent classes by a (...) combination of features, and then compare these features to features of the target. (shrink)

Randomized experiments, as developed by Esther Duflo and Abhijit Banerjee at the Abdul Latif Jameel Poverty Action Lab, offer a novel, evidence-based approach to fighting poverty. This approach is original, in that it imports the methodology of clinical trials for application in development economics. This paper examines the analogy between J-PAL’s field experiments in development economics and randomized controlled trials in medicine. RCTs and randomized field experiments are commonly treated as identical, but such treatment neglects some of the major distinguishing (...) features that make each experiment specifically apt for use in its respective field. The central claim of this paper is that the analogy between medicine and development economics is incomplete because the central dimensions of RCTs are not simply different but altogether lacking in J-PAL’s approach. This weakens both the political and the theoretical power of such experiments in development economics. (shrink)

Neuroscience is a laboratory-based science that spans multiple levels of analysis from molecular genetics to behavior. At every level of analysis experiments are designed in order to answer empirical questions about phenomena of interest. Understanding the nature and structure of experimentation in neuroscience is fundamental for assessing the quality of the evidence produced by such experiments and the kinds of claims that are warranted by the data. This article provides a general conceptual framework for thinking about evidence and experimentation in (...) neuroscience with a particular focus on two research areas: cognitive neuroscience and cognitive neurobiology. (shrink)

External validity is typically regarded as the downside of case study research by methodologists and social scientists; case studies, however, are often aimed at drawing lessons that are generalizable to new contexts. The gap between the generalizability potential of case studies and the research goals demands closer scrutiny. I suggest that the conclusion that case study research is weak in external validity follows from a set of assumptions that I term the "traditional view," which are disputable at best. In this (...) view, external validity is treated as a matter of mere representativeness. I argue that it is best understood instead as a problem of inference and that the emphasis should be placed on the comparability of the study rather than on the typicality of the case. By making case studies highly comparable, their external validity can be reliably and efficiently assessed and, in this way, their generalizability potential enhanced. (shrink)

The article identifies a latent debate in the recent literature on the role and worth of experiments in economics and other social sciences concerning the relationship between the external and the internal validity of experimental designs. Our work identifies two incompatible views regarding the relationship between internal and external validity of experiments. While in the methodological literature references to the idea that there is a trade-off between the internal and external validity of experiments abound, this view coexists with the position (...) stating that internal validity is rather a prerequisite of external validity. By identifying the contours of this implicit debate in the recent methodological literature around the use of experiments in the social sciences we call attention upon a series of insufficiently conceptualized issues regarding the central notions of internal and external validity and we question the standard view positing a trade-off between the two. This article stands against common associations of internal validity and external validity with the distinction between field and laboratory experiments and assesses critically the arguments that link the artificiality of experimental settings done in the laboratory with the purported trade-off between internal and external validity. (shrink)

Descriptive accounts of the nature of explanation in neuroscience and the global goals of such explanation have recently proliferated in the philosophy of neuroscience and with them new understandings of the experimental practices of neuroscientists have emerged. In this paper, I consider two models of such practices; one that takes them to be reductive; another that takes them to be integrative. I investigate those areas of the neuroscience of learning and memory from which the examples used to substantiate these models (...) are culled, and argue that the multiplicity of experimental protocols used in these research areas presents specific challenges for both models. In my view, these challenges have been overlooked largely because philosophers have hitherto failed to pay sufficient attention to fundamental features of experimental practice. I demonstrate that when we do pay attention to such features, evidence for reduction and integrative unity in neuroscience is simply not borne out. I end by suggesting some new directions for the philosophy of neuroscience that pertain to taking a closer look at the nature of neuroscientific experiments. (shrink)

The use of neuroscientific evidence in criminal trials has been steadily increasing. Despite progress made in recent decades in understanding the mechanisms of psychological and behavioral functioning, neuroscience is still in an early stage of development and its potential for influencing legal decision-making is highly contentious. Scholars disagree about whether or how neuroscientific evidence might impact prescriptions of criminal culpability, particularly in instances in which evidence of an accused’s history of mental illness or brain abnormality is offered to support a (...) plea of not criminally responsible. In the context of these debates, philosophers and legal scholars have identified numerous problems with admitting neuroscientific evidence in legal contexts. To date, however, less has been said about the challenges of evaluating the evidence upon which integrative mechanistic explanations that bring together evidence from different areas of neuroscience are based. As we explain, current criteria for evaluating such evidence to determine its admissibility in legal contexts are inadequate. Appealing to literature in the philosophy of scientific experimentation and theoretical work in the social, cognitive and behavioral sciences, we lay the groundwork for reforming these criteria and identify some of the implications of modifying them. (shrink)

I consider two transcendental arguments for realism in the philosophy of science, which are due to Roy Bhaskar (A realist theory of science, 1975) and Nancy Cartwright (The dappled world, 1999). Bhaskar and Cartwright are both influential figures, however there is little discussion of their use of transcendental arguments in the literature. Here I seek to correct this oversight. I begin by describing the role of the transcendental arguments in question, in the context of the broader philosophical theories in which (...) they are embedded, by Bhaskar and Cartwright respectively. I then consider some specific problems that arise for these particular transcendental arguments, in the context of contemporary philosophy of science. I raise two general problems for transcendental arguments for realism and I finish by spelling out what needs to be done to address the criticisms raised in this paper. (shrink)

Zusammenfassung Case studies in the history of science and technology have shown that scientific norms, so called standards, contribute significantly to the evolution of scientific practices. They arise predominantly, but not exclusively, on the basis of interactions with instruments of measurement and other technical devices. As regards experimental practices standards are mandatory preparatory procedures in a variety of designs, including the inbreeding and genetic engineering of experimental organisms (e.g. transgenic mice). I claim that scientific norms not only regulate mere technical (...) preconditions of research but also guide experimental practices, for example with regard to the stabilisation and validation of phenomena. Against this background, the paper introduces different kinds of scientific norms and elaborates on the question if they are means to epistemic ends (e.g. stability). (shrink)

We show that the common claim that internal validity should be understood as prior to external validity has, at least, three epistemologically problematic aspects: experimental artefacts, the implications of causal relations, and how the mechanism is measured. Each aspect demonstrates how important external validity is for the internal validity of the experimental result.

These preprints were automatically compiled into a PDF from the collection of papers deposited in PhilSci-Archive in conjunction with the PSA 2012.

The replicability of a research claim is often positioned as an important step in establishing the credibility of scientific research. This expectation persists despite ongoing disagreements over how to characterise replication practices in various contexts. Rather than attempt to explain or resolve these disagreements, we propose that there is value in exploring the variable uses of the replicability concept. To this end, we treat the replicability concept as a goal-directed tool for studying scientific practices. This approach extends scholarship on the (...) goal-directed uses of concepts within investigative practices to explore the value of reflexively interrogating those concepts used to study the sciences as practiced. In doing so, we highlight the importance of considering how and when a given concept is an appropriate tool for a given goal when studying scientific practices. For instance, in the case of the replicability concept, our examination suggests that there is value in clarifying how a given characterisation of replicability is appropriate to a given context-specific analytic and/or evaluative goal. In addition, we hope to draw attention to how Metaresearch and HPS contexts provide complementary insights for those seeking to understand when and how replication practices can help in assessing a given research claim. (shrink)

The experimental revolution in the social sciences is one of the most significant methodological shifts undergone by the field since the ‘quantitative revolution’ in the nineteenth century. One of the often valued features of social science experimentation is precisely the fact that there are clear methodological rules regarding hypothesis testing that come from the methods of the natural sciences and from the methodology of RCTs in the biomedical sciences, and that allow for the adjudication among contentious causal claims. We examine (...) critically this claim and argue that some current understandings of the practices that surround social science experimentation overestimate the degree to which experiments can actually fulfil this role as “objective” adjudicators, by neglecting the importance of shared background knowledge or assumptions and of consensus regarding the validity of the constructs involved in an experiment. We take issue with the way the distinction between internal and external validity is often used to comment on the inferential import of experiments, used both among practitioners and among philosophers of science. We describe the ways in which the more common use of the internal/external distinction differs from Cook and Campbell’s original methodological project, in which construct validity and the four-fold validity typology were all important in assessing the inferential import of experiments. We argue that the current uses of the labels internal and external, as applied to experimental validity, help to encroach a simplistic view on the inferential import of experiments that, in turn, misrepresents their capacity to provide objective knowledge about the causal relations between variables. (shrink)

Since the discovery of dark matter in the 1980s, multiple experiments have been set up to detect dark matter particles through some other mode than gravity. Particle physicists provide detailed jus...

Francesco Guala once wrote that ‘The problem of extrapolation is a minor scandal in the philosophy of science’. This paper agrees with the statement, but for reasons different from Guala’s. The scandal is not, or not any longer, that the problem has been ignored in the philosophy of science. The scandal is that framing the problem as one of external validity encourages poor evidential reasoning. The aim of this paper is to propose an alternative—an alternative which constitutes much better evidential (...) reasoning about target systems of interest, and which makes do without consideration of external validity. (shrink)

Animal models have received particular attention as key examples of material models. In this paper, we argue that the specificities of establishing animal models—acknowledging their status as living beings and as epistemological tools—necessitate a more complex account of animal models as materialised models. This becomes particularly evident in animal-based models of diseases that only occur in humans: in these cases, the representational relation between animal model and human patient needs to be generated and validated. The first part of this paper (...) presents an account of how disease-specific animal models are established by drawing on the example of transgenic mice models for Alzheimer’s disease. We will introduce an account of validation that involves a three-fold process including from human being to experimental organism; from experimental organism to animal model; and from animal model to human patient. This process draws upon clinical relevance as much as scientific practices and results in disease-specific, yet incomplete, animal models. The second part of this paper argues that the incompleteness of models can be described in terms of multi-level abstractions. We qualify this notion by pointing to different experimental techniques and targets of modelling, which give rise to a plurality of models for a specific disease. (shrink)

While the history and methodology of laboratory experiments in economics have been extensively studied by philosophers, those of field experiments have not attracted much attention until recently. What is the historical context in which field experiments have been advocated? And what are the methodological rationales for conducting experiments in the field as opposed to in the lab? This article addresses these questions by combining historical and methodological perspectives. In terms of history, we show that the movement toward field experiments in (...) economics has two distinct roots. One is the general orientation of medical and social sciences to evidence-based policy evaluation, which gave rise to randomized field experiments in economics (e.g., behavioral public policy, poverty alleviation policy). The other is an awareness of several methodological limitations of lab experiments in economics, which required practitioners to get out of the lab and into the field. In these senses, the movement is a consequence of influences from both outside and inside economics: the general evidence-based trend in policy science and an internal methodological development of experimental economics. In terms of methodology, we show that these two roots resulted in two somewhat different notions of “external validity” as methodological rationales of field experiment. Finally, we suggest that analysis of experiments as exhibits highlights a methodological strategy in which both strands complement each other. (shrink)

Francesco Guala has developed some novel and radical ideas on the problem of external validity, a topic that has not received much attention in the experimental economics literature. In this paper I argue that his views on external validity are not justified and the conclusions which he draws from these views, if widely adopted, could substantially undermine the experimental economics enterprise. In rejecting the justification of these views, the paper reaffirms the importance of experiments in economics.

Résumé L’article évoque les bases de la méthode expérimentale en général et son adaptation à l’étude des phénomènes économiques tout en rendant compte des réflexions philosophiques récentes sur la nature des expériences en laboratoire. Il recense par ailleurs les principales procédures expérimentales qui caractérisent la bonne pratique des économistes – dont certaines témoignent d’une grande ingéniosité – en pointant en parallèle ce qui la sépare de celle plus ancienne des psychologues. La question des incitations financières y fait l’objet d’une attention (...) toute particulière. L’article souligne les immenses progrès réalisés en quelques décennies comme le caractère très professionnel de la démarche expérimentale, sans pour autant occulter les désaccords qui persistent entre certains expérimentalistes et l’existence de quelques questions « ouvertes » d’importance. A cet égard, l’article tente de faire le point sur l’un des questionnements méthodologiques qui, bien que central dans un argumentaire de défense contre la critique courante du caractère « artificiel » des expériences, n’est vraiment abordé de manière sérieuse que depuis peu : le problème de la « validité externe » des résultats expérimentaux, i.e. leur capacité à être généralisés en dehors du laboratoire. (shrink)

A hallmark of ecological research is dealing with complexity in the systems under investigation. One strategy is to diminish this complexity by constructing models and theories that are general. Alternatively, ecologists can constrain the scope of their generalizations to particular phenomena or types of systems. However, research employing the second strategy is often met with scathing criticism. I offer a theoretical argument in support of moderate generalizations in ecological research, based on the notions of interdependence and causal heterogeneity and their (...) effect on the trade-off between generality and realism. (shrink)

Allan Franklin has identified a number of strategies that scientists use to build confidence in experimental results. This paper shows that Franklin's strategies have direct analogues in the context of computer simulation and then suggests that one of his strategies—the so-called 'Sherlock Holmes' strategy—deserves a privileged place within the epistemologies of experiment and simulation. In particular, it is argued that while the successful application of even several of Franklin's other strategies (or their analogues in simulation) may not be sufficient for (...) justified belief in results, the successful application of a slightly elaborated version of the Sherlock Holmes strategy is sufficient. (shrink)

It is sometimes said that simulation can serve as epistemic substitute for experimentation. Such a claim might be suggested by the fast-spreading use of computer simulation to investigate phenomena not accessible to experimentation. But what does that mean? The paper starts with a clarification of the terms of the issue and then focuses on two powerful arguments for the view that simulation and experimentation are ‘epistemically on a par’. One is based on the claim that, in experimentation, no less than (...) in simulation, it is not the system under study that is manipulated but a system that ‘stands-in’ for it. The other one highlights the pervasive use of models in experimentation. It will be argued that these arguments, as compelling as they might seem, are each based on a mistaken interpretation of experimentation and that, far from simulation and experimentation being epistemically on a par, they do not have the same epistemic function, do not produce the same kind of epistemic results. (shrink)

Reliability and external validity are two fundamental values that pose incompatible constraints on neurobiological experiments. The more reliability an experimental result achieves, the less external validity it earns, and vice versa. In this article, I propose an externalist interpretation of external validity: the external validity of an experimental result depends not only on how much complexity is built into an experimental design, but also on the relationship between the experimental result and other related experiments. This externalist interpretation, which explains how (...) a neurobiological experiment can be reliable and externally valid simultaneously, suggests a new way to understand the epistemology of neurobiological experimentation. (shrink)

This PhD thesis focuses on the philosophical foundations of Neuroeconomics, an innovative research program which combines findings and modelling tools from economics, psychology and neuroscience to account for human choice behaviour. The proponents of Neuroeconomics often manifest the ambition to foster radical modifications in the accounts of choice behaviour developed by its parent disciplines. This enquiry provides a philosophically informed appraisal of the potential for success and the relevance of neuroeconomic research for economics. My central claim is that neuroeconomists can (...) help other economists to build more predictive and explanatory models, yet are unlikely to foster revolutionary modifications in the economic theory of choice. The contents are organized as follows. In chapters 1-2, I present neuroeconomists’ investigative tools, distinguish the most influential approaches to neuroeconomic research and reconstruct the case in favour of a neural enrichment of economic theory. In chapters 3-7, I combine insights from neuro-psychology, economic methodology and philosophy of science to develop a systematic critique of Neuroeconomics. In particular, I articulate four lines of argument to demonstrate that economists are provisionally justified in retaining a methodologically distinctive approach to the modelling of decision making. My first argument points to several evidential and epistemological concerns which complicate the interpretation of neural data and cast doubt on the inferences neuroeconomists often make in their studies. My second argument aims to show that the trade-offs between the modelling desiderata that neuroeconomists and other economists respectively value severely constrain the incorporation of neural insights into economic models. My third argument questions neuroeconomists’ attempts to develop a unified theory of choice behaviour by identifying some central issues on which they hold contrasting positions. My fourth argument differentiates various senses of the term ‘revolution’ and illustrates that neuroeconomists are unlikely to provide revolutionary contributions to economic theory in any of these senses. (shrink)

Over the past seven decades, randomised controlled trials (RCTs) have revolutionised clinical research and achieved a gold standard status. However, extending their use to evaluate complex interventions is problematic. In this paper we will demonstrate that complex intervention RCTs violate the necessary premises that govern the RCTs logic and underpin their rigour. The lack of blinding, heterogeneity of participants, as well as poor treatment standardisation and difficulty of controlling for confounders, which characterise complex intervention RCTs, can potentially be profoundly detrimental (...) to their integrity. Proponents of this approach argue that matching “real world” circumstances, while maintaining the randomised design, enhances external validity. We counter this argument by pointing out that an inverted U relation exists between internal and external validity, and thus relaxing the experimental conditions beyond a certain threshold can potentially paradoxically render the RCT externally invalid, i.e. its results cannot be used anywhere. We shall illustrate the inappropriate use of RCTs to evaluate community treatment orders and propose an alternative epistemic model that is based on mechanistic reasoning and Cartwright's capacity concept. (shrink)