The replicability of a research claim is often positioned as an important step in establishing the credibility of scientific research. This expectation persists despite ongoing disagreements over how to characterise replication practices in various contexts. Rather than attempt to explain or resolve these disagreements, we propose that there is value in exploring the variable uses of the replicability concept. To this end, we treat the replicability concept as a goal-directed tool for studying scientific practices. This approach extends scholarship on the (...) goal-directed uses of concepts within investigative practices to explore the value of reflexively interrogating those concepts used to study the sciences as practiced. In doing so, we highlight the importance of considering how and when a given concept is an appropriate tool for a given goal when studying scientific practices. For instance, in the case of the replicability concept, our examination suggests that there is value in clarifying how a given characterisation of replicability is appropriate to a given context-specific analytic and/or evaluative goal. In addition, we hope to draw attention to how Metaresearch and HPS contexts provide complementary insights for those seeking to understand when and how replication practices can help in assessing a given research claim. (shrink)

This essay is a contribution to the history of methodological thought. I focus on key methodological criteria for successful experimentation, replication and multiple determinations of empirical evidence. Drawing on reports of experiments with viper venom from the late seventeenth and late eighteenth centuries, as well as on present-day methodological thought I examine whether past experimenters regarded repetition, replication, and multiple determinations as criteria for validity; what exactly they meant by this; what they hoped to gain by repeating, varying, triangulating, and (...) replicating; and how relevant these criteria were for them. I also consider if this analysis has implications for current philosophical work on the methodology of experimental practice. (shrink)

The replication or reproducibility crisis in psychological science has renewed attention to philosophical aspects of its methodology. I provide herein a new, functional account of the role of replication in a scientific discipline: to undercut the underdetermination of scientific hypotheses from data, typically by hypotheses that connect data with phenomena. These include hypotheses that concern sampling error, experimental control, and operationalization. How a scientific hypothesis could be underdetermined in one of these ways depends on a scientific discipline’s epistemic goals, theoretical (...) development, material constraints, institutional context, and their interconnections. I illustrate how these apply to the case of psychological science. I then contrast this “bottom-up” account with “top-down” accounts, which assume that the role of replication in a particular science, such as psychology, must follow from a uniform role that it plays in science generally. Aside from avoiding unaddressed problems with top-down accounts, my bottom-up account also better explains the variability of importance of replication of various types across different scientific disciplines. (shrink)

This paper examines how scientists conceptualize their research methodologies. Do scientists raise concerns about vague criteria and genuine uncertainties in experimental practice? If so, what sorts of issues do they identify as problematic? Do scientists acknowledge the presence of value judgments in scientific research, and do they reflect on the relation between epistemic and non-epistemic criteria for decisionmaking? We present findings from an analysis of qualitative interviews with 63 scientific researchers who talk about their views on good research practice. We (...) argue that analysts of science should care about scientists’ conceptualizations of the criteria and of the practical judgments that scientific inquiry involves. While scientists’ accounts of their own research methodologies alone do not give us a full picture of how science really works, they can point us to areas of concern. They can inspire and direct philosophical reflections about how science works. Throughout the interviews, the participating researchers provided specific examples from their own research contexts as illustrations of their methodological points. These examples reveal that scientists often struggle to evaluate the quality of their data, to figure out whether the available evidence confirms their hypothesis, whether a replication was successful, or to what extent they can rely on peer-reviewed papers. General ideas about good research methods do not directly translate into specific evaluation criteria or strategies that can guide research and help validate empirical data. (shrink)

Abstract: The aim of this paper is to analyse and criticize the argument of the experimenters' regress proposed by Harry Collins in 1985. In order to do that, I begin with an introduction to the experiments that aimed to detect gravity waves performed by Joseph Weber during 1970, then I analyse and discuss both forms of the argument: the epistemological and the ontological. Finally, after giving an outline of a theory of experimental reproduction and an explication of the concept of (...) replication, I propose two ways of avoiding the experimenters' regress. (shrink)

Laboratory experimentation was once considered impossible or irrelevant in economics. Recently, however, economic science has gone through a real ‘laboratory revolution’, and experimental economics is now a most lively subfield of the discipline. The methodological advantages and disadvantages of controlled experimentation constitute the main subject of this thesis. After a survey of the literature on experiments in philosophy and economics, the problem of testing normative theories of rationality is tackled. This philosophical issue was at the centre of a famous controversy (...) in decision theory, during which a methodology of normative falsification was first articulated and used to assess experimental results. In the third chapter, the methodological advantages of controlled experimentation are illustrated and discussed with examples taken from the experiments on the so-called ‘preference reversal’ phenomenon. Laboratory testing allows to establish with a high degree of certainty that certain phenomena lie behind the experimental data, by means of independent testing, elimination of alternative hypotheses, and the use of different instruments of observation. The fourth chapter is devoted to a conceptual analysis of the problem of ‘parallelism’. This is the problem of inferring from the occurrence of a phenomenon in the laboratory, to its instantiation also in non-laboratory environments. Experimental economists have discussed parallelism at length, and their views are presented and criticised. Eventually, it is argued that parallelism is a factual matter and as such can only be established on empirical grounds. The fifth chapter provides an example of how one can argue for parallelism, focusing on the case of experimentation on the ‘winner’s curse’ phenomenon. The role of experiments as ‘mediators’ between theoretical models and their target domain of application is illustrated, and the structure of parallelism arguments analysed in detail. Finally, in the last chapter, economic experiments are compared to simulations, in order to highlight their specific characteristics. (shrink)

This paper explores whether we can interpret the notion of ‘forensic culture’ as something akin to what Knorr-Cetina called an ‘epistemic culture’. Can we speak of a ‘forensic culture’, and, if so, how is it similar to, or different from, other epistemic cultures that exist in what is conventionally called ‘science’? This question has important policy implications given the National Academy Science’s recent identification of ‘culture’ as one of the problems at the root of what it identified as ‘serious deficiencies’ (...) in U.S. forensic science and ‘scientific culture’ as an antidote to those problems. Finding the NAS’s characterisation of ‘scientific culture’ overly general and naïve, this paper offers a preliminary exploration of what might be called a ‘forensic culture’. Specifically, the paper explores the way in which few of the empirical findings accumulated by sociologists of science about research science seem to apply to forensic science. Instead, forensic science seems to have developed a distinct culture for which a sociological analysis will require new explanatory tools. Faithful sociological analysis of ‘forensic culture’ will be a necessary prerequisite for the kind of culture change prescribed by external reformist bodies like the NAS. (shrink)

Despite the fact that reproduction of experiments by peers has traditionally been regarded as of the utmost importance in enabling the intersubjectivity of scientific practice, reproductions may yield discordant results and deciding which result should be favored may not be an easy task. According to Harry Collins, experimental disagreement is resolved by the action of social, political and economic factors, but not by means of epistemic and scientific, or so-called internal reasons. His motivation for such a claim is the presence (...) of an infinite regress at the core of experimental activity that, according to him, cannot be stopped with scientific resources alone: the experimenters’ regress. The goal of this... (shrink)