In his influential book, "Changing Order", H.M. Collins puts forward the following three claims concerning experimental replication. (i) Replication is rarely practiced by experimentalists; (ii) replication cannot be used as an objective test of scientific knowledge claims, because of the occurrence of the so-called experimenters' regress; and (iii) stopping this regress at some point depends upon the enculturation in a local community of practitioners, who tacitly learn the relevant skills. In my paper I discuss and assess these claims on the (...) basis of a more comprehensive analysis of experimentation and experimental reproducibility. The main point is that Collins' claims are not, strictly speaking, wrong, but rather too one-sided and therefore inadequate. This point also calls for a reconsideration of the radical (social constructivist) conclusions that Collins has drawn from his studies of scientific experimentation. (shrink)

Imre Lakatos; II—Criticism and the Methodology of Scientific Research Programmes, Proceedings of the Aristotelian Society, Volume 69, Issue 1, 1 June 1969, Page.

Because physical theories typically predict numerical values, an improvement in experimental precision reduces the tolerance range and hence increases corroborability. In most psychological research, improved power of a statistical design leads to a prior probability approaching 1/2 of finding a significant difference in the theoretically predicted direction. Hence the corroboration yielded by "success" is very weak, and becomes weaker with increased precision. "Statistical significance" plays a logical role in psychology precisely the reverse of its role in physics. This problem is (...) worsened by certain unhealthy tendencies prevalent among psychologists, such as a premium placed on experimental "cuteness" and a free reliance upon ad hoc explanations to avoid refutation. (shrink)

I propose a framework that explicates and distinguishes the epistemic roles of data and models within empirical inquiry through consideration of their use in scientific practice. After arguing that Suppes’ characterization of data models falls short in this respect, I discuss a case of data processing within exploratory research in plant phenotyping and use it to highlight the difference between practices aimed to make data usable as evidence and practices aimed to use data to represent a specific phenomenon. I then (...) argue that whether a set of objects functions as data or models does not depend on intrinsic differences in their physical properties, level of abstraction or the degree of human intervention involved in generating them, but rather on their distinctive roles towards identifying and characterizing the targets of investigation. The paper thus proposes a characterization of data models that builds on Suppes’ attention to data practices, without however needing to posit a fixed hierarchy of data and models or a highly exclusionary definition of data models as statistical constructs. (shrink)

Current debates about the replication crisis in psychology take it for granted that direct replication is valuable and focus their attention on questionable research practices in regard to statistical analyses. This paper takes a broader look at the notion of replication as such. It is argued that all experimentation/replication involves individuation judgments and that research in experimental psychology frequently turns on probing the adequacy of such judgments. In this vein, I highlight the ubiquity of conceptual and material questions in research, (...) and I argue that replication is not as central to psychological research as it is sometimes taken to be. (shrink)

Many philosophers of science and methodologists have argued that the ability to repeat studies and obtain similar results is an essential component of science. A finding is elevated from single observation to scientific evidence when the procedures that were used to obtain it can be reproduced and the finding itself can be replicated. Recent replication attempts show that some high profile results – most notably in psychology, but in many other disciplines as well – cannot be replicated consistently. These replication (...) attempts have generated a considerable amount of controversy, and the issue of whether direct replications have value has, in particular, proven to be contentious. However, much of this discussion has occurred in published commentaries and social media outlets, resulting in a fragmented discourse. To address the need for an integrative summary, we review various types of replication studies and then discuss the most commonly voiced concerns about direct replication. We provide detailed responses to these concerns and consider different statistical ways to evaluate replications. We conclude there are no theoretical or statistical obstacles to making direct replication a routine aspect of psychological science. (shrink)

The replication (replicability, reproducibility) crisis in social psychology and clinical medicine arises from the fact that many apparently well-confirmed experimental results are subsequently overturned by studies that aim to replicate the original study. The culprit is widely held to be poor science: questionable research practices, failure to publish negative results, bad incentives, and even fraud. In this article I argue that the high rate of failed replications is consistent with high-quality science. We would expect this outcome if the field of (...) science in question produces a high proportion of false hypotheses prior to testing. If most of the hypotheses under test are false, then there will be many false hypotheses that are apparently supported by the outcomes of well conducted experiments and null hypothesis significance tests with a type-I error rate (α) of 5%. Failure to recognize this is to commit the fallacy of ignoring the base rate. I argue that this is a plausible diagnosis of the replication crisis and examine what lessons we thereby learn for the future conduct of science. (shrink)

This article focuses on the role of statistical concepts in both experiment and theory in various scientific disciplines, especially physics, including astronomy, and psychology. In Sect. 1 the concept of uncertainty in astronomy is analyzed from Ptolemy to Laplace and Gauss. In Sect. 2 theoretical uses of probability and statistics in science are surveyed. Attention is focused on the historically important example of radioactive decay. In Sect. 3 the use of statistics in biology and the social sciences is examined, with (...) detailed consideration of various Chi-square statistical tests. Such tests are essential for proper evaluation of many different kinds of scientific hypotheses. (shrink)

Scientists have used models for hundreds of years as a means of describing phenomena and as a basis for further analogy. In Scientific Models in Philosophy of Science, Daniela Bailer-Jones assembles an original and comprehensive philosophical analysis of how models have been used and interpreted in both historical and contemporary contexts. Bailer-Jones delineates the many forms models can take (ranging from equations to animals; from physical objects to theoretical constructs), and how they are put to use. She examines early mechanical (...) models employed by nineteenth-century physicists such as Kelvin and Maxwell, describes their roots in the mathematical principles of Newton and others, and compares them to contemporary mechanistic approaches. Bailer-Jones then views the use of analogy in the late nineteenth century as a means of understanding models and to link different branches of science. She reveals how analogies can also be models themselves, or can help to create them. The first half of the twentieth century saw little mention of models in the literature of logical empiricism. Focusing primarily on theory, logical empiricists believed that models were of temporary importance, flawed, and awaiting correction. The later contesting of logical empiricism, particularly the hypothetico-deductive account of theories, by philosophers such as Mary Hesse, sparked a renewed interest in the importance of models during the 1950s that continues to this day. Bailer-Jones analyzes subsequent propositions of: models as metaphors; Kuhn's concept of a paradigm; the Semantic View of theories; and the case study approaches of Cartwright and Morrison, among others. She then engages current debates on topics such as phenomena versus data, the distinctions between models and theories, the concepts of representation and realism, and the discerning of falsities in models. (shrink)

The replicability of experiment is routinely offered as the gold standard of evidence. I argue that it is not supported by a universal principle of replicability in inductive logic. A failure of replication may not impugn a credible experimental result; and a successful replication can fail to vindicate an incredible experimental result. Rather, employing a material approach to inductive inference, the evidential import of successful replication of an experiment is determined by the prevailing background facts. Commonly, these background facts do (...) support successful replication as a good evidential guide and this has fostered the illusion of a deeper, exceptionless principle. (shrink)

Using the Schwarzschild calculation of the Relativistic bending of starlight near the sun as an illustration, it is shown that the relationship between theory and data requires a hierarchy of structures of different logical type. An essential feature of this hierarchy is the use of idealizations and approximate truths. On the basis of a counterfactual analysis of these concepts, it is shown that confirmation is possible even though statistical measures of goodness of fit are not satisfied. The consequences of this (...) view of confirmation and hierarchical structure for scientific realism are then considered. (shrink)

This article develops a new, general account of replication. I argue that a replication is an experiment that resamples the experimental components of an ori...

This paper presents an analysis of the forms of response that scientists make when confronted with anomalous data. We postulate that there are seven ways in which an individual who currently holds a theory can respond to anomalous data: (1) ignore the data; (2) reject the data; (3) exclude the data from the domain of the current theory; (4) hold the data in abeyance; (5) reinterpret the data; (6) make peripheral changes to the current theory; or (7) change the theory. (...) We analyze psychological experiments and cases from the history of science to support this proposal. Implications for the philosophy of science are discussed. (shrink)

This paper offers an account of data manipulation in scientific experiments. It will be shown that in many cases raw, unprocessed data is not produced, but rather a form of processed data that will be referred to as a data model. The language of data models will be used to provide a framework within which to understand a recent debate about the status of data and data manipulation. It will be seen that a description in terms of data models allows (...) one to understand cases in which data acquisition and data manipulation cannot be separated into two independent activities. (shrink)

Discussions about a replicability crisis in science have been driven by the normative claim that all of science should be replicable and the empirical claim that most of it isn’t. Recently, such crisis talk has been challenged by a new localism, which argues a) that serious problems with replicability are not a general occurrence in science and b) that replicability itself should not be treated as a universal standard. The goal of this article is to introduce this emerging strand of (...) the debate and to discuss some of its implications and limitations. I will in particular highlight the issue of demarcation that localist accounts have to address, i.e. the question of how we can distinguish replicable science from disciplines where replicability does not apply. (shrink)

Researchers display confirmation bias when they persevere by revising procedures until obtaining a theory-predicted result. This strategy produces findings that are overgeneralized in avoidable ways, and this in turn binders successful applications. (The 40-year history of an attitude-change phenomenon.