This book consists of a collection of essays written between 1965 and 1981. Some have been published elsewhere; others appear here for the first time. Although dealing with different figures and different periods, they have a common theme: all are concerned with examining how the method of hy pothesis came to be the ruling orthodoxy in the philosophy of science and the quasi-official methodology of the scientific community. It might have been otherwise. Barely three centuries ago, hypothetico deduction was in (...) both disfavor and disarray. Numerous rival methods for scientific inquiry - including eliminative and enumerative induction, analogy and derivation from first principles - were widely touted. The method of hypothesis, known since antiquity, found few proponents between 1700 and 1850. During the last century, of course, that ordering has been inverted and - despite an almost universal acknowledgement of its weaknesses - the method of hypothesis (usually under such descriptions as 'hypothetico deduction' or 'conjectures and refutations') has become the orthodoxy of the 20th century. Behind the waxing and waning of the method of hypothesis, embedded within the vicissitudes of its fortunes, there is a fascinating story to be told. It is a story that forms an integral part of modern science and its philosophy. (shrink)

The use of power to infer null hypotheses from negative results has recently come under severe attack. In this article, I show that the power of a test can justify accepting the null hypothesis. This argument also gives us a new powerful reason for not treating p-values and power as measures of the strength of evidence.

Epistemic trust is crucial for science. This article aims to identify the kinds of assumptions that are involved in epistemic trust as it is required for the successful operation of science as a collective epistemic enterprise. The relevant kind of reliance should involve working from the assumption that the epistemic endeavors of others are appropriately geared towards the truth, but the exact content of this assumption is more difficult to analyze than it might appear. The root of the problem is (...) that methodological decisions in science typically involve a complex trade-off between the reliability of positive results, the reliability of negative results, and the investigation's power. Which balance between these is the ‘correct’ one can only be determined in light of an evaluation of the consequences of all the different possible outcomes of the inquiry. What it means for the investigation to be ‘appropriately geared towards the truth’ thus depends on certain value judgments. I conclude that in the optimal case, trusting someone in her capacity as an information provider also involves a reliance on her having the right attitude towards the possible consequences of her epistemic work. 1 Introduction2 Epistemic Reliance within the Sciences3 Methodological Conventionalism4 Trust in Science5 Conclusions. (shrink)

Robustness is a common platitude: hypotheses are better supported with evidence generated by multiple techniques that rely on different background assumptions. Robustness has been put to numerous epistemic tasks, including the demarcation of artifacts from real entities, countering the “experimenter’s regress,” and resolving evidential discordance. Despite the frequency of appeals to robustness, the notion itself has received scant critique. Arguments based on robustness can give incorrect conclusions. More worrying is that although robustness may be valuable in ideal evidential circumstances (i.e., (...) when evidence is concordant), often when a variety of evidence is available from multiple techniques, the evidence is discordant. †To contact the author, please write to: Jacob Stegenga, Department of Philosophy, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093; e‐mail: [email protected]. (shrink)

First published in 1949 expressly to introduce logical positivism to English speakers. Reichenbach, with Rudolph Carnap, founded logical positivism, a form of epistemofogy that privileged scientific over metaphysical truths.

The use of multiple means of determination to “triangulate” on the existence and character of a common phenomenon, object, or result has had a long tradition in science but has seldom been a matter of primary focus. As with many traditions, it is traceable to Aristotle, who valued having multiple explanations of a phenomenon, and it may also be involved in his distinction between special objects of sense and common sensibles. It is implicit though not emphasized in the distinction between (...) primary and secondary qualities from Galileo onward. It is arguably one of several conceptions involved in Whewell’s method of the “consilience of inductions” (Laudan 1971) and is to be found in several places in Peirce. (From M. Brewer and B. Collins, eds., (1981); Scientific Inquiry in the Social Sciences (a festschrift for Donald T. Campbell), San Francisco: Jossey-Bass, pp. 123–162.). (shrink)

This article develops a new, general account of replication. I argue that a replication is an experiment that resamples the experimental components of an ori...

Current debates about the replication crisis in psychology take it for granted that direct replication is valuable and focus their attention on questionable research practices in regard to statistical analyses. This paper takes a broader look at the notion of replication as such. It is argued that all experimentation/replication involves individuation judgments and that research in experimental psychology frequently turns on probing the adequacy of such judgments. In this vein, I highlight the ubiquity of conceptual and material questions in research, (...) and I argue that replication is not as central to psychological research as it is sometimes taken to be. (shrink)

A heated debate surrounds the significance of reproducibility as an indicator for research quality and reliability, with many commentators linking a "crisis of reproducibility" to the rise of fraudulent, careless and unreliable practices of knowledge production. Through the analysis of discourse and practices across research fields, I point out that reproducibility is not only interpreted in different ways, but also serves a variety of epistemic functions depending on the research at hand. Given such variation, I argue that the uncritical pursuit (...) of reproducibility as an overarching epistemic value is misleading and potentially damaging to scientific advancement. Requirements for reproducibility, however they are interpreted, are one of many available means to secure reliable research outcomes. Furthermore, there are cases where the focus on enhancing reproducibility turns out not to foster high-quality research. Scientific communities and Open Science advocates should learn from inferential reasoning from irreproducible data, and promote incentives for all researchers to explicitly and publicly discuss their methodological commitments, the ways in which they learn from mistakes and problems in everyday practice, and the strategies they use to choose which research component of any project needs to be preserved in the long term, and how. (shrink)

Responding to recent concerns about the reliability of the published literature in psychology and other disciplines, we formed the X-Phi Replicability Project to estimate the reproducibility of experimental philosophy. Drawing on a representative sample of 40 x-phi studies published between 2003 and 2015, we enlisted 20 research teams across 8 countries to conduct a high-quality replication of each study in order to compare the results to the original published findings. We found that x-phi studies – as represented in our sample (...) – successfully replicated about 70% of the time. We discuss possible reasons for this relatively high replication rate in the field of experimental philosophy and offer suggestions for best research practices going forward. (shrink)

The reward system of science is the priority rule. The first scientist making a new discovery is rewarded with prestige, while second runners get little or nothing. Michael Strevens, following Philip Kitcher, defends this reward system, arguing that it incentivizes an efficient division of cognitive labor. I argue that this assessment depends on strong implicit assumptions about the replicability of findings. I question these assumptions on the basis of metascientific evidence and argue that the priority rule systematically discourages replication. My (...) analysis leads us to qualify Kitcher and Strevens’s contention that a priority-based reward system is normatively desirable for science. (shrink)

Advocates of the self-corrective thesis argue that scientific method will refute false theories and find closer approximations to the truth in the long run. I discuss a contemporary interpretation of this thesis in terms of frequentist statistics in the context of the behavioral sciences. First, I identify experimental replications and systematic aggregation of evidence (meta-analysis) as the self-corrective mechanism. Then, I present a computer simulation study of scientific communities that implement this mechanism to argue that frequentist statistics may converge upon (...) a correct estimate or not depending on the social structure of the community that uses it. Based on this study, I argue that methodological explanations of the “replicability crisis” in psychology are limited and propose an alternative explanation in terms of biases. Finally, I conclude suggesting that scientific self-correction should be understood as an interaction effect between inference methods and social structures. (shrink)

Offers a new approach to a number of central issues concerning the theoretical interpretation and normative evaluation of contemporary science and technology.

Varied evidence confirms more strongly than less varied evidence, ceteris paribus. This epistemological Variety of Evidence Thesis enjoys widespread intuitive support. We put forward a novel explication of one notion of varied evidence and the Variety of Evidence Thesis within Bayesian models of scientific inference by appealing to measures of entropy. Our explication of the Variety of Evidence Thesis holds in many of our models which also pronounce on disconfirmatory and discordant evidence. We argue that our models pronounce rightly. Against (...) a backdrop of failures of the Variety of Evidence Thesis, the intuitive case for the Variety of Evidence Thesis emerges strengthened. Our models do however not support the general case for the thesis since our explication of it fails to hold in certain cases. The parameter space of this failure is explored and an explanation for the failure is offered. (shrink)

Benjamin et al. Nature Human Behavior 2, 6–10 proposed decreasing the significance level by an order of magnitude to improve the replicability of psychology. This modest, practical proposal has been widely criticized, and its prospects remain unclear. This article defends this proposal against these criticisms and highlights its virtues.

. Scientists, for the most part, want to get it right. However, the social structures that govern their work undermine that aim, and this leads to nonreplicable findings in many fields. Because the social structure of science is a decentralized system, it is difficult to intervene. In this article, I discuss how we might do so, focusing on self-corrective-labor schemes. First, I argue that we need to implement a scheme that makes replication work outcome independent, systematic, and sustainable. Second, I (...) use these three criteria to evaluate extant proposals, which place the responsibility for replication on original researchers, consumers of their research, students, or many labs. Third, on the basis of a philosophical analysis of the reward system of science and the benefits of the division of cognitive labor, I propose a scheme that satisfies the criteria better: the professional scheme. This scheme has two main components. First, the scientific community is organized into two groups: discovery researchers, who produce new findings, and confirmation researchers, whose primary function is to do confirmation work. Second, a distinct reward system is established for confirmation researchers so that their career advancement is separated from whether they obtain positive experimental results. (shrink)

Open Science policies encourage researchers to disclose a wide range of outputs from their work, thus codifying openness as a specific set of research practices and guidelines that can be interpreted and applied consistently across disciplines and geographical settings. In this paper, we argue that this “one-size-fits-all” view of openness sidesteps key questions about the forms, implications, and goals of openness for research practice. We propose instead to interpret openness as a dynamic and highly situated mode of valuing the research (...) process and its outputs, which encompasses economic as well as scientific, cultural, political, ethical, and social considerations. This interpretation creates a critical space for moving beyond the economic definitions of value embedded in the contemporary biosciences landscape and Open Science policies, and examining the diversity of interests and commitments that affect research practices in the life sciences. To illustrate these claims, we use three case studies that highlight the challenges surrounding decisions about how––and how best––to make things open. These cases, drawn from ethnographic engagement with Open Science debates and semistructured interviews carried out with UK-based biologists and bioinformaticians between 2013 and 2014, show how the enactment of openness reveals judgments about what constitutes a legitimate intellectual contribution, for whom, and with what implications. (shrink)

The enduring replication crisis in many scientific disciplines casts doubt on the ability of science to estimate effect sizes accurately, and in a wider sense, to self-correct its findings and to produce reliable knowledge. We investigate the merits of a particular countermeasure—replacing null hypothesis significance testing with Bayesian inference—in the context of the meta-analytic aggregation of effect sizes. In particular, we elaborate on the advantages of this Bayesian reform proposal under conditions of publication bias and other methodological imperfections that are (...) typical of experimental research in the behavioral sciences. Moving to Bayesian statistics would not solve the replication crisis single-handedly. However, the move would eliminate important sources of effect size overestimation for the conditions we study. (shrink)

The enduring replication crisis in many scientific disciplines casts doubt on the ability of science to estimate effect sizes accurately, and in a wider sense, to self-correct its findings and to produce reliable knowledge. We investigate the merits of a particular countermeasure—replacing null hypothesis significance testing with Bayesian inference—in the context of the meta-analytic aggregation of effect sizes. In particular, we elaborate on the advantages of this Bayesian reform proposal under conditions of publication bias and other methodological imperfections that are (...) typical of experimental research in the behavioral sciences. Moving to Bayesian statistics would not solve the replication crisis single-handedly. However, the move would eliminate important sources of effect size overestimation for the conditions we study. (shrink)

It is great to see that, after all, there are some books that recognize technology as a topic which might have some impact on philosophical discourse. Even the importance of real experiments was not appreciated in philosophy of science until recently, except as means to support or refute scientific theories. In this book, Hans Radder present a new kind of “referential realism” which believes that the work of philosophy of science can be ‘In and About’ the world.