Hypothesizing after the results are known, or HARKing, occurs when researchers check their research results and then add or remove hypotheses on the basis of those results without acknowledging this process in their research report (Kerr, 1998). In the present article, I discuss three forms of HARKing: (1) using current results to construct post hoc hypotheses that are then reported as if they were a priori hypotheses; (2) retrieving hypotheses from a post hoc literature search and reporting them as a (...) priori hypotheses; and (3) failing to report a priori hypotheses that are unsupported by the current results. These three types of HARKing are often characterized as being bad for science and a potential cause of the current replication crisis. In the present article, I use insights from the philosophy of science to present a more nuanced view. Specifically, I identify the conditions under which each of these three types of HARKing is most and least likely to be bad for science. I conclude with a brief discussion about the ethics of each type of HARKing. (shrink)

Once upon a time, logic was the philosopher’s tool for analyzing scientific reasoning. Nowadays, probability and statistics have largely replaced logic, and their most popular application—Bayesianism—has replaced the qualitative deductive relationship between a hypothesis h and evidence e with a quantitative measure of h’s probability in light of e.

Few can imagine a world without telephones or televisions; many depend on computers and the Internet as part of daily life. Without scientific theory, these developments would not have been possible. In this exceptionally clear and engaging introduction to philosophy of science, James Ladyman explores the philosophical questions that arise when we reflect on the nature of the scientific method and the knowledge it produces. He discusses whether fundamental philosophical questions about knowledge and reality might be answered by science, and (...) considers in detail the debate between realists and antirealists about the extent of scientific knowledge. Along the way, central topics in philosophy of science, such as the demarcation of science from non-science, induction, confirmation and falsification, the relationship between theory and observation and relativism are all addressed. Important and complex current debates over underdetermination, inference to the best explaination and the implications of radical theory change are clarified and clearly explained for those new to the subject. (shrink)

Scientific reasoning is—and ought to be—conducted in accordance with the axioms of probability. This Bayesian view—so called because of the central role it accords to a theorem first proved by Thomas Bayes in the late eighteenth ...

The paper presents a further articulation and defence of the view on prediction and accommodation that I have proposed earlier. It operates by analysing two accounts of the issue-by Patrick Maher and by Marc Lange-that, at least at first sight, appear to be rivals to my own. Maher claims that the time-order of theory and evidence may be important in terms of degree of confirmation, while that claim is explicitly denied in my account. I argue, however, that when his account (...) is analysed, Maher reveals no scientifically significant way in which the time-order counts, and that indeed his view is in the end best regarded as a less than optimally formulated version of my own. Lange has also responded to Maher by arguing that the apparent relevance of temporal considerations is merely apparent: what is really involved, according to Lange, is whether or not a hypothesis constitutes an "arbitrary conjunction." I argue that Lange's suggestion fails: the correct analysis of his and Maher's examples is that provided by my account. (shrink)

Popper formulates and explains his non-justificationist theory of knowledge. Science--empirical science--aims at true explanatory theories, yet it can never prove, finally establish, or justify any of its theories as true, not even if it is in fact a true theory. Science must continue to question and criticize all its theories, even those which happen to be true.

Realism and the Aim of Science is one of the three volumes of Karl Popper’s Postscript to the Logic of scientific Discovery. The Postscript is the culmination of Popper’s work in the philosophy of physics and a new famous attack on subjectivist approaches to philosophy of science. Realism and the Aim of Science is the first volume of the Postcript . Popper here formulates and explains his non-justificationist theory of knowledge: science aims at true explanatory theories, yet it can never (...) prove, or justify, any theory to be true, not even if is a true theory. Science must continue to question and criticise all its theories, even those that happen to be true. Realism and the Aim of Science presents Popper’s mature statement on scientific knowledge and offers important insights into his thinking on problems of method within science. (shrink)

Statistical procedures can be applied to episodes in the history of science in order to weight attributes to predict short-term survival of theories; an asymptotic method is used to show that short-term survival is a valid proxy for ultimate survival; and a theoretical argument is made that ultimate survival is a valid proxy for objective truth. While realists will appreciate this last step, instrumentalists do not need it to benefit from the actuarial procedures of cliometric metatheory. Introduction A plausible proxy (...) for Peircean consensus Assessing the validity of theory attributes as predictors of theory survival 3.1 Linear discriminant function 3.2 Factor analysis 3.3 Taxometric analysis Verisimilitude index Satisfying both instrumentalists and realists Recapitulation Implementation of cliometric metatheory * Correspondence about this article may be addressed to Leslie Yonce at pemeehle{at}umn.edu' + u + '@' + d + ''//--> This article had been completed by Paul Meehl at the time of his death on 14 February 2003. His wife, Leslie J. Yonce, is grateful to Keith Gunderson (University of Minnesota, Center for Philosophy of Science) and Niels G. Waller (Psychology Department, Vanderbilt University) for advice with some final editing details. (shrink)

an observation to formulate a theory, it is no surprise that the resulting theory accurately captures that observation. However, when the theory makes a novel prediction—when it predicts an observation that was not used in its formulation—this seems to provide more substantial confirmation of the theory. This paper presents a new approach to the vexed problem of understanding the epistemic difference between prediction and accommodation. In fact, there are several problems that need to be disentangled; in all of them, the (...) key is the concept of overfitting. We float the hypothesis that accommodation is a defective methodology only when the methods used to accommodate the data fail to guard against the risk of overfitting. We connect our analysis with the proposals that other philosophers have made. We also discuss its bearing on the conflict between instrumentalism and scientific realism. Introduction Predictivisms—a taxonomy Observations Formulating the problem What might Annie be doing wrong? Solutions Observations explained Mayo on severe tests The miracle argument and scientific realism Concluding comments. (shrink)

The issues of double-counting, use-constructing, and selection effects have long been the subject of debate in the philosophical as well as statistical literature. I have argued that it is the severity, stringency, or probativeness of the test—or lack of it—that should determine if a double-use of data is admissible. Hitchcock and Sober ([2004]) question whether this ‘severity criterion' can perform its intended job. I argue that their criticisms stem from a flawed interpretation of the severity criterion. Taking their criticism as (...) a springboard, I elucidate some of the central examples that have long been controversial, and clarify how the severity criterion is properly applied to them. Severity and Use-Constructing: Four Points (and Some Clarificatory Notes) 1.1 Point 1: Getting beyond ‘all or nothing’ standpoints 1.2 Point 2: The rationale for prohibiting double-counting is the requirement that tests be severe 1.3 Point 3: Evaluate severity of a test T by its associated construction rule R 1.4 Point 4: The ease of passing vs. ease of erroneous passing: Statistical vs. ‘Definitional’ probability The False Dilemma: Hitchcock and Sober 2.1 Marsha measures her desk reliably 2.2 A false dilemma Canonical Errors of Inference 3.1 How construction rules may alter the error-probing performance of tests 3.2 Rules for accounting for anomalies 3.3 Hunting for statistically significant differences Concluding Remarks. (shrink)

Popper’s original definition of truthlikeness relied on a central insight: that truthlikeness combines truth and information, in the sense that a proposition is closer to the truth the more true consequences and the less false consequences it entails. As intuitively compelling as this definition may be, it is untenable, as proved long ago; still, one can arguably rely on Popper’s intuition to provide an adequate account of truthlikeness. To this aim, we mobilize some classical work on partial entailment in defining (...) a new measure of truthlikeness which satisfies a number of desiderata. The resulting account has some interesting and surprising connections with other accounts on the market, thus shedding new light on current attempts of systematizing different approaches to verisimilitude. (shrink)

This article generalizes the explanationist account of inference to the best explanation. It draws a clear distinction between IBE and abduction and presents abduction as the first step of IBE. The second step amounts to the evaluation of explanatory power, which consist in the degree of explanatory virtues that a hypothesis exhibits. Moreover, even though coherence is the most often cited explanatory virtue, on pain of circularity, it should not be treated as one of the explanatory virtues. Rather, coherence should (...) be equated with explanatory power and considered to be derivable from the other explanatory virtues: unification, explanatory depth and simplicity. (shrink)

Data fraud and selective reporting both present serious threats to the credibility of science. However, there remains considerable disagreement among scientists about how best to sanction data fraud, and about the ethicality of selective reporting. The public is arguably the largest stakeholder in the reproducibility of science; research is primarily paid for with public funds, and flawed science threatens the public’s welfare. Members of the public are able to make meaningful judgments about the morality of different behaviors using moral intuitions. (...) Legal scholars emphasize that to maintain legitimacy, social control policies must be developed with some consideration given to the public’s moral intuitions. Although there is a large literature on popular attitudes toward science, there is no existing evidence about public opinion on data fraud or selective reporting. We conducted two studies—a survey experiment with a nationwide convenience sample, and a follow-up survey with a representative sample of US adults —to explore community members’ judgments about the morality of data fraud and selective reporting in science. The findings show that community members make a moral distinction between data fraud and selective reporting, but overwhelmingly judge both behaviors to be immoral and deserving of punishment. Community members believe that scientists who commit data fraud or selective reporting should be fired and banned from receiving funding. For data fraud, most Americans support criminal penalties. Results from an ordered logistic regression analysis reveal few demographic and no significant partisan differences in punitiveness toward data fraud. (shrink)