Prediction in Science

Many philosophers have argued that a hypothesis is better confirmed by some data if the hypothesis was not specifically designed to fit the data. ‘Prediction’, they argue, is superior to ‘accommodation’. Others deny that there is any epistemic advantage to prediction, and conclude that prediction and accommodation are epistemically on a par. This paper argues that there is a respect in which accommodation is superior to prediction. Specifically, the information that the data was accommodated rather than predicted suggests that the (...) data is less likely to have been manipulated or fabricated, which in turn increases the likelihood that the hypothesis is correct in light of the data. In some cases, this epistemic advantage of accommodation may even outweigh whatever epistemic advantage there might be to prediction, making accommodation epistemically superior to prediction all things considered. (shrink)

This note contains a corrective and a generalization of results by Borsboom et al. (2008), based on Heesen and Romeijn (2019). It highlights the relevance of insights from psychometrics beyond the context of psychological testing.

A self-fulfilling prophecy is, roughly, a prediction that brings about its own truth. Although true predictions are hard to fault, self-fulfilling prophecies are often regarded with suspicion. In this article, we vindicate this suspicion by explaining what self-fulfilling prophecies are and what is problematic about them, paying special attention to how their problems are exacerbated through automated prediction. Our descriptive account of self-fulfilling prophecies articulates the four elements that define them. Based on this account, we begin our critique by showing (...) that typical self-fulfilling prophecies arise due to mistakes about the relationship between a prediction and its object. Such mistakes—along with other mistakes in predicting or in the larger practical endeavor—are easily overlooked when the predictions turn out true. Thus we note that self-fulfilling prophecies prompt no error signals; truth shrouds their mistakes from humans and machines alike. Consequently, self-fulfilling prophecies create several obstacles to accountability for the outcomes they produce. We conclude our critique by showing how failures of accountability, and the associated failures to make corrections, explain the connection between self-fulfilling prophecies and feedback loops. By analyzing the complex relationships between accuracy and other evaluatively significant features of predictions, this article sheds light both on the special case of self-fulfilling prophecies and on the ethics of prediction more generally. (shrink)

Based on a review of several “anomalies” in research using implicit measures, Machery (2021) dismisses the modal interpretation of participant responses on implicit measures and, by extension, the value of implicit measures. We argue that the reviewed findings are anomalies only for specific—influential but long-contested—accounts that treat responses on implicit measures as uncontaminated indicators of trait-like unconscious representations that coexist with functionally independent conscious representations. However, the reviewed findings are to-be-expected “normalities” when viewed from the perspective of long-standing alternative frameworks (...) that treat responses on implicit measures as the product of dynamic processes that operate on momentarily activated, consciously accessible information. Thus, although we agree with Machery that the modal view is empirically unsupported, we argue that implicit measures can make a valuable contribution to understanding the complexities of human behavior if they are used wisely in a way that acknowledges what they can and cannot do. (shrink)

Neuroscientists have in recent years turned to building models that aim to generate predictions rather than explanations. This “predictive turn” has swept across domains including law, marketing, and neuropsychiatry. Yet the norms of prediction remain undertheorized relative to those of explanation. I examine two styles of predictive modeling and show how they exemplify the normative dynamics at work in prediction. I propose an account of how predictive models, conceived of as technological devices for aiding decision-making, can come to be adequate (...) for purposes that are defined by both their guiding research questions and their larger social context of application. (shrink)

In this paper, I critically evaluate several related, provocative claims made by proponents of data-intensive science and “Big Data” which bear on scientific methodology, especially the claim that scientists will soon no longer have any use for familiar concepts like causation and explanation. After introducing the issue, in Section 2, I elaborate on the alleged changes to scientific method that feature prominently in discussions of Big Data. In Section 3, I argue that these methodological claims are in tension with a (...) prominent account of scientific method, often called “Inference to the Best Explanation”. Later on, in Section 3, I consider an argument against IBE that will be congenial to proponents of Big Data, namely, the argument due to Roche and Sober Analysis, 73:659–668, that “explanatoriness is evidentially irrelevant.” This argument is based on Bayesianism, one of the most prominent general accounts of theory-confirmation. In Section 4, I consider some extant responses to this argument, especially that of Climenhaga Philosophy of Science, 84:359–368,. In Section 5, I argue that Roche and Sober’s argument does not show that explanatory reasoning is dispensable. In Section 6, I argue that there is good reason to think explanatory reasoning will continue to prove indispensable in scientific practice. Drawing on Cicero’s oft-neglected De Divinatione, I formulate what I call the “Ciceronian Causal-nomological Requirement”, which states, roughly, that causal-nomological knowledge is essential for relying on correlations in predictive inference. I defend a version of the CCR by appealing to the challenge of “spurious correlations,” chance correlations which we should not rely upon for predictive inference. In Section 7, I offer some concluding remarks. (shrink)

A review of Gary Smith and Jay Cordes: The Phantom Pattern Problem: The Mirage of Big Data. New York: Oxford University Press, 2020.

In this paper, I examine Cicero’s oft-neglected De Divinatione, a dialogue investigating the legitimacy of the practice of divination. First, I offer a novel analysis of the main arguments for divination given by Quintus, highlighting the fact that he employs two logically distinct argument forms. Next, I turn to the first of the main arguments against divination given by Marcus. Here I show, with the help of modern probabilistic tools, that Marcus’ skeptical response is far from the decisive, proto-naturalistic assault (...) on superstition that it is sometimes portrayed to be. Then, I offer an extended analysis of the second of the main arguments against divination given by Marcus. Inspired by Marcus’ second main argument, I formulate, explicate, and defend a substantive principle of scientific methodology that I call the “Ciceronian Causal-Nomological Requirement” (CCR). Roughly, this principle states that causal knowledge is essential for relying on correlations in predictive inference. Although I go on to argue that Marcus’ application of the CCR in his debate with Quintus is dialectically inadequate, I conclude that De Divinatione deserves its place in Cicero’s philosophical corpus, and that ultimately, its significance for the history and philosophy of science ought to be recognized. (shrink)

We identify several ongoing debates related to implicit measures, surveying prominent views and considerations in each debate. First, we summarize the debate regarding whether performance on implicit measures is explained by conscious or unconscious representations. Second, we discuss the cognitive structure of the operative constructs: are they associatively or propositionally structured? Third, we review debates whether performance on implicit measures reflects traits or states. Fourth, we discuss the question of whether a person’s performance on an implicit measure reflects characteristics of (...) the person who is taking the test or characteristics of the situation in which the person is taking the test. Finally, we survey the debate about the relationship between implicit measures and (other kinds of) behavior. (shrink)

Several authors have claimed that prediction is essentially impossible in the general theory of relativity, the case being particularly strong, it is said, when one fully considers the epistemic predicament of the observer. Each of these claims rests on the support of an underdetermination argument and a particular interpretation of the concept of prediction. I argue that these underdetermination arguments fail and depend on an implausible explication of prediction in the theory. The technical results adduced in these arguments can be (...) related to certain epistemic issues, but can only be misleadingly or mistakenly characterized as related to prediction. (shrink)

Hypothesizing after the results are known, or HARKing, occurs when researchers check their research results and then add or remove hypotheses on the basis of those results without acknowledging this process in their research report (Kerr, 1998). In the present article, I discuss three forms of HARKing: (1) using current results to construct post hoc hypotheses that are then reported as if they were a priori hypotheses; (2) retrieving hypotheses from a post hoc literature search and reporting them as a (...) priori hypotheses; and (3) failing to report a priori hypotheses that are unsupported by the current results. These three types of HARKing are often characterized as being bad for science and a potential cause of the current replication crisis. In the present article, I use insights from the philosophy of science to present a more nuanced view. Specifically, I identify the conditions under which each of these three types of HARKing is most and least likely to be bad for science. I conclude with a brief discussion about the ethics of each type of HARKing. (shrink)

In their critique of Klein (2014a), Trafimow and Earp present two theses. First, they argue that, contra Klein, a well-specified theory is not a necessary condition for successful replication. Second, they contend that even when there is a well-specified theory, replication depends more on auxiliary assumptions than on theory proper. I take issue with both claims, arguing that (a) their first thesis confuses a material conditional (what I said) with a modal claim (T&E’s misreading of what I said), and (b) (...) their second thesis has the unfortunate consequence of refuting their first thesis. (shrink)

From the early-1950s on, F.A. Hayek was concerned with the development of a methodology of sciences that study systems of complex phenomena. Hayek argued that the knowledge that can be acquired about such systems is, in virtue of their complexity (and the comparatively narrow boundaries of human cognitive faculties), relatively limited. The paper aims to elucidate the implications of Hayek’s methodology with respect to the specific dimensions along which the scientist’s knowledge of some complex phenomena may be limited. Hayek’s fallibilism (...) was an essential (if not always explicit) aspect of his arguments against the defenders of both socialism ([1935] 1948, [1940] 1948) and countercyclical monetary policy ([1975] 1978); yet, despite the fact that his conceptions of both complex phenomena and the methodology appropriate to their investigation imply that ignorance might beset the scientist in multiple respects, he never explicated all of these consequences. The specificity of a scientific prediction depends on the extent of the scientist’s knowledge concerning the phenomena under investigation. The paper offers an account of the considerations that determine the extent to which a theory’s implications prohibit the occurrence of particular events in the relevant domain. This theory of “predictive degree” both expresses and – as the phenomena of scientific prediction are themselves complex in Hayek’s sense – exemplifies the intuition that the specificity of a scientific prediction depends on the relevant knowledge available. (shrink)

There is a middle ground of imperfect knowledge in fields like medicine and the social sciences. It stands between our day-to-day relatively certain knowledge obtained from ordinary basic observation of regularities in our world and our knowledge from well-validated theories in the physical sciences. -/- The latter enable reliable prediction a great deal of the time of the happening of events never before experienced. The former enable prediction only of what has happened before and beyond that of educated guesses which (...) may sometimes prove right and others not when we test them. -/- The imperfection of our knowledge between those limits is a consequence of complexity. -/- Reductionist empiricism fails when faced with complexity but we all still have to live in a complex world where reductionist science cannot help us devise reliable theories from which we can predict the behaviour of the world around us. We cannot predict reliably. We can only monitor actively and manage actively the world around us if we want to try to control our environment. We have a limited prediction horizon. -/- Science and empiricism work well if we want to send Voyager I and II over 3 billion miles away but not for most other things. -/- In medicine how and when to apply probabilistic, conjectural, incomplete medical theories and explanations requires professional expertise, intuition and judgement (non scientific knowledge) which is essential. Medical diagnosis is a skill of predicting from knowledge of what has happened before that the past will recur in the current patient applying expertise and intuition from knowledge and experience of prior cases and probabilistic medical research and theory. The physician is left to an educated guess as to how the future will develop for the particular individual patient the focus of his or her current attention. (shrink)

There is considerable disagreement about the epistemic value of novel predictive success, i.e. when a scientist predicts an unexpected phenomenon, experiments are conducted, and the prediction proves to be accurate. We survey the field on this question, noting both fully articulated views such as weak and strong predictivism, and more nascent views, such as pluralist reasons for the instrumental value of prediction. By examining the various reasons offered for the value of prediction across a range of inferential contexts , we (...) can see that neither weak nor strong predictivism captures all of the reasons for valuing prediction available. A third path is presented, Pluralist Instrumental Predictivism; PIP for short. (shrink)

It is taken for granted that the explanation of the Universe’s space-time dimension belongs to the host of the arguments that exhibit the superiority of modern (inflationary) cosmology over the standard model. In the present paper some doubts are expressed . They are based upon the fact superstring theory is too formal to represent genuine unification of general relativity and quantum field theory. Neveretheless, the fact cannot exclude the opportunity that in future the superstring theory can become more physical. Hence (...) this paper does not aim to query neither string cosmology, nor superstring theory; it asks for “tolerance in the matters cosmological”. It advices the researchers not to dwell on the common way of unification and to take into consideration the other ways as well. (shrink)

The thesis of theory-ladenness of observations, in its various guises, is widely considered as either ill-conceived or harmless to the rationality of science. The latter view rests partly on the work of the proponents of New Experimentalism who have argued, among other things, that experimental practices are efficient in guarding against any epistemological threat posed by theory-ladenness. In this paper I show that one can generate a thesis of theory-ladenness for experimental practices from an influential New Experimentalist account. The notion (...) I introduce for this purpose is the concept of ‘theory-driven data reliability judgments’, according to which theories which are sought to be tested with a particular set of data guide reliability judgments about those very same data. I provide various prominent historical examples to show that TDRs are used by scientists to resolve data conflicts. I argue that the rationality of the practices which employ TDRs can be saved if the independent support of the theories driving TDRs is construed in a particular way. (shrink)

There are different kinds of uncertainty. I outline some of the various ways that uncertainty enters science, focusing on uncertainty in climate science and weather prediction. I then show how we cope with some of these sources of error through sophisticated modelling techniques. I show how we maintain confidence in the face of error.

Philosophy can shed light on mathematical modeling and the juxtaposition of modeling and empirical data. This paper explores three philosophical traditions of the structure of scientific theory—Syntactic, Semantic, and Pragmatic—to show that each illuminates mathematical modeling. The Pragmatic View identifies four critical functions of mathematical modeling: (1) unification of both models and data, (2) model fitting to data, (3) mechanism identification accounting for observation, and (4) prediction of future observations. Such facets are explored using a recent exchange between two groups (...) of mathematical modelers in plant biology. Scientific debate can arise from different modeling philosophies. (shrink)

WE AIM HERE to outline a theory of evidence for use. More specifically we lay foundations for a guide for the use of evidence in predicting policy effectiveness in situ, a more comprehensive guide than current standard offerings, such as the Maryland rules in criminology, the weight of evidence scheme of the International Agency for Research on Cancer (IARC), or the US ‘What Works Clearinghouse’. The guide itself is meant to be well-grounded but at the same time to give practicable (...) advice, that is, advice that can be used by policy-makers not expert in the natural and social sciences, assuming they are well-intentioned and have a reasonable but limited amount of time and resources available for searching out evidence and deliberating. (shrink)

‘The problem with simulations is that they are doomed to succeed.’ So runs a common criticism of simulations—that they can be used to ‘prove’ anything and are thus of little or no scientific value. While this particular objection represents a minority view, especially among those who work with simulations in a scientific context, it raises a difficult question: what standards should we use to differentiate a simulation that fails from one that succeeds? In this paper we build on a structural (...) analysis of simulation developed in previous work to provide an evaluative account of the variety of ways in which simulations do fail. We expand the structural analysis in terms of the relationship between a simulation and its real-world target emphasizing the important role of aspects intended to correspond and also those specifically intended not to correspond to reality. The result is an outline both of the ways in which simulations can fail and the scientific importance of those various forms of failure. (shrink)

Even though social scientists continue to discuss the problems posed by self-fulfilling and self-frustrating predictions, philosophers of science have ignored the topic since the 1970s. Back then, the prevailing view was that the methodological problems posed by reflexive predictions are either minor or easily avoided. I believe that this consensus was premature, ultimately relying on an overly narrow understanding of the phenomenon. I present an improved way to understand reflexive predictions (framed in probabilistic terms) and show that, once such predictions (...) are understood this way, the methodological problems they pose may turn out to be neither minor nor easily avoided. (shrink)

Selectionist evolutionary theory has often been faulted for not making novel predictions that are surprising, risky, and correct. I argue that it in fact exhibits the theoretical virtue of predictive capacity in addition to two other virtues: explanatory unification and model fitting. Two case studies show the predictive capacity of selectionist evolutionary theory: parallel evolutionary change in E. coli, and the origin of eukaryotic cells through endosymbiosis.

Is simulation some new kind of science? We argue that instead simulation fits smoothly into existing scientific practice, but does so in several importantly different ways. Simulations in general, and computer simulations in particular, ought to be understood as techniques which, like many scientific techniques, can be employed in the service of various and diverse epistemic goals. We focus our attentions on the way in which simulations can function as (i) explanatory and (ii) predictive tools. We argue that a wide (...) variety of simulations, both computational and physical, are best conceived in terms of a set of common features: initial or input conditions, a mechanism or set of rules, and a set of results or output conditions. Studying simulations in these terms yields a new understanding of their character as well as a body of normative recommendations for the care and feeding of scientific simulations. (shrink)

_ Source: _Volume 5, Issue 2, pp 228 - 250 Despite inroads made by critical realism against the ‘scientific method’ in social science, the latter remains strong in subject-areas like human resource management. One argument for the alleged superiority of the scientific method lies in the taken-for-granted belief that it alone can formulate empirically testable predictions. Many of those who employ the scientific method are, however, confused about the way they understand and practice prediction. This paper takes as a case (...) study empirical research on the alleged empirical association between human resource management practices and organisational performance. By unpacking the confusion surrounding the two basic notions of prediction used, it reveals what is wrong with them, why the scientific method cannot actually make accurate predictions and why, therefore, the scientific method fails to meet its own criteria for scientificity. Finally, explanation is considered in order to prevent any confusion between it and prediction and to offer what we call _tendential prediction_. (shrink)

Our aim in this paper is to bring the woefully neglected literature on predictive modeling to bear on some central questions in the philosophy of science. The lesson of this literature is straightforward: For a very wide range of prediction problems, statistical prediction rules (SPRs), often rules that are very easy to implement, make predictions than are as reliable as, and typically more reliable than, human experts. We will argue that the success of SPRs forces us to reconsider our views (...) about what is involved in understanding, explanation, good reasoning, and about how we ought to do philosophy of science. (shrink)

First published in 1949 expressly to introduce logical positivism to English speakers. Reichenbach, with Rudolph Carnap, founded logical positivism, a form of epistemofogy that privileged scientific over metaphysical truths.