While many philosophers of science have accorded special evidential significance to tests whose results are "novel facts", there continues to be disagreement over both the definition of novelty and why it should matter. The view of novelty favored by Giere, Lakatos, Worrall and many others is that of use-novelty: An accordance between evidence e and hypothesis h provides a genuine test of h only if e is not used in h's construction. I argue that what lies behind the intuition that (...) novelty matters is the deeper intuition that severe tests matter. I set out a criterion of severity akin to the notion of a test's power in Neyman-Pearson statistics. I argue that tests which are use-novel may fail to be severe, and tests that are severe may fail to be use-novel. I discuss the 1919 eclipse data as a severe test of Einstein's law of gravity. (shrink)

I document some of the main evidence showing that E. S. Pearson rejected the key features of the behavioral-decision philosophy that became associated with the Neyman-Pearson Theory of statistics (NPT). I argue that NPT principles arose not out of behavioral aims, where the concern is solely with behaving correctly sufficiently often in some long run, but out of the epistemological aim of learning about causes of experimental results (e.g., distinguishing genuine from spurious effects). The view Pearson did hold gives a (...) deeper understanding of NPT tests than their typical formulation as accept-reject routines, against which criticisms of NPT are really directed. The Pearsonian view that emerges suggests how NPT tests may avoid these criticisms while still retaining what is central to these methods: the control of error probabilities. (shrink)

While orthodox (Neyman-Pearson) statistical tests enjoy widespread use in science, the philosophical controversy over their appropriateness for obtaining scientific knowledge remains unresolved. I shall suggest an explanation and a resolution of this controversy. The source of the controversy, I argue, is that orthodox tests are typically interpreted as rules for making optimal decisions as to how to behave--where optimality is measured by the frequency of errors the test would commit in a long series of trials. Most philosophers of statistics, however, (...) view the task of statistical methods as providing appropriate measures of the evidential-strength that data affords hypotheses. Since tests appropriate for the behavioral-decision task fail to provide measures of evidential-strength, philosophers of statistics claim the use of orthodox tests in science is misleading and unjustified. What critics of orthodox tests overlook, I argue, is that the primary function of statistical tests in science is neither to decide how to behave nor to assign measures of evidential strength to hypotheses. Rather, tests provide a tool for using incomplete data to learn about the process that generated it. This they do, I show, by providing a standard for distinguishing differences (between observed and hypothesized results) due to accidental or trivial errors from those due to systematic or substantively important discrepancies. I propose a reinterpretation of a commonly used orthodox test to make this learning model of tests explicit. (shrink)

Even those generally skeptical of propensity interpretations of probability must now grant the following two points. First, the above single-case propensity interpretation meets recognized formal conditions for being a genuine interpretation of probability. Second, this interpretation is not logically reducible to a hypothetical relative frequency interpretation, nor is it only vacuously different from such an interpretation.The main objection to this propensity interpretation must be not that it is too vague or vacuous, but that it is metaphysically too extravagant. It asserts (...) not only that there are physical possibilities in nature, but further that nature itself contains innate tendencies toward these possibilities, tendencies which have the logical structure of probabilities. Thus the basic dispute between advocates of an actualist relative frequency interpretation and a single-case propensity interpretation is not a matter of epistemology, but metaphysics. The frequency theorist wishes to maintain that claims about physical probabilities are nothing more than claims about relative frequencies that will occur in the actual history of the world, be it infinite or no. It is a substantial, though hardly conclusive, argument for the propensity view that the mathematical structures commonly employed in studies of stochastic processes and statistical inference are richer than can be accommodated by a relative frequency interpretation. Whether it is possible to bridge this gap without going beyond an actualist metaphysics remains to be seen.38. (shrink)

We analyse the issue of using prior information in frequentist statistical inference. For that purpose, we scrutinise different kinds of sampling designs in Jerzy Neyman’s theory to reveal a variety of ways to explicitly and objectively engage with prior information. Further, we turn to the debate on sampling paradigms (design-based vs. model-based approaches) to argue that Neyman’s theory supports an argument for the intermediate approach in the frequentism vs. Bayesianism debate. We also demonstrate that Neyman’s theory, by allowing non-epistemic values (...) to influence evidence collection and formulation of statistical conclusions, does not compromise the epistemic reliability of the procedures and may improve it. This undermines the value-free ideal of scientific inference. (shrink)