Theories of statistical testing may be seen as attempts to provide systematic means for evaluating scientific conjectures on the basis of incomplete or inaccurate observational data. The Neyman-Pearson Theory of Testing (NPT) has purported to provide an objective means for testing statistical hypotheses corresponding to scientific claims. Despite their widespread use in science, methods of NPT have themselves been accused of failing to be objective; and the purported objectivity of scientific claims based upon NPT has been called into question. The (...) purpose of this paper is first to clarify this question by examining the conceptions of (I) the function served by NPT in science, and (II) the requirements of an objective theory of statistics upon which attacks on NPT's objectivity are based. Our grounds for rejecting these conceptions suggest altered conceptions of (I) and (II) that might avoid such attacks. Second, we propose a reformulation of NPT, denoted by NPT*, based on these altered conceptions, and argue that it provides an objective theory of statistics. The crux of our argument is that by being able to objectively control error frequencies NPT* is able to objectively evaluate what has or has not been learned from the result of a statistical test. (shrink)

While orthodox (Neyman-Pearson) statistical tests enjoy widespread use in science, the philosophical controversy over their appropriateness for obtaining scientific knowledge remains unresolved. I shall suggest an explanation and a resolution of this controversy. The source of the controversy, I argue, is that orthodox tests are typically interpreted as rules for making optimal decisions as to how to behave--where optimality is measured by the frequency of errors the test would commit in a long series of trials. Most philosophers of statistics, however, (...) view the task of statistical methods as providing appropriate measures of the evidential-strength that data affords hypotheses. Since tests appropriate for the behavioral-decision task fail to provide measures of evidential-strength, philosophers of statistics claim the use of orthodox tests in science is misleading and unjustified. What critics of orthodox tests overlook, I argue, is that the primary function of statistical tests in science is neither to decide how to behave nor to assign measures of evidential strength to hypotheses. Rather, tests provide a tool for using incomplete data to learn about the process that generated it. This they do, I show, by providing a standard for distinguishing differences (between observed and hypothesized results) due to accidental or trivial errors from those due to systematic or substantively important discrepancies. I propose a reinterpretation of a commonly used orthodox test to make this learning model of tests explicit. (shrink)

The orthodox view in statistics has it that frequentism and Bayesianism are diametrically opposed—two totally incompatible takes on the problem of statistical inference. This paper argues to the contrary that the two approaches are complementary and need to mesh if probabilistic reasoning is to be carried out correctly.

In guarding against inferential mistakes, experimental comparative cognition errs on the side of underattributing sophisticated cognition to animals, or what I refer to as the underattribution bias. I propose eliminating this bias by altering the method of choosing the default, or null, hypothesis. Rather than choosing the most parsimonious null hypothesis, as is current practice, I argue for choosing the best-evidenced hypothesis. Doing so at once preserves the risk-controlling structure of the current statistical paradigm and introduces a sensitivity to probability-conferring (...) empirical and theoretical information. This analysis illustrates how values like parsimony can covertly shape statistical-experimental design and inference. (shrink)

On pp. 55–58 of Philosophical Problems of Statistical Inference, I argue that in light of unsatisfactory after-trial properties of “best” Neyman-Pearson confidence intervals, we can strengthen a traditional criticism of the orthodox N-P theory. The criticism is that, once particular data become available, we see that the pre-trial concern for tests of maximum power may then misrepresent the conclusion of such a test. Specifically, I offer a statistical example where there exists a Uniformly Most Powerful test, a test of highest (...) N-P credentials, which generates a system of “best” confidence intervals with exact confidence coefficients. But the [CIλ] intervals have the unsatisfactory feature that, for a recognizable set of outcomes, the interval estimates cover all parameter values consistent with the data, at strictly less than 100% confidence. (shrink)