What has science actually achieved? A theory of achievement should define what has been achieved, describe the means or methods used in science, and explain how such methods lead to such achievements. Predictive accuracy is one truth-related achievement of science, and there is an explanation of why common scientific practices tend to increase predictive accuracy. Akaike's explanation for the success of AIC is limited to interpolative predictive accuracy. But therein lies the strength of the general framework, for it also provides (...) a clear formulation of many open problems of research. (shrink)

Within the context of psychological measurement, realist commitments pervade methodology. Further, there are instances where particular scientific practices and decisions are explicable most plausibly against a background assumption of epistemic realism. That psychometrics is a realist enterprise provides a possible toehold for Stephen Jay Gould’s objections to psychometrics in The Mismeasure of Man and Joel Michell’s charges that psychometrics is a “pathological science.” These objections do not withstand scrutiny. There are no fewer than three activities in ongoing psychometric research which (...) presuppose a commitment to a minimal epistemic realism. Those activities include selecting between different models for representing data, estimating ability in the context of item response theory, and the move to make the individual the fundamental unit of analysis in psychometrics thereby calling for a shift in what sorts of data are evidentially relevant. In none of these activities are the commitments and disregard for evidence that Gould and Michell find objectionable or “pathological.”. (shrink)

What has science actually achieved? A theory of achievement should define what has been achieved, describe the means or methods used in science, and explain how such methods lead to such achievements. Predictive accuracy is one truth‐related achievement of science, and there is an explanation of why common scientific practices tend to increase predictive accuracy. Akaike’s explanation for the success of AIC is limited to interpolative predictive accuracy. But therein lies the strength of the general framework, for it also provides (...) a clear formulation of many open problems of research. (shrink)

an observation to formulate a theory, it is no surprise that the resulting theory accurately captures that observation. However, when the theory makes a novel prediction—when it predicts an observation that was not used in its formulation—this seems to provide more substantial confirmation of the theory. This paper presents a new approach to the vexed problem of understanding the epistemic difference between prediction and accommodation. In fact, there are several problems that need to be disentangled; in all of them, the (...) key is the concept of overfitting. We float the hypothesis that accommodation is a defective methodology only when the methods used to accommodate the data fail to guard against the risk of overfitting. We connect our analysis with the proposals that other philosophers have made. We also discuss its bearing on the conflict between instrumentalism and scientific realism. Introduction Predictivisms—a taxonomy Observations Formulating the problem What might Annie be doing wrong? Solutions Observations explained Mayo on severe tests The miracle argument and scientific realism Concluding comments. (shrink)

According to the thesis of Strong Predictionism, we typically have stronger evidence for a theory if it was used to predict certain data, than if it was deliberately constructed to accommodate those same data, even if we fully grasp the theory and all the evidence on which it was based. This thesis faces powerful objections and the existing arguments in support of it are seriously flawed. I offer a new defence of Strong Predictionism which overcomes the objections and provides a (...) deeper understanding of the epistemic importance of prediction. I conclude by applying this account to strategies for defending scientific realism. (shrink)

Traditional analyses of the curve fitting problem maintain that the data do not indicate what form the fitted curve should take. Rather, this issue is said to be settled by prior probabilities, by simplicity, or by a background theory. In this paper, we describe a result due to Akaike [1973], which shows how the data can underwrite an inference concerning the curve's form based on an estimate of how predictively accurate it will be. We argue that this approach throws light (...) on the theoretical virtues of parsimoniousness, unification, and non ad hocness, on the dispute about Bayesianism, and on empiricism and scientific realism. * Both of us gratefully acknowledge support from the Graduate School at the University of Wisconsin-Madison, and NSF grant DIR-8822278 (M.F.) and NSF grant SBE-9212294 (E.S.). Special thanks go to A. W. F. Edwards.William Harper. Martin Leckey. Brian Skyrms, and especially Peter Turney for helpful comments on an earlier draft. (shrink)

The replicability of experiment is routinely offered as the gold standard of evidence. I argue that it is not supported by a universal principle of replicability in inductive logic. A failure of replication may not impugn a credible experimental result; and a successful replication can fail to vindicate an incredible experimental result. Rather, employing a material approach to inductive inference, the evidential import of successful replication of an experiment is determined by the prevailing background facts. Commonly, these background facts do (...) support successful replication as a good evidential guide and this has fostered the illusion of a deeper, exceptionless principle. (shrink)

We describe anytime search procedures that (1) find disjoint subsets of recorded variables for which the members of each subset are d-separated by a single common unrecorded cause, if such exists; (2) return information about the causal relations among the latent factors so identified. We prove the procedure is point-wise consistent assuming (a) the causal relations can be represented by a directed acyclic graph (DAG) satisfying the Markov Assumption and the Faithfulness Assumption; (b) unrecorded variables are not caused by recorded (...) variables; and (c) dependencies are linear. We compare the procedure with standard approaches over a variety of simulated structures and sample sizes, and illustrate its practical value with brief studies of social science data sets. Finally, we consider generalizations for non-linear systems. Keywords: latent variable models, causality, graphical models.. (shrink)

In what follows I will discuss an example of the Duhem-Quine problem in which Pr(H A), Pr(A H), and Pr(OI +H& ?A) (where H is the hypothesis, A the auxiliary assumptions, and O the observational prediction) can be construed objectively; however, only some of those quantities are relevant to the analysis that I provide. The example involves medical diagnosis. The goal is to test the hypothesis that someone has tuberculosis; the auxiliary assumptions describe the er- ror characteristics of the test (...) procedure. Although it can make sense to talk about the objective probability that someone (randomly drawn from a given population) has tuberculosis and it also can make sense to talk about the objective probability that a test procedure has a certain set of error characteristics, neither of these quantities will enter into the analysis. The analysis proceeds entirely via likelihoods, what one needs to consider is just the probability of the observations conditional on four conjunctions of the form (?H & +A).' It is a special feature of the example that all four of these conjunctions are simple statistical hypotheses in the technical sense that each unambigu-ously confers a probability on the observations.'o After describing how the likelihood concept applies to the example concerning medical diagnosis, I will show how similar patterns can arise in the context of a second inferential framework-that of H. Akaike's criterion of model selection;" this time the example will involve phylogenetic inference. (shrink)

The paper examines the claim that significance testing violates the Principle of Total Evidence. I argue that p-values violate PTE for two-sided tests but satisfy PTE for one-sided tests invoking a sufficient test statistic independent of the preferred theory of evidence. While the focus of the paper is to evaluate a particular claim about the relationship of significance testing and PTE, I clarify the reading of this methodological principle along the way.

Using Gebharter’s representation, we consider aspects of the problem of discovering the structure of unmeasured submechanisms when the variables in those submechanisms have not been measured. Exploiting an early insight of Sober’s, we provide a correct algorithm for identifying latent, endogenous structure—submechanisms—for a restricted class of structures. The algorithm can be merged with other methods for discovering causal relations among unmeasured variables, and feedback relations between measured variables and unobserved causes can sometimes be learned.

This article focuses on the role of statistical concepts in both experiment and theory in various scientific disciplines, especially physics, including astronomy, and psychology. In Sect. 1 the concept of uncertainty in astronomy is analyzed from Ptolemy to Laplace and Gauss. In Sect. 2 theoretical uses of probability and statistics in science are surveyed. Attention is focused on the historically important example of radioactive decay. In Sect. 3 the use of statistics in biology and the social sciences is examined, with (...) detailed consideration of various Chi-square statistical tests. Such tests are essential for proper evaluation of many different kinds of scientific hypotheses. (shrink)

The Akaike Information Criterion can be a valuable tool of scientific inference. This statistic, or any other statistical method for that matter, cannot, however, be the whole of scientific methodology. In this paper some of the limitations of Akaikean statistical methods are discussed. It is argued that the full import of empirical evidence is realized only by adopting a richer ideal of empirical success than predictive accuracy, and that the ability of a theory to turn phenomena into accurate, agreeing measurements (...) of causally relevant parameters contributes to the evidential support of the theory. This is illustrated by Newton's argument from orbital phenomena to the inverse-square law of gravitation. (shrink)

The Akaike Information Criterion can be a valuable tool of scientific inference. This statistic, or any other statistical method for that matter, cannot, however, be the whole of scientific methodology. In this paper some of the limitations of Akaikean statistical methods are discussed. It is argued that the full import of empirical evidence is realized only by adopting a richer ideal of empirical success than predictive accuracy, and that the ability of a theory to turn phenomena into accurate, agreeing measurements (...) of causally relevant parameters contributes to the evidential support of the theory. This is illustrated by Newton's argument from orbital phenomena to the inverse-square law of gravitation. (shrink)

Famously, scientific theories are underdetermined by their evidence. This occurs in the factor analytic model, which is often used to connect concrete data to hypothetical notions. After introducing FA, three general topics are addressed. Underdetermination: the precise reasons why FA is underdetermined illuminates various claims about underdetermination, abduction, and theoretical terms. Uncertainties: FA helps distinguish at least four kinds of uncertainties. The prevailing practice, often encoded in statistical software, is to ignore the most difficult kinds, which are essential to FA's (...) underdetermination. What to do: some suggestions for dealing with these hardest types of uncertainty are offered. (shrink)

A widely endorsed thesis in the philosophy of science holds that if evidence for a hypothesis was not known when the hypothesis was proposed, then that evidence confirms the hypothesis more strongly than would otherwise be the case. The thesis has been thought to be inconsistent with Bayesian confirmation theory, but the arguments offered for that view are fallacious. This paper shows how the special value of prediction can in fact be given Bayesian explanation. The explanation involves consideration of the (...) reliability of the method by which the hypothesis was discovered, and thus reveals an intimate connection between the 'logic of discovery' and confirmation theory. (shrink)