Frequentism

Although probabilistic statements are ubiquitous, probability is still poorly understood. This shows itself, for example, in the mere stipulation of policies like expected utility maximisation and in disagreements about the correct interpretation of probability. In this work, we provide an account of probabilistic predictions that explains when, how, and why they can be useful for decision-making. We demonstrate that a calibration criterion on finite sets of predictions allows one to anticipate the distribution of utilities that a given policy will yield. (...) Based on this, we specify assumptions under which expected utility maximisation is a sensible decision criterion. We also introduce the notion of prediction methods, highlighting the dependence on schemes of abstraction, and argue that all probabilities are outputs of such prediction methods. This helps explain how the calibration criterion can be satisfied and shows that also supposedly objective probabilities are model-dependent. We compare our account of probability with conventional interpretations and show that it recovers key intuitions behind the latter. We, thus, provide a general account of what probabilities are and how they can enable successful decision-making under uncertainty. (shrink)

Frequency-based arguments against rational belief in a miracle occurring have been present for centuries, the most notable being from David Hume. In this essay, I will show Hume's argument rests on an equivocation of probability, with him using the term interchangeably to refer to two different and incompatible perspectives: Bayesianism and Frequentism. Additionally, I will show that any frequentist arguments against miracles relies on a view of probability that is only dubiously linked to rationality. In other words, the frequentist cannot (...) have it both ways: if probability is indeed frequency, then miracles are indeed improbable but not necessarily irrational to believe. On the other hand, if probability is an agent's confidence in a given belief, than under certain assumptions miracles are indeed highly probable and thus rational to believe despite their rarity. As a result, regardless of which view of probability one takes, it does not follow that believing in a miracle is irrational simply because miracles are rare. (shrink)

It is argued that Maximum Likelihood Estimation (MLE) is wrong, both conceptually and in terms of results it produces (except in two very special cases, which are discussed). While the use of MLE can still be justified on the basis of its practical performance, we argue there are better estimation methods that overcome MLE's empirical and philosophical shortcomings while retaining all of MLE's benefits.

One justification for preregistering research hypotheses, methods, and analyses is that it improves the transparent evaluation of the severity of hypothesis tests. In this article, I consider two cases in which preregistration does not improve this evaluation. First, I argue that, although preregistration can facilitate the transparent evaluation of severity in Mayo’s error statistical philosophy of science, it does not facilitate this evaluation in Popper’s theory-centric approach. To illustrate, I show that associated concerns about Type I error rate inflation are (...) only relevant in the error statistical approach and not in a theory-centric approach. Second, I argue that a preregistered test procedure that allows deviations in its implementation does not provide a more transparent evaluation of Mayoian severity than a non-preregistered procedure. In particular, I argue that sample-based validity-enhancing deviations cause an unknown inflation of the test procedure’s Type I (familywise) error rate and, consequently, an unknown reduction in its capability to license inferences severely. I conclude that preregistration does not improve the transparent evaluation of severity in Popper’s philosophy of science or when deviations are allowed. (shrink)

In the following we will investigate whether von Mises’ frequency interpretation of probability can be modified to make it philosophically acceptable. We will reject certain elements of von Mises’ theory, but retain others. In the interpretation we propose we do not use von Mises’ often criticized ‘infinite collectives’ but we retain two essential claims of his interpretation, stating that probability can only be defined for events that can be repeated in similar conditions, and that exhibit frequency stabilization. The central idea (...) of the present article is that the mentioned ‘conditions’ should be well-defined and ‘partitioned’. More precisely, we will divide probabilistic systems into object, initializing, and probing subsystem, and show that such partitioning allows to solve problems. Moreover we will argue that a key idea of the Copenhagen interpretation of quantum mechanics (the determinant role of the observing system) can be seen as deriving from an analytic definition of probability as frequency. Thus a secondary aim of the article is to illustrate the virtues of analytic definition of concepts, consisting of making explicit what is implicit. (shrink)

Like Lewis, many philosophers hold reductionist accounts of chance (on which claims about chance are to be understood as claims that certain patterns of events are instantiated) and maintain that rationality requires that credence should defer to chance (in the sense that under certain circumstances one's credence in an event must coincide with the chance of that event). It is a shortcoming of an account of chance if it implies that this norm of rationality is unsatisfiable by computable agents. This (...) shortcoming is more common than one might have hoped. (shrink)

The inflation of Type I error rates is thought to be one of the causes of the replication crisis. Questionable research practices such as p-hacking are thought to inflate Type I error rates above their nominal level, leading to unexpectedly high levels of false positives in the literature and, consequently, unexpectedly low replication rates. In this article, I offer an alternative view. I argue that questionable and other research practices do not usually inflate relevant Type I error rates. I begin (...) by introducing the concept of Type I error rates and distinguishing between statistical errors and theoretical errors. I then illustrate my argument with respect to model misspecification, multiple testing, selective inference, forking paths, exploratory analyses, p-hacking, optional stopping, double dipping, and HARKing. In each case, I demonstrate that relevant Type I error rates are not usually inflated above their nominal level, and in the rare cases that they are, the inflation is easily identified and resolved. I conclude that the replication crisis may be explained, at least in part, by researchers’ misinterpretation of statistical errors and their underestimation of theoretical errors. (shrink)

Hájek (Erkenntnis 70(2):211–235, 2009) argues that probabilities cannot be the limits of relative frequencies in counterfactual infinite sequences. I argue for a different understanding of these limits, drawing on Norton’s (Philos Sci 79(2):207–232, 2012) distinction between approximations (inexact descriptions of a target) and idealizations (separate models that bear analogies to the target). Then, I adapt Hájek’s arguments to this new context. These arguments provide excellent reasons not to use hypothetical frequencies as idealizations, but no reason not to use them as (...) approximations. (shrink)

In this paper, we illustrate some serious difficulties involved in conveying information about uncertain risks and securing informed consent for risky interventions in a clinical setting. We argue that in order to secure informed consent for a medical intervention, physicians often need to do more than report a bare, numerical probability value. When probabilities are given, securing informed consent generally requires communicating how probability expressions are to be interpreted and communicating something about the quality and quantity of the evidence for (...) the probabilities reported. Patients may also require guidance on how probability claims may or may not be relevant to their decisions, and physicians should be ready to help patients understand these issues. (shrink)

Book Review of Kenneth Westphal's Kant’s Critical Epistemology: Why Epistemology Must Consider Judgment First.

Philosophers now seem to agree that frequentism is an untenable strategy to explain the meaning of probabilities. Nevertheless, I want to revive frequentism, and I will do so by grounding probabilities on typicality in the same way as the thermodynamic arrow of time can be grounded on typicality within statistical mechanics. This account, which I will call typicality frequentism, will evade the major criticisms raised against previous forms of frequentism. In this theory, probabilities arise within a physical theory from statistical (...) behavior of almost all initial conditions. The main advantage of typicality frequentism is that it shows which kinds of probabilities (that also have empirical relevance) can be derived from physics. Although one cannot recover all probability talk in this account, this is rather a virtue than a vice, because it shows which types of probabilities can in fact arise from physics and which types need to be explained in different ways, thereby opening the path for a pluralistic account of probabilities. (shrink)

Laura Papish’s Kant on Evil, Self-Deception, and Moral Reform is an ambitious attempt to breath new life into old debates and a welcome contribution to a recent renaissance of interest in Kant’s theory of evil. ​The book has eight chapters, and these chapters fall into three main divisions. Chapters 1 and 2 focus on the psychology of nonmoral and immoral action. Chapters 3, 4, and 5 focus on self-deception, evil, and dissimulation. And chapters 6, 7, and 8 focus on self-cognition, (...) moral reform, and moral progress. I shall begin with a brief summary of these chapters. Then I shall turn to commentary. (shrink)

The replication crisis has caused researchers to distinguish between exact replications, which duplicate all aspects of a study that could potentially affect the results, and direct replications, which duplicate only those aspects of the study that are thought to be theoretically essential to reproduce the original effect. The replication crisis has also prompted researchers to think more carefully about the possibility of making Type I errors when rejecting null hypotheses. In this context, the present article considers the utility of two (...) types of Type I error probability: the Neyman–Pearson long run Type I error rate and the Fisherian sample-specific Type I error probability. It is argued that the Neyman–Pearson Type I error rate is inapplicable in social science because it refers to a long run of exact replications, and social science deals with irreversible units that make exact replications impossible. Instead, the Fisherian sample-specific Type I error probability is recommended as a more meaningful way to conceptualize false positive results in social science because it can be applied to each sample-specific decision about rejecting the same substantive null hypothesis in a series of direct replications. It is concluded that the replication crisis may be partly due to researchers’ unrealistic expectations about replicability based on their consideration of the Neyman–Pearson Type I error rate across a long run of exact replications. (shrink)

Fisher criticised the Neyman-Pearson approach to hypothesis testing by arguing that it relies on the assumption of “repeated sampling from the same population.” The present article considers the responses to this criticism provided by Pearson and Neyman. Pearson interpreted alpha levels in relation to imaginary replications of the original test. This interpretation is appropriate when test users are sure that their replications will be equivalent to one another. However, by definition, scientific researchers do not possess sufficient knowledge about the relevant (...) and irrelevant aspects of their tests and populations to be sure that their replications will be equivalent to one another. Pearson also interpreted the alpha level as a personal rule that guides researchers’ behavior during hypothesis testing. However, this interpretation fails to acknowledge that the same researcher may use different alpha levels in different testing situations. Addressing this problem, Neyman proposed that the average alpha level adopted by a particular researcher can be viewed as an indicator of that researcher’s typical Type I error rate. Researchers’ average alpha levels may be informative from a metascientific perspective. However, they are not useful from a scientific perspective. Scientists are more concerned with the error rates of specific tests of specific hypotheses, rather than the error rates of their colleagues. It is concluded that neither Neyman nor Pearson adequately rebutted Fisher’s “repeated sampling” criticism. Fisher’s significance testing approach is briefly considered as an alternative to the Neyman-Pearson approach. (shrink)

Where E is the proposition that [If H and O were true, H would explain O], William Roche and Elliot Sober have argued that P(H|O&E) = P(H|O). In this paper I argue that not only is this equality not generally true, it is false in the very kinds of cases that Roche and Sober focus on, involving frequency data. In fact, in such cases O raises the probability of H only given that there is an explanatory connection between them.

Early work on the frequency theory of probability made extensive use of the notion of randomness, conceived of as a property possessed by disorderly collections of outcomes. Growing out of this work, a rich mathematical literature on algorithmic randomness and Kolmogorov complexity developed through the twentieth century, but largely lost contact with the philosophical literature on physical probability. The present chapter begins with a clarification of the notions of randomness and probability, conceiving of the former as a property of a (...) sequence of outcomes, and the latter as a property of the process generating those outcomes. A discussion follows of the nature and limits of the relationship between the two notions, with largely negative verdicts on the prospects for any reduction of one to the other, although the existence of an apparently random sequence of outcomes is good evidence for the involvement of a genuinely chancy process. (shrink)

At the heart of Drazen Prelec’s chapter is the distinction between outcome utility and diagnostic utility. There is a particular distinction in the literature on causal networks (Pearl 2000), namely the distinction between observing and intervening, that maps onto Prelec’s distinction between diagnostic and outcome utility. I will explore the connection between both frameworks.

In the face of continuing assumptions by many scientists and journal editors that p-values provide a gold standard for inference, counter warnings are published periodically. But the core problem is not with p-values, per se. A finding that “p-value is less than α” could merely signal that a critical value has been exceeded. The question is why, when estimating a parameter, we provide a range (a confidence interval), but when testing a hypothesis about a parameter (e.g. µ = x) we (...) proceed as if “=” entails exact equality of the parameter with x. That standard is hard to meet, and is not a standard expected for power calculations, where we are satisfied to reject a null hypothesis H0 if the result is merely “detectably” different from (exact) H0. This paper explores, with resampling (simulation) methods, the impacts on p-values, and alternatives, if the null hypothesis is defined as a thick or thin range of values. It also examines, empirically, the extent to which the p-value may or may not be a good predictor of the probability that H0 is true, given the distribution of the data. (shrink)

The economic paradigms of Ludwig von Mises on the one hand and of John Maynard Keynes on the other have been correctly recognized as antithetical at the theoretical level, and as antagonistic with respect to their practical and public policy implications. Characteristically they have also been vindicated by opposing sides of the political spectrum. Nevertheless the respective views of these authors with respect to the meaning and interpretation of probability exhibit a closer conceptual affinity than has been acknowledged in the (...) literature. In particular it is argued that in some relevant respects Ludwig von Mises´ interpretation of the concept of probability exhibits a closer affinity with the interpretation of probability developed by his opponent John Maynard Keynes than with the views on probability espoused by his brother Richard von Mises. Nevertheless there also exist significant differences between the views of Ludwig von Mises and those of John Maynard Keynes with respect to probability. One of these is highlighted more particularly: where John Maynard Keynes advocated a monist view of probability, Ludwig von Mises embraced a dualist view of probability, according to which the concept of probability has two different meanings each of which is valid in a particular area or context. It is concluded that both John Maynard Keynes and Ludwig von Mises presented highly nuanced views with respect to the meaning and interpretation of probability. (shrink)