I present a new proof of the likelihood principle that avoids two responses to a well-known proof due to Birnbaum ([1962]). I also respond to arguments that Birnbaum’s proof is fallacious, which if correct could be adapted to this new proof. On the other hand, I urge caution in interpreting proofs of the likelihood principle as arguments against the use of frequentist statistical methods. 1 Introduction2 The New Proof3 How the New Proof Addresses Proposals to Restrict Birnbaum’s Premises4 A Response (...) to Arguments that the Proofs Are Fallacious5 Conclusion. (shrink)

Frequency-based arguments against rational belief in a miracle occurring have been present for centuries, the most notable being from David Hume. In this essay, the authors show Hume's argument rests on an equivocation of probability, with him using the term interchangeably to refer to two different and incompatible perspectives: Bayesianism and Frequentism. Additionally, the authors show that any frequentist arguments against miracles relies on a view of probability that is only dubiously linked to rationality. In other words, the frequentist cannot (...) have it both ways: if probability is indeed frequency, then miracles are indeed improbable but not necessarily irrational to believe. On the other hand, if probability is an agent's confidence in a given belief, than under certain assumptions miracles are indeed highly probable and thus rational to believe despite their rarity. As a result, regardless of which view of probability one takes, it does not follow that believing in a miracle is irrational simply because miracles are rare. (shrink)

Many biologists, especially in ecology and evolution, analyze their data by estimating fits to a set of candidate models and selecting the best model according to the Akaike Information Criterion (AIC) or the Bayesian Information Criteria (BIC). When the candidate models represent alternative hypotheses, biologists may want to limit the chance of a false positive to a specified level. Existing model selection methodology, however, allows for only indirect control over error rates by setting a threshold for the difference in AIC (...) scores. We present a novel theoretical framework for parametric Neyman-Pearson (NP) model selection using information criteria that does not require a pre-data null and applies to three or more non-nested models simultaneously. We apply the theoretical framework to the Error Control for Information Criteria (ECIC) procedure introduced by Cullan et al. (J Appl Stat 47: 2565–2581, 2019), and we show it shares many of the desirable properties of AIC-type methods, including false positive and negative rates that converge to zero asymptotically. We discuss implications for the compatibility of evidentialist and severity-based approach to evidence in philosophy of science. (shrink)

There has long been an impression that reliabilism implies externalism and that frequentist statistics, due to its reliabilist nature, is inherently externalist. I argue, however, that frequentist statistics can plausibly be understood as a form of internalist reliabilism -- internalist in the conventional sense, yet reliabilist in certain unconventional and intriguing ways. Crucially, in developing the thesis that reliabilism does not imply externalism, my aim is not to stretch the meaning of ‘reliabilism’ merely to sever the implication. Instead, it is (...) to gain a deeper understanding of frequentist statistics, which stands as one of the most sustained attempts by scientists to develop an epistemology for their own use. (shrink)

Why should moral philosophers, moral psychologists, and machine ethicists care about computational complexity? Debates on whether artificial intelligence (AI) can or should be used to solve problems in ethical domains have mainly been driven by what AI can or cannot do in terms of human capacities. In this paper, we tackle the problem from the other end by exploring what kind of moral machines are possible based on what computational systems can or cannot do. To do so, we analyze normative (...) ethics through the lens of computational complexity. First, we introduce computational complexity for the uninitiated reader and discuss how the complexity of ethical problems can be framed within Marr’s three levels of analysis. We then study a range of ethical problems based on consequentialism, deontology, and virtue ethics, with the aim of elucidating the complexity associated with the problems themselves (e.g., due to combinatorics, uncertainty, strategic dynamics), the computational methods employed (e.g., probability, logic, learning), and the available resources (e.g., time, knowledge, learning). The results indicate that most problems the normative frameworks pose lead to tractability issues in every category analyzed. Our investigation also provides several insights about the computational nature of normative ethics, including the differences between rule- and outcome-based moral strategies, and the implementation-variance with regard to moral resources. We then discuss the consequences complexity results have for the prospect of moral machines in virtue of the trade-off between optimality and efficiency. Finally, we elucidate how computational complexity can be used to inform both philosophical and cognitive-psychological research on human morality by advancing the moral tractability thesis. (shrink)

Scientists often adjust their significance threshold during null hypothesis significance testing in order to take into account multiple testing and multiple comparisons. This alpha adjustment has become particularly relevant in the context of the replication crisis in science. The present article considers the conditions in which this alpha adjustment is appropriate and the conditions in which it is inappropriate. A distinction is drawn between three types of multiple testing: disjunction testing, conjunction testing, and individual testing. It is argued that alpha (...) adjustment is only appropriate in the case of disjunction testing, in which at least one test result must be significant in order to reject the associated joint null hypothesis. Alpha adjustment is inappropriate in the case of conjunction testing, in which all relevant results must be significant in order to reject the joint null hypothesis. Alpha adjustment is also inappropriate in the case of individual testing, in which each individual result must be significant in order to reject each associated individual null hypothesis. The conditions under which each of these three types of multiple testing is warranted are examined. It is concluded that researchers should not automatically assume that alpha adjustment is necessary during multiple testing. Illustrations are provided in relation to joint studywise hypotheses and joint multiway ANOVAwise hypotheses. (shrink)

Bayesian confirmation theory is a leading theory to decide the confirmation/refutation of a hypothesis based on probability calculus. While it may be much discussed in philosophy of science, is it actually practiced in terms of hypothesis testing by scientists? Since the assignment of some of the probabilities in the theory is open to debate and the risk of making the wrong decision is unknown, many scientists do not use the theory in hypothesis testing. Instead, they use alternative statistical tests that (...) can measure the risk or the reliability in decision making, circumventing some of the theoretical problems in practice. Therefore, the theory is not very popular in hypothesis testing among scientists at present. However, there are some proponents of Bayesian hypothesis testing, and software packages are made available to accelerate utilization by scientists. Time will tell whether Bayesian confirmation theory can become both a leading theory and a widely practiced method. In addition, this theory can be used to model the belief of scientists when testing hypotheses. (shrink)

Complex constraints like conditionals ('If A, then B') and probabilistic constraints ('The probability that A is p') pose problems for Bayesian theories of learning. Since these propositions do not express constraints on outcomes, agents cannot simply conditionalize on the new information. Furthermore, a natural extension of conditionalization, relative information minimization, leads to many counterintuitive predictions, evidenced by the sundowners problem and the Judy Benjamin problem. Building on the notion of a `paradigm shift' and empirical research in psychology and economics, I (...) argue that the model of hypothesis testing can explain how people learn complex, theory-laden propositions like conditionals and probability constraints. Theories are formalized as probability distributions over a set of possible outcomes and theory change is triggered by a constraint which is incompatible with the initial theory. This leads agents to consult a higher order probability function, or a 'prior over priors,' to choose the most likely alternative theory which satisfies the constraint. The hypothesis testing model is applied to three examples: learning a simple probabilistic constraint involving coin bias, the sundowners problem for conditional learning, and the Judy Benjamin problem for learning conditional probability constraints. (shrink)

A number of philosophers of science and statisticians have attempted to justify conclusions drawn from a finite sequence of evidence by appealing to results about what happens if the length of that sequence tends to infinity. If their justifications are to be successful, they need to rely on the finite sequence being either indefinitely increasing or of a large size. These assumptions are often not met in practice. This paper analyzes a simple model of collecting evidence and finds that the (...) practice of collecting only very small sets of evidence before taking a question to be settled is rationally justified. This shows that the appeal to long run results can be used neither to explain the success of actual scientific practice nor to give a rational reconstruction of that practice. (shrink)

After sketching the conflict between objectivists and subjectivists on the foundations of statistics, this paper discusses an issue facing statisticians of both schools, namely, model validation. Statistical models originate in the study of games of chance, and have been successfully applied in the physical and life sciences. However, there are basic problems in applying the models to social phenomena; some of the difficulties will be pointed out. Hooke's law will be contrasted with regression models for salary discrimination, the latter being (...) a fairly typical application in the social sciences. (shrink)

Multilevel models are one of the most frequently used methods for analyzing multilevel data. These types of data occur when observations (Level 1) are clustered within higher-level collectives (Level 2), for example, students nested in schools or employees nested in work teams. Unfortunately, multilevel data often contain missing data, for example, when participants omit certain items in a questionnaire or they drop out before the end of a study. If treated improperly, missing data can distort parameter estimates and compromise statistical (...) decision making. For this reason, it is often recommended to use principled methods for dealing with missing data such as multiple imputation (MI) or maximum likelihood estimation (ML). In the present dissertation, I consider the treatment of missing data with an emphasis on multilevel MI. In multilevel research, it is important that the imputation model takes the multilevel structure into account. However, many open questions remain about how this can be achieved in practice. In multiple studies, I considered different implementations of multilevel MI and examined how missing data can be treated in multilevel models with random intercepts, random slopes, interaction effects, continuous and categorical data, and missing data at Level 2. In addition, I considered the analysis of multiply imputed data sets and examined different procedures for pooling the results of multiparameter tests (e.g., model comparisons), including applications in the analysis of variance (ANOVA) and multilevel models with tests about fixed effects and variance components. Finally, I present the R package mitml, which provides a set of practical tools for conducting multilevel MI. This includes tools for the specification of the imputation model, convergence diagnostics, analyzing multiply imputed data sets, and various pooling methods along with a tutorial article that illustrates its features and provides a nontechnical introduction to multilevel MI. (shrink)

The problem of understanding how statistical inference is, and can be, applied in empirical sciences is important for the methodology of science. It is the objective of this paper to gain a better understanding of the role of statistical methods in scientific modeling. The important question of whether the applicability reduces to the representational properties of statistical models is discussed. It will be shown that while the answer to this question is positive, representation in statistical models is not purely structural. (...) In spite of the fact that representation in statistical models is based on the structural similarities between the statistical model and the empirical systems under study, these relationships are shown to be appropriate for representing relations in the target system by agent function, too. A second aspect of the paper involves the claim that agent-based components of statistical modeling are: a) interpretation of random variables, b) selection of the goal of statistical research, and c) selection of estimator properties. To justify these claims, a preliminary discussion will be presented on the role of statistics in modeling, as in regression and other structural models. This role will be explored and realized using a structural viewpoint. Also the role of statistical estimation in statistical modeling is discussed to explain the representational role of models and the inferential role of the agent in modeling. (shrink)

Two-stage voting is prone to majority inversions, a situation in which the outcome of an election is not backed by a majority of popular votes. We study the probability of majority inversion in a model with two candidates, three states and uniformly distributed fractions of supporters for each candidate. The model encompasses equal or distinct population sizes, with equal, population-based or arbitrary voting weights in the second stage. We prove that, when no state can dictate the outcome of the election (...) by commanding a voting weight in excess of one half, the probability of majority inversion increases with the size disparity among the states. (shrink)

Classical statistical inferences have been criticised for various reasons. To assess the soundness of such criticisms is a very important task because they are widely used in everyday scientific research. This is one of the reasons why the philosophy of statistics is an exciting field of study. In this paper, I focus on two such criticisms. The first one claims that the use of the p-value violates the principle of total evidence. It is a thesis that has been defended by (...) Elliott Sober and Bengt Autzen. The second one says that the result of classical tests does not only depend on the data but on the sampling plan of the experimenter also. The underlying criticism of course is that the sampling plan is not part of the evidence and that classical tests therefore violate PTE. The intentions of the experimenter should not affect the result of an inference. My aim is to show that both criticisms are unsound. Doing so, I hope to clarify the concept of p-value and the nature of the evidence in classical statistical tests. The point of my paper is to show that the identification of the evidence on which those criticisms rest is inadequate. (shrink)