After sketching the conflict between objectivists and subjectivists on the foundations of statistics, this paper discusses an issue facing statisticians of both schools, namely, model validation. Statistical models originate in the study of games of chance, and have been successfully applied in the physical and life sciences. However, there are basic problems in applying the models to social phenomena; some of the difficulties will be pointed out. Hooke's law will be contrasted with regression models for salary discrimination, the latter being (...) a fairly typical application in the social sciences. (shrink)

Why should moral philosophers, moral psychologists, and machine ethicists care about computational complexity? Debates on whether artificial intelligence (AI) can or should be used to solve problems in ethical domains have mainly been driven by what AI can or cannot do in terms of human capacities. In this paper, we tackle the problem from the other end by exploring what kind of moral machines are possible based on what computational systems can or cannot do. To do so, we analyze normative (...) ethics through the lens of computational complexity. First, we introduce computational complexity for the uninitiated reader and discuss how the complexity of ethical problems can be framed within Marr’s three levels of analysis. We then study a range of ethical problems based on consequentialism, deontology, and virtue ethics, with the aim of elucidating the complexity associated with the problems themselves (e.g., due to combinatorics, uncertainty, strategic dynamics), the computational methods employed (e.g., probability, logic, learning), and the available resources (e.g., time, knowledge, learning). The results indicate that most problems the normative frameworks pose lead to tractability issues in every category analyzed. Our investigation also provides several insights about the computational nature of normative ethics, including the differences between rule- and outcome-based moral strategies, and the implementation-variance with regard to moral resources. We then discuss the consequences complexity results have for the prospect of moral machines in virtue of the trade-off between optimality and efficiency. Finally, we elucidate how computational complexity can be used to inform both philosophical and cognitive-psychological research on human morality by advancing the moral tractability thesis. (shrink)

Scientists often adjust their significance threshold during null hypothesis significance testing in order to take into account multiple testing and multiple comparisons. This alpha adjustment has become particularly relevant in the context of the replication crisis in science. The present article considers the conditions in which this alpha adjustment is appropriate and the conditions in which it is inappropriate. A distinction is drawn between three types of multiple testing: disjunction testing, conjunction testing, and individual testing. It is argued that alpha (...) adjustment is only appropriate in the case of disjunction testing, in which at least one test result must be significant in order to reject the associated joint null hypothesis. Alpha adjustment is inappropriate in the case of conjunction testing, in which all relevant results must be significant in order to reject the joint null hypothesis. Alpha adjustment is also inappropriate in the case of individual testing, in which each individual result must be significant in order to reject each associated individual null hypothesis. The conditions under which each of these three types of multiple testing is warranted are examined. It is concluded that researchers should not automatically assume that alpha adjustment is necessary during multiple testing. Illustrations are provided in relation to joint studywise hypotheses and joint multiway ANOVAwise hypotheses. (shrink)

Complex constraints like conditionals ('If A, then B') and probabilistic constraints ('The probability that A is p') pose problems for Bayesian theories of learning. Since these propositions do not express constraints on outcomes, agents cannot simply conditionalize on the new information. Furthermore, a natural extension of conditionalization, relative information minimization, leads to many counterintuitive predictions, evidenced by the sundowners problem and the Judy Benjamin problem. Building on the notion of a `paradigm shift' and empirical research in psychology and economics, I (...) argue that the model of hypothesis testing can explain how people learn complex, theory-laden propositions like conditionals and probability constraints. Theories are formalized as probability distributions over a set of possible outcomes and theory change is triggered by a constraint which is incompatible with the initial theory. This leads agents to consult a higher order probability function, or a 'prior over priors,' to choose the most likely alternative theory which satisfies the constraint. The hypothesis testing model is applied to three examples: learning a simple probabilistic constraint involving coin bias, the sundowners problem for conditional learning, and the Judy Benjamin problem for learning conditional probability constraints. (shrink)

Two-stage voting is prone to majority inversions, a situation in which the outcome of an election is not backed by a majority of popular votes. We study the probability of majority inversion in a model with two candidates, three states and uniformly distributed fractions of supporters for each candidate. The model encompasses equal or distinct population sizes, with equal, population-based or arbitrary voting weights in the second stage. We prove that, when no state can dictate the outcome of the election (...) by commanding a voting weight in excess of one half, the probability of majority inversion increases with the size disparity among the states. (shrink)

A number of philosophers of science and statisticians have attempted to justify conclusions drawn from a finite sequence of evidence by appealing to results about what happens if the length of that sequence tends to infinity. If their justifications are to be successful, they need to rely on the finite sequence being either indefinitely increasing or of a large size. These assumptions are often not met in practice. This paper analyzes a simple model of collecting evidence and finds that the (...) practice of collecting only very small sets of evidence before taking a question to be settled is rationally justified. This shows that the appeal to long run results can be used neither to explain the success of actual scientific practice nor to give a rational reconstruction of that practice. (shrink)

Elliott Sober is both an empiricist and an instrumentalist. His empiricism rests on a principle called actualism, whereas his instrumentalism violates this. This violation generates a tension in his work. We argue that Sober is committed to a conflicting methodological imperative because of this tension. Our argument illuminates the contemporary debate between realism and empiricism which is increasingly focused on the application of scientific inference to testing scientific theories. Sober’s position illustrates how the principle of actualism drives a wedge between (...) two conceptions of scientific inference and at the same time brings to the surface a deep conflict between empiricism and instrumentalism. (shrink)

The problem of understanding how statistical inference is, and can be, applied in empirical sciences is important for the methodology of science. It is the objective of this paper to gain a better understanding of the role of statistical methods in scientific modeling. The important question of whether the applicability reduces to the representational properties of statistical models is discussed. It will be shown that while the answer to this question is positive, representation in statistical models is not purely structural. (...) In spite of the fact that representation in statistical models is based on the structural similarities between the statistical model and the empirical systems under study, these relationships are shown to be appropriate for representing relations in the target system by agent function, too. A second aspect of the paper involves the claim that agent-based components of statistical modeling are: a) interpretation of random variables, b) selection of the goal of statistical research, and c) selection of estimator properties. To justify these claims, a preliminary discussion will be presented on the role of statistics in modeling, as in regression and other structural models. This role will be explored and realized using a structural viewpoint. Also the role of statistical estimation in statistical modeling is discussed to explain the representational role of models and the inferential role of the agent in modeling. (shrink)

Multilevel models are one of the most frequently used methods for analyzing multilevel data. These types of data occur when observations (Level 1) are clustered within higher-level collectives (Level 2), for example, students nested in schools or employees nested in work teams. Unfortunately, multilevel data often contain missing data, for example, when participants omit certain items in a questionnaire or they drop out before the end of a study. If treated improperly, missing data can distort parameter estimates and compromise statistical (...) decision making. For this reason, it is often recommended to use principled methods for dealing with missing data such as multiple imputation (MI) or maximum likelihood estimation (ML). In the present dissertation, I consider the treatment of missing data with an emphasis on multilevel MI. In multilevel research, it is important that the imputation model takes the multilevel structure into account. However, many open questions remain about how this can be achieved in practice. In multiple studies, I considered different implementations of multilevel MI and examined how missing data can be treated in multilevel models with random intercepts, random slopes, interaction effects, continuous and categorical data, and missing data at Level 2. In addition, I considered the analysis of multiply imputed data sets and examined different procedures for pooling the results of multiparameter tests (e.g., model comparisons), including applications in the analysis of variance (ANOVA) and multilevel models with tests about fixed effects and variance components. Finally, I present the R package mitml, which provides a set of practical tools for conducting multilevel MI. This includes tools for the specification of the imputation model, convergence diagnostics, analyzing multiply imputed data sets, and various pooling methods along with a tutorial article that illustrates its features and provides a nontechnical introduction to multilevel MI. (shrink)

Classical statistical inferences have been criticised for various reasons. To assess the soundness of such criticisms is a very important task because they are widely used in everyday scientific research. This is one of the reasons why the philosophy of statistics is an exciting field of study. In this paper, I focus on two such criticisms. The first one claims that the use of the p-value violates the principle of total evidence. It is a thesis that has been defended by (...) Elliott Sober and Bengt Autzen. The second one says that the result of classical tests does not only depend on the data but on the sampling plan of the experimenter also. The underlying criticism of course is that the sampling plan is not part of the evidence and that classical tests therefore violate PTE. The intentions of the experimenter should not affect the result of an inference. My aim is to show that both criticisms are unsound. Doing so, I hope to clarify the concept of p-value and the nature of the evidence in classical statistical tests. The point of my paper is to show that the identification of the evidence on which those criticisms rest is inadequate. (shrink)

Bayesian confirmation theory is a leading theory to decide the confirmation/refutation of a hypothesis based on probability calculus. While it may be much discussed in philosophy of science, is it actually practiced in terms of hypothesis testing by scientists? Since the assignment of some of the probabilities in the theory is open to debate and the risk of making the wrong decision is unknown, many scientists do not use the theory in hypothesis testing. Instead, they use alternative statistical tests that (...) can measure the risk or the reliability in decision making, circumventing some of the theoretical problems in practice. Therefore, the theory is not very popular in hypothesis testing among scientists at present. However, there are some proponents of Bayesian hypothesis testing, and software packages are made available to accelerate utilization by scientists. Time will tell whether Bayesian confirmation theory can become both a leading theory and a widely practiced method. In addition, this theory can be used to model the belief of scientists when testing hypotheses. (shrink)