There is no strong reason to believe that human-level intelligence represents an upper limit of the capacity of artificial intelligence, should it be realized. This poses serious safety issues, since a superintelligent system would have great power to direct the future according to its possibly flawed motivation system. Solving this issue in general has proven to be considerably harder than expected. This paper looks at one particular approach, Oracle AI. An Oracle AI is an AI that does not act in (...) the world except by answering questions. Even this narrow approach presents considerable challenges. In this paper, we analyse and critique various methods of controlling the AI. In general an Oracle AI might be safer than unrestricted AI, but still remains potentially dangerous. (shrink)

According to a simple form of consequentialism, we should base decision on our judgments about their consequences for achieving out goals. Our goals give us reason to endorse consequentialism as a standard of decision making. Alternative standards invariably lead to consequences that are less good in this sense. Yet some people knowingly follow decision rules that violate consequentialism. For example, they prefer harmful omissions to less harmful acts, they favor the status quo over alternatives they would otherwise judge to be (...) better, they provide third-party compensation on the basis of the cause of an injury rather than the benefit from the compensation, they ignore deterrent effects in decisions about punishment, and they resist coercive reforms they judge to be beneficial. I suggest that nonconsequentialist principles arise from overgeneralizing rules that are consistent with consequentialism in a limited set of cases. Commitment to such rules is detached from their original purposes. The existence of such nonconsequentialist decision biases has implications for philosophical and experimental methodology, the relation between psychology and public policy, and education. (shrink)

This essay lays the groundwork for a novel conception of autonomy that may be called “effective autonomy”—a conception designed to be genuinely action guiding in bioethics. As empirical psychology research on the heuristics and biases approach shows, decision making commonly fails to correspond to people’s desires because of the biases arising from bounded cognition. People who are classified as autonomous on contemporary philosophical accounts may fail to be effectively autonomous because their decisions are uncoupled from their autonomous desires. Accordingly, continuing (...) attempts to value patient autonomy must go beyond existing philosophical conceptions of autonomy to consider the background conditions of human decision making. (shrink)

This paper uses the idea of a proxy, which figures in discussions of bounded rationality, to construct an argument for a revisionary conclusion about ideal instrumental rationality. I consider how subjective responses can figure as proxies in heuristics and develop the following argument: (1) Proxies, even if relatively easy to recognize, can sometimes be messy, prompting incomplete or cyclic preferences. (2) From the point of view of ideal instrumental rationality, it is permissible for an agent to be concerned with a (...) proxy rather than with what it is a proxy for. And so, (3) although it is standardly assumed, as part of the prevailing conception of ideal instrumental rationality, that rational preferences are complete and acyclic, neither incomplete preferences nor cyclic preferences can be dismissed as invariably irrational from the point of view of ideal instrumental rationality. (shrink)

Recent decades have seen a surge in interest in metaphilosophy. In particular there has been an interest in philosophical methodology. Various questions have been asked about philosophical methods. Are our methods any good? Can we improve upon them? Prior to such evaluative and ameliorative concerns, however, is the matter of what methods philosophers actually use. Worryingly, our understanding of philosophical methodology is impoverished in various respects. This article considers one particular respect in which we seem to be missing an important (...) part of the picture. While it is a received wisdom that the word “ intuition ” has exploded across analytic philosophy in recent decades, the article presents evidence that the explosion is apparent across a broad swathe of academia. It notes various implications for current methodological debates about the role of intuitions in philosophy. (shrink)

Cognitive algebra strongly disproved the representativeness heuristic almost before it was published; and therewith it also disproved the base rate fallacy. Cognitive algebra provides a theoretical foundation for judgment-decision theory through its joint solution to the two fundamental problems – true measurement of subjective values, and cognitive rules for integration of multiple determinants.

Gerd Gigerenzer's views on probabilistic reasoning in humans have come under close scrutiny. Very little attention, however, has been paid to his evolutionary component of his argument. According to Gigerenzer, reasoning about probabilities as frequencies is so common today because it was favored by natural selection in the past. This paper presents a critical examination of this argument. It will show first, that, _pace_ Gigerenzer, there are some reasons to believe that using the frequency format was not more adaptive than (...) using the standard (percentage) format and, second, that Gigerenzer's evolutionary argument and his other arguments such as his historical description of the notion of probability are in tension with each other. (shrink)

Some generic sentences seem to be true despite the fact that almost all the members of the relevant kind are exceptions. It’s controversial whether generics of this type express relatively weak generalizations or relatively strong ones. If the latter, then we’re systematically mistaken about their truth, but they make no trouble for our semantic theorizing. In this brief note, I present several arguments for the former: sentences of the relevant type are weak generics.

Professions have a social obligation to ensure that their members’ professional behavior is morally appropriate. The psychology profession in most jurisdictions delegates the responsibility of ensuring that psychologists entering the profession are ethically competent to pre-practicum training programs. Educators responsible for teaching the ethics courses in these programs often base them on Rest’s (1984, 1994) theory that does not take into account a vast amount of contemporary psychological and neuroscientific research data on moral decision making. My aim with this article (...) is therefore to summarize the psychological and neuroscience research findings relevant to moral decision making and consider how educators who teach ethics courses can apply them in developing ethics courses grounded in research. (shrink)

The ArgumentThis paper applies the approach developed by the congnitive sciences to a classical field of social anthropology—i.e., the analysis of represetations and behaviors relative to misfortune in “traditional” societies.The initial argument is that the conceptual division and the modes of description and explanation of anthropology suffer from serious weaknesses: these concepts cannot serve to understand empirical phenomena ; they rely on a confused and erroneous conception of the different domains involved and the causalities between them; and they use simplistic (...) hypotheses about the existence and causal status of the entities that usually form the ultima ratio of anthropological reasoning. These entities would directly “cause” other individual representations or behaviors. This simplification also affects the analysis of states of belief in these entities, to which individuals would supposedly “adhere”.I argue here that the cognitivist approach, within a “methodological individualism” framework, provides a more adequate description of phenomena observed in the field. This enables the various levels and domains to be more finely defined. The analysis of “typical” utterances and inferences in a “tranditional” society, the Senufo of the Ivory Coast, is here used to clarify these anthropological problems. Two levels can be distinguished: a priori representations, which are underdetermined, enabling them to occur within valid inferences; perception and/or action, which obeys different cognitive constraints. The existential status of unobservable entities appearing in causal inferences is not equivalent depending on whether they are determined as antecedent or consequent.This paper suggests a theory of interpretive processes and beliefs having flexible references, because they are incomplete and domain-specific. It allows a comparison with facts observed in Western societies. It is also in contrast to the ordinary conception of religious states of belief — i.e., these states would be purely psychological, states of “adherence,” collective, autonomous, obligatory, part of a systemized set of knowledge; collective notions would here logically precede individual representations. (shrink)

Twenty years ago Leif Lewin made the case that altruistic motives are more common than selfish motives among voters, politicians, and bureaucrats. We propose that motives and beliefs emerge as reactions to immediate feedback from technical-causal, material-economic, and moral-social aspects of the political task environment. In the absence of certain kinds of technical-causal and material-economic feedback, moral-social feedback leads individuals to the altruism Lewin documents, but also to righteousness (moralized regard for the in-group and disregard for the out-group) and myopia (...) (disregard for distant consequences). The mix of altruism, righteousness, and myopia increases the focus on winning the next high-stakes election rather than on discovering or enforcing socially productive institutions. (shrink)

Is logic normative for belief? A standard approach to answering this question has been to investigate bridge principles relating claims of logical consequence to norms for belief. Although the question is naturally an epistemic one, bridge principles have typically been investigated in isolation from epistemic debates over the correct norms for belief. In this paper we tackle the question of whether logic is normative for belief by proposing a Kripkean model theory accounting for the interaction between logical, doxastic, epistemic and (...) deontic notions and using this model theory to show which bridge principles are implied by epistemic norms that we have independent reason to accept, for example, theknowledge normand thetruth norm. We propose a preliminary theory of the interaction between logical, doxastic, epistemic and deontic notions that has among its commitments bridge principles expressing how logic is normative for belief. We also show how our framework suggests that logic is exceptionally normative. (shrink)

Bounded rationality gets a bad rap in epistemology. It is argued that theories of bounded rationality are overly context‐sensitive; conventionalist; or dependent on ordinary language (Carr, 2022; Pasnau, 2013). In this paper, I have three aims. The first is to set out and motivate an approach to bounded rationality in epistemology inspired by traditional theories of bounded rationality in cognitive science. My second aim is to show how this approach can answer recent challenges raised for theories of bounded rationality. My (...) third aim is to clarify the role of rational ideals in bounded rationality. (shrink)

Behavioral scientists routinely publish broad claims about human psychology and behavior in the world's top journals based on samples drawn entirely from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies. Researchers – often implicitly – assume that either there is little variation across human populations, or that these “standard subjects” are as representative of the species as any other population. Are these assumptions justified? Here, our review of the comparative database from across the behavioral sciences suggests both that there is (...) substantial variability in experimental results across populations and that WEIRD subjects are particularly unusual compared with the rest of the species – frequent outliers. The domains reviewed include visual perception, fairness, cooperation, spatial reasoning, categorization and inferential induction, moral reasoning, reasoning styles, self-concepts and related motivations, and the heritability of IQ. The findings suggest that members of WEIRD societies, including young children, are among the least representative populations one could find for generalizing about humans. Many of these findings involve domains that are associated with fundamental aspects of psychology, motivation, and behavior – hence, there are no obviousa priorigrounds for claiming that a particular behavioral phenomenon is universal based on sampling from a single subpopulation. Overall, these empirical patterns suggests that we need to be less cavalier in addressing questions ofhumannature on the basis of data drawn from this particularly thin, and rather unusual, slice of humanity. We close by proposing ways to structurally re-organize the behavioral sciences to best tackle these challenges. (shrink)

Behavioral scientists routinely publish broad claims about human psychology and behavior in the world's top journals based on samples drawn entirely from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies. Researchers – often implicitly – assume that either there is little variation across human populations, or that these “standard subjects” are as representative of the species as any other population. Are these assumptions justified? Here, our review of the comparative database from across the behavioral sciences suggests both that there is (...) substantial variability in experimental results across populations and that WEIRD subjects are particularly unusual compared with the rest of the species – frequent outliers. The domains reviewed include visual perception, fairness, cooperation, spatial reasoning, categorization and inferential induction, moral reasoning, reasoning styles, self-concepts and related motivations, and the heritability of IQ. The findings suggest that members of WEIRD societies, including young children, are among the least representative populations one could find for generalizing about humans. Many of these findings involve domains that are associated with fundamental aspects of psychology, motivation, and behavior – hence, there are no obviousa priorigrounds for claiming that a particular behavioral phenomenon is universal based on sampling from a single subpopulation. Overall, these empirical patterns suggests that we need to be less cavalier in addressing questions ofhumannature on the basis of data drawn from this particularly thin, and rather unusual, slice of humanity. We close by proposing ways to structurally re-organize the behavioral sciences to best tackle these challenges. (shrink)

Commentators agree that simple conclusions about a general base rate fallacy are not appropriate. It is more constructive to identify conditions under which base rates are differentially weighted. Commentators also agree that improving the ecological validity of the research is desirable, although this is less important to those interested exclusively in psychological processes. The philosophers and ecologists among the commentators offer a kinder perspective on base rate reasoning than the psychologists. My own perspective is that the interesting questions (both psychological (...) and normative/prescriptive) are best answered through empirical studies that use real-world performance standards. (shrink)

We have been oversold on the base rate fallacy in probabilistic judgment from an empirical, normative, and methodological standpoint. At the empirical level, a thorough examination of the base rate literature (including the famous lawyer–engineer problem) does not support the conventional wisdom that people routinely ignore base rates. Quite the contrary, the literature shows that base rates are almost always used and that their degree of use depends on task structure and representation. Specifically, base rates play a relatively larger role (...) in tasks where base rates are implicitly learned or can be represented in frequentist terms. Base rates are also used more when they are reliable and relatively more diagnostic than available individuating information. At the normative level, the base rate fallacy should be rejected because few tasks map unambiguously into the narrow framework that is held up as the standard of good decision making. Mechanical applications of Bayes's theorem to identify performance errors are inappropriate when (1) key assumptions of the model are either unchecked or grossly violated, and (2) no attempt is made to identify the decision maker's goals, values, and task assumptions. Methodologically, the current approach is criticized for its failure to consider how the ambiguous, unreliable, and unstable base rates of the real world are and should be used. Where decision makers' assumptions and goals vary, and where performance criteria are complex, the traditional Bayesian standard is insufficient. Even where predictive accuracy is the goal in commonly defined problems, there may be situations (e.g., informationally redundant environments) in which base rates can be ignored with impunity. A more ecologically valid research program is called for. This program should emphasize the development of prescriptive theory in rich, realistic decision environments. (shrink)

Base rates have no necessary relation to judgments that are not themselves probabilities. There is no logical imperative, for instance, that behavioral base rates must affect causal attributions or that base rate information should affect judgments of legal liability. Decision theorists should be cautious in arguing that base rates place normative constraints on judgments of anything other than posterior probabilities.

Abstract(1) The miscitations of seminal experiments in the base rate literature adds to the existing database of systematic miscitations of wellknown psychological experiments. These miscitations may be caused by a process of reconstructive remembering. (2) Representative design should be the methodological core of Koehler's call for ecologically valid research. This approach can benefit both basic and applied research.

Koehler's summary and critique of research on the base rate fallacy is cogent and persuasive. However, he may have overstated the case, and his suggestions for future research may be too restrictive. We agree that methodological approaches to this topic should be broadened, but we argue that experimental laboratory research and the Bayesian normative standard are useful and should not be abandoned.

This commentary presents a self-assessment inventory that will allow readers to determine their own attitude toward the base rate fallacy and its literature. The inventory is scientifically valid but not Medicare/Medicaid reimbursable.

When base rates are learned and used in an experiential manner subjects show better base rate use, perhaps because the implicit learning system is engaged. A causal framework in which base rates are relevant might also be necessary. Humans might thus perform better on more ecologically valid tasks, which are likely to contain those three components.

Base rates are vital in predicting violent criminal recidivism. However, both lay people given simulated prediction tasks and professionals milking real life predictions appear insensitive to variations in the base rate of violent recidivism. Although there are techniques to help decision makers attend to base rates, increased decision accuracy is better sought in improved actuarial models as opposed to improved clinicians.

Consistent with Koehler's position, we propose a generalization of the base rate fallacy and earlier conservatism literatures. In studies using both traditional tasks and new tasks based on ecologically valid base rates, our subjects typically underweight individuating information at least as much as they underweight base rates. The implications of cue consistency for averaging heuristics are discussed.

Koehler is right that base rate information is used, to various degrees, both in laboratory tasks and in everyday life. However, it is not time to turn our backs on laboratory tasks and focus solely on ecologically valid decision making. Tightly controlled experimental data are still needed to understandhowbase rate information is used, and how this varies among groups.

Any instance (i.e., event, behavior, trait) belongs to infinitely many reference classes, hence there are infinitely many base rates from which to choose. People clearly do not entertain all possible reference classes, however, so something must be limiting the search space. We suggest some possible mechanisms that determine which reference class is evoked for the purpose of judgment and decision.

The fallacy beneath the base rate fallacy is that we know what a base rate is. We talk as if base rates and individuating information were two different kinds of information. From a Bayesian perspective, however, the only difference between base rate and individuating information is – which comes first.

Koehler is right to argue for more nuanced interpretation of base rate anomalies. These anomalies are best understood in relation to a broader class of cognitive anomalies, which are important for theory and practice. Recognizing a need for more nuanced analysis should not be taken as a license for treating the effects as “explained away.”.

Koehler's target article attempts a balanced view of the relevance of knowledge of base rates to judgments of subjective or credal probability, but he is not sensitive enough to the difference between requiring and permitting the equation of probability judgments with base rates, the interaction between precision of base rate and reference class information, and the possibility of indeterminate probability judgment.

Two distinct issues are sometimes confused in the base rate literature: Why do people make logical mistakes in the assessment of probabilities? and why do subjects not use base rates the way experimenters do? The latter problem may often reflect differences in an implicit reference class rather than a disinclination to update a base rate by Bayes' theorem. Also important are considerations concerning the interaction of several potentially relevant base rates.

The underutilization of base rates is a consistent finding. The strong claim that base rates are ignored has been rejected and this needs no further emphasis. Following the path of “normal science,” research examines the conditions predicting changes in the degree of underutilization. A scientific revolution that might dethrone the heuristics and biases paradigm is not in sight.

Koehler's call for a reanalysis of the base rate fallacy is particularly important in the applied domain of accounting, since base rate data appear to be an important input for many accounting tasks. In this commentary I discuss the use of base rates in accounting and explain why more flexible standards of performance are important when judging the use of base rates.

This commentary discusses three points: (1) The implications of the fact that it is rational to ignore base rates if probabilities are estimated by frequencies from samples without missing data (natural sampling); (2) second order probabilities distributions are a plausible way to model imprecise probabilities; and (3) Bayesian networks represent a normative reference for multi-cue models of probabilistic inference.

Failure to consider base rate is regarded as potentially hazardous, mainly because its consideration is assumed to be determined solely by P(H/D), the probability of the individuating data if the hypothesis is true, and not at all by P(D/˜H), the probability if the hypothesis is false. However, when P(D/˜H) is unconfounded from P(D/H), it turns out to be the stronger determinant of base rate consideration.

Setting the two hypotheses of complete neglect and full use of base rates against each other is inappropriate. The proper question concerns the degree to which base rates are used (or neglected), and under what conditions. We outline alternative approaches and recommend regression analysis. Koehler's conclusion that we have been oversold on the base rate fallacy seems to be premature.

A recent study showed physicians' reasoning about a realistic case to be ignorant of base rate. It also showed physicians interpreting information pertinent to base rate differently, depending on whether it was presented early or late in the case. Although these adult reasoners might do better if given hints through talk of relative frequencies, this would not prove that they had no problem of base rate neglect.

The environment in which humans evolved is strongly and positively autocorrelated in space and time. Probabilistic judgments based on the assumption of independence may not yield evolutionarily adaptive behavior. A number of “faults” of human reasoning are not faulty under fuzzy arithmetic, a nonprobabilistic calculus of reasoning under uncertainty that may be closer to that underlying human decision making.

In contrast to traditional research on base-rate neglect, an ecologically-oriented research program would analyze the correspondence between cognitive algorithms and the nature of information in the environment. Bayesian computations turn out to be simpler when information is represented in frequency formats as opposed to the probability formats used in previous research. Frequency formats often enable even uninstructed subjects to perform Bayesian reasoning.

The base rate literature has an opposite twin in the social psychological literature on stereotypes, which concludes that people use their preexisting beliefs about probabilistic category attributes too much, rather than not enough. This ironic discrepancy arises because beliefs about category attributes enhance accuracy when the beliefs are accurate and diminish accuracy when they are not. To determine the accuracy of base rate/stereotype beliefs requires research that addresses specific content.

Abstract(1) There is enough contradictory evidence regarding the role of base rates in category learning to confirm the nonexistence of biases in such learning. (2) It is not always possible to activate statistical reasoning through frequentist representation. (3) It is necessary to use the concept of systematic processing in reconsidering the published work on biases.

The important question is how people process probabilistic information, not whether they process it in accordance with a normative model that we never should have expected them to be capable of following. Experience is not the cure, as widely thought, to problems with utilizing base rate information.

In The Uses of Argument, Stephen Toulmin proposed a model for the layout of arguments: claim, data, warrant, qualifier, rebuttal, backing. Since then, Toulmin’s model has been appropriated, adapted and extended by researchers in speech communications, philosophy and artificial intelligence. This book assembles the best contemporary reflection in these fields, extending or challenging Toulmin’s ideas in ways that make fresh contributions to the theory of analysing and evaluating arguments.