The paper takes a closer look at the role of knowledge and evidence in legal theory. In particular, the paper examines a puzzle arising from the evidential standard Preponderance of the Evidence and its application in civil procedure. Legal scholars have argued since at least the 1940s that the rule of the Preponderance of the Evidence gives rise to a puzzle concerning the role of statistical evidence in judicial proceedings, sometimes referred to as the Problem of Bare Statistical Evidence. While (...) this puzzle has led to the development of a multitude of accounts and approaches in the legal literature, I argue here that the problem can be resolved fairly straightforwardly within a knowledge-first framework. (shrink)

Commentators agree that simple conclusions about a general base rate fallacy are not appropriate. It is more constructive to identify conditions under which base rates are differentially weighted. Commentators also agree that improving the ecological validity of the research is desirable, although this is less important to those interested exclusively in psychological processes. The philosophers and ecologists among the commentators offer a kinder perspective on base rate reasoning than the psychologists. My own perspective is that the interesting questions (both psychological (...) and normative/prescriptive) are best answered through empirical studies that use real-world performance standards. (shrink)

We have been oversold on the base rate fallacy in probabilistic judgment from an empirical, normative, and methodological standpoint. At the empirical level, a thorough examination of the base rate literature (including the famous lawyer–engineer problem) does not support the conventional wisdom that people routinely ignore base rates. Quite the contrary, the literature shows that base rates are almost always used and that their degree of use depends on task structure and representation. Specifically, base rates play a relatively larger role (...) in tasks where base rates are implicitly learned or can be represented in frequentist terms. Base rates are also used more when they are reliable and relatively more diagnostic than available individuating information. At the normative level, the base rate fallacy should be rejected because few tasks map unambiguously into the narrow framework that is held up as the standard of good decision making. Mechanical applications of Bayes's theorem to identify performance errors are inappropriate when (1) key assumptions of the model are either unchecked or grossly violated, and (2) no attempt is made to identify the decision maker's goals, values, and task assumptions. Methodologically, the current approach is criticized for its failure to consider how the ambiguous, unreliable, and unstable base rates of the real world are and should be used. Where decision makers' assumptions and goals vary, and where performance criteria are complex, the traditional Bayesian standard is insufficient. Even where predictive accuracy is the goal in commonly defined problems, there may be situations (e.g., informationally redundant environments) in which base rates can be ignored with impunity. A more ecologically valid research program is called for. This program should emphasize the development of prescriptive theory in rich, realistic decision environments. (shrink)

Base rates have no necessary relation to judgments that are not themselves probabilities. There is no logical imperative, for instance, that behavioral base rates must affect causal attributions or that base rate information should affect judgments of legal liability. Decision theorists should be cautious in arguing that base rates place normative constraints on judgments of anything other than posterior probabilities.

Abstract(1) The miscitations of seminal experiments in the base rate literature adds to the existing database of systematic miscitations of wellknown psychological experiments. These miscitations may be caused by a process of reconstructive remembering. (2) Representative design should be the methodological core of Koehler's call for ecologically valid research. This approach can benefit both basic and applied research.

Koehler's summary and critique of research on the base rate fallacy is cogent and persuasive. However, he may have overstated the case, and his suggestions for future research may be too restrictive. We agree that methodological approaches to this topic should be broadened, but we argue that experimental laboratory research and the Bayesian normative standard are useful and should not be abandoned.

This commentary presents a self-assessment inventory that will allow readers to determine their own attitude toward the base rate fallacy and its literature. The inventory is scientifically valid but not Medicare/Medicaid reimbursable.

When base rates are learned and used in an experiential manner subjects show better base rate use, perhaps because the implicit learning system is engaged. A causal framework in which base rates are relevant might also be necessary. Humans might thus perform better on more ecologically valid tasks, which are likely to contain those three components.

Base rates are vital in predicting violent criminal recidivism. However, both lay people given simulated prediction tasks and professionals milking real life predictions appear insensitive to variations in the base rate of violent recidivism. Although there are techniques to help decision makers attend to base rates, increased decision accuracy is better sought in improved actuarial models as opposed to improved clinicians.

Consistent with Koehler's position, we propose a generalization of the base rate fallacy and earlier conservatism literatures. In studies using both traditional tasks and new tasks based on ecologically valid base rates, our subjects typically underweight individuating information at least as much as they underweight base rates. The implications of cue consistency for averaging heuristics are discussed.

Koehler is right that base rate information is used, to various degrees, both in laboratory tasks and in everyday life. However, it is not time to turn our backs on laboratory tasks and focus solely on ecologically valid decision making. Tightly controlled experimental data are still needed to understandhowbase rate information is used, and how this varies among groups.

Any instance (i.e., event, behavior, trait) belongs to infinitely many reference classes, hence there are infinitely many base rates from which to choose. People clearly do not entertain all possible reference classes, however, so something must be limiting the search space. We suggest some possible mechanisms that determine which reference class is evoked for the purpose of judgment and decision.

The fallacy beneath the base rate fallacy is that we know what a base rate is. We talk as if base rates and individuating information were two different kinds of information. From a Bayesian perspective, however, the only difference between base rate and individuating information is – which comes first.

Koehler is right to argue for more nuanced interpretation of base rate anomalies. These anomalies are best understood in relation to a broader class of cognitive anomalies, which are important for theory and practice. Recognizing a need for more nuanced analysis should not be taken as a license for treating the effects as “explained away.”.

Koehler's target article attempts a balanced view of the relevance of knowledge of base rates to judgments of subjective or credal probability, but he is not sensitive enough to the difference between requiring and permitting the equation of probability judgments with base rates, the interaction between precision of base rate and reference class information, and the possibility of indeterminate probability judgment.

Two distinct issues are sometimes confused in the base rate literature: Why do people make logical mistakes in the assessment of probabilities? and why do subjects not use base rates the way experimenters do? The latter problem may often reflect differences in an implicit reference class rather than a disinclination to update a base rate by Bayes' theorem. Also important are considerations concerning the interaction of several potentially relevant base rates.

The underutilization of base rates is a consistent finding. The strong claim that base rates are ignored has been rejected and this needs no further emphasis. Following the path of “normal science,” research examines the conditions predicting changes in the degree of underutilization. A scientific revolution that might dethrone the heuristics and biases paradigm is not in sight.

Koehler's call for a reanalysis of the base rate fallacy is particularly important in the applied domain of accounting, since base rate data appear to be an important input for many accounting tasks. In this commentary I discuss the use of base rates in accounting and explain why more flexible standards of performance are important when judging the use of base rates.

This commentary discusses three points: (1) The implications of the fact that it is rational to ignore base rates if probabilities are estimated by frequencies from samples without missing data (natural sampling); (2) second order probabilities distributions are a plausible way to model imprecise probabilities; and (3) Bayesian networks represent a normative reference for multi-cue models of probabilistic inference.

Failure to consider base rate is regarded as potentially hazardous, mainly because its consideration is assumed to be determined solely by P(H/D), the probability of the individuating data if the hypothesis is true, and not at all by P(D/˜H), the probability if the hypothesis is false. However, when P(D/˜H) is unconfounded from P(D/H), it turns out to be the stronger determinant of base rate consideration.

Setting the two hypotheses of complete neglect and full use of base rates against each other is inappropriate. The proper question concerns the degree to which base rates are used (or neglected), and under what conditions. We outline alternative approaches and recommend regression analysis. Koehler's conclusion that we have been oversold on the base rate fallacy seems to be premature.

A recent study showed physicians' reasoning about a realistic case to be ignorant of base rate. It also showed physicians interpreting information pertinent to base rate differently, depending on whether it was presented early or late in the case. Although these adult reasoners might do better if given hints through talk of relative frequencies, this would not prove that they had no problem of base rate neglect.

The environment in which humans evolved is strongly and positively autocorrelated in space and time. Probabilistic judgments based on the assumption of independence may not yield evolutionarily adaptive behavior. A number of “faults” of human reasoning are not faulty under fuzzy arithmetic, a nonprobabilistic calculus of reasoning under uncertainty that may be closer to that underlying human decision making.

In contrast to traditional research on base-rate neglect, an ecologically-oriented research program would analyze the correspondence between cognitive algorithms and the nature of information in the environment. Bayesian computations turn out to be simpler when information is represented in frequency formats as opposed to the probability formats used in previous research. Frequency formats often enable even uninstructed subjects to perform Bayesian reasoning.

The base rate literature has an opposite twin in the social psychological literature on stereotypes, which concludes that people use their preexisting beliefs about probabilistic category attributes too much, rather than not enough. This ironic discrepancy arises because beliefs about category attributes enhance accuracy when the beliefs are accurate and diminish accuracy when they are not. To determine the accuracy of base rate/stereotype beliefs requires research that addresses specific content.

Abstract(1) There is enough contradictory evidence regarding the role of base rates in category learning to confirm the nonexistence of biases in such learning. (2) It is not always possible to activate statistical reasoning through frequentist representation. (3) It is necessary to use the concept of systematic processing in reconsidering the published work on biases.

The important question is how people process probabilistic information, not whether they process it in accordance with a normative model that we never should have expected them to be capable of following. Experience is not the cure, as widely thought, to problems with utilizing base rate information.

The primary aim of this chapter is to explain the nature of evidential reasoning, the characteristic difficulties encountered, and the tools to address these difficulties. Our focus is on evidential reasoning in criminal cases. There is an extensive scholarly literature on these topics, and it is a secondary aim of the chapter to provide readers the means to find their way in historical and ongoing debates.

This dissertation looks at a set of interconnected questions concerning the foundations of probability, and gives a series of interconnected answers. At its core is a piece of old-fashioned philosophical analysis, working out what probability is. Or equivalently, investigating the semantic question of what is the meaning of ‘probability’? Like Keynes and Carnap, I say that probability is degree of reasonable belief. This immediately raises an epistemological question, which degrees count as reasonable? To solve that in its full generality would (...) mean the end of human inquiry, so that won’t be attempted here. Rather I will follow tradition and merely investigate which sets of partial beliefs are coherent. -/- The standard answer to this question, what is commonly called the Bayesian answer, says that degrees of belief are coherent iff they form a probability function. I disagree with the way this is usually justified, but subject to an important qualification I accept the answer. The important qualification is that degrees of belief may be imprecise, or vague. Part one of the dissertation, chapters 1 to 6, looks largely at the consequences of this qualification for the semantic and epistemological questions already mentioned. It turns out that when we allow degrees of belief to be imprecise, we can discharge potentially fatal objections to some philosophically attractive theses. Two of these, that probability is degree of reasonable belief and that the probability calculus provides coherence constraints on partial beliefs, have been mentioned. Others include the claim, defended in chapter 4, that chance is probability given total history. -/- As well as these semantic and epistemological questions, studies of the foundations of probability usually include a detailed discussion of decision theory. For reasons set out in chapter 2, I deny we can gain epistemological insights from decision theory. Nevertheless, it is an interesting field to study on its own, and it might be expected that there would be decision theoretic consequences of allowing imprecise degrees of belief. As I show in part two, this expectation seems to be mistaken. Chapter 9 shows that there aren’t interesting consequences of this theory for decision theory proper, and chapters 10 and 11 show that Keynes’s attempt to use imprecision in degrees of belief to derive a distinctive theory of interest rates is unsound. -/- Chapters 7 and 8 provide a link between these two parts. In chapter 7 I look at some previous philosophical investigations into the effects of imprecision. In chapter 8 I develop what I take to be the best competitor to the theory defended here – a constructivist theory of probability. On this view degrees of belief are precise, but the relevant coherence constraint is a constructivist probability calculus. This view is, I think, mistaken, but the calculus has some intrinsic interest, and there are at least enough arguments for it to warrant a chapter-length examination. (shrink)

(1999). Francis Bacon's “VERULAMIUM” the common‐law template of the modern in english science and culture. Angelaki: Vol. 4, Judging the law, pp. 7-26.

This essay presents results about a deviation from independence measure called focused correlation . This measure explicates the formal relationship between probabilistic dependence of an evidence set and the incremental confirmation of a hypothesis, resolves a basic question underlying Peter Klein and Ted Warfield's ‘truth-conduciveness’ problem for Bayesian coherentism, and provides a qualified rebuttal to Erik Olsson's claim that there is no informative link between correlation and confirmation. The generality of the result is compared to recent programs in Bayesian epistemology (...) that attempt to link correlation and confirmation by utilizing a conditional evidential independence condition. Several properties of focused correlation are also highlighted. Introduction Correlation Measures 2.1 Standard covariance and correlation measures 2.2 The Wayne–Shogenji measure 2.3 Interpreting correlation measures 2.4 Correlation and evidential independence Focused Correlation Conclusion Appendix CiteULike Connotea Del.icio.us What's this? (shrink)

In place of the traditional epistemological view of knowledge as justified true belief we argue that artificial intelligence and law needs an evidence-based epistemology according to which scientific knowledge is based on critical analysis of evidence using argumentation. This new epistemology of scientific evidence (ESE) models scientific knowledge as achieved through a process of marshaling evidence in a scientific inquiry that results in a convergence of scientific theories and research results. We show how a dialogue interface of argument from expert (...) opinion, along with its set of critical questions, provides the argumentation component of the ESE. It enables internal scientific knowledge to be translated over into a wider arena in which individual nonexpert citizens and groups can make use of it. The external component shows how evidence is presented and used in a legal procedural setting that includes fact-finding, weighing the credibility of expert witnesses, and critical questioning of arguments. The paper critically reviews the standards of admissibility of scientific evidence using the ESE. (shrink)

Even though tools for identifying and analyzing arguments are now in wide use in the field of argumentation studies, so far there is a paucity of resources for evaluating real arguments, aside from using deductive logic or Bayesian rules that apply to inductive arguments. In this paper it is shown that recent developments in artificial intelligence in the area of computational systems for modeling defeasible argumentation reveal a different approach that is currently making interesting progress. It is shown how these (...) systems provide the general outlines for a system of argument evaluation that can be applied to legal arguments as well as everyday conversational arguments to assist a user to evaluate an argument. (shrink)

Evidentiary propositions E 1 and E 2, each p-positively relevant to some hypothesis H, are mutually corroborating if p > p, i = 1, 2. Failures of such mutual corroboration are instances of what may be called the corroboration paradox. This paper assesses two rather different analyses of the corroboration paradox due, respectively, to John Pollock and Jonathan Cohen. Pollock invokes a particular embodiment of the principle of insufficient reason to argue that instances of the corroboration paradox are of negligible (...) probability, and that it is therefore defeasibly reasonable to assume that items of evidence positively relevant to some hypothesis are mutually corroborating. Taking a different approach, Cohen seeks to identify supplementary conditions that are sufficient to ensure that such items of evidence will be mutually corroborating, and claims to have identified conditions which account for most cases of mutual corroboration. Combining a proposed common framework for the general study of paradoxes of positive relevance with a simulation experiment, we conclude that neither Pollock’s nor Cohen’s claims stand up to detailed scrutiny. I am quite prepared to be told…”oh, that is an extreme case: it could never really happen!” Now I have observed that this answer is always given instantly, with perfect confidence, and without any examination of the proposed case. It must therefore rest on some general principle: the mental process being something like this—“I have formed a theory. This case contradicts my theory. Therefore, this is an extreme case, and would never occur in practice.”Rev. Charles L. Dodgson. (shrink)

The paper provides a conceptual distinction between evidence assessment criteria and standards of proof. Evidence must be assessed in order to check whether it satisfies a relevant standard of proof, and the assessment is operated with some criterion; so both criteria and standards are necessary for fact-finding. In addition to this conceptual point, the article addresses three main questions: Why do some scholars and decision-makers take assessment criteria as standards of proof and vice versa? Why do systems differ as to (...) criteria and standards? How can a system work if it neglects one of these things? The answers to the first and second question come from the historical and procedural differences between the systems. The answer to the third focuses on the functional connection between criteria and standards. (shrink)

This target article presents a new computational theory of explanatory coherence that applies to the acceptance and rejection of scientific hypotheses as well as to reasoning in everyday life, The theory consists of seven principles that establish relations of local coherence between a hypothesis and other propositions. A hypothesis coheres with propositions that it explains, or that explain it, or that participate with it in explaining other propositions, or that offer analogous explanations. Propositions are incoherent with each other if they (...) are contradictory, Propositions that describe the results of observation have a degree of acceptability on their own. An explanatory hypothesis is acccpted if it coheres better overall than its competitors. The power of the seven principles is shown by their implementation in a connectionist program called ECHO, which treats hypothesis evaluation as a constraint satisfaction problem. Inputs about the explanatory relations are used to create a network of units representing propositions, while coherende and incoherence relations are encoded by excitatory and inbihitory links. ECHO provides an algorithm for smoothly integrating theory evaluation based on considerations of explanatory breadth, simplicity, and analogy. It has been applied to such important scientific cases as Lovoisier's argument for oxygen against the phlogiston theory and Darwin's argument for evolution against creationism, and also to cases of legal reasoning. The theory of explanatory coherence has implications for artificial intelligence, psychology, and philosophy. (shrink)