This experiment elicits beliefs about other people’s overconfidence and abilities. We find that most people believe that others are unbiased, and only few think that others are overconfident. There is a remarkable heterogeneity between these groups. Those people who think others are underconfident or unbiased are overconfident themselves. Those who think others are overconfident are underconfident themselves. Despite this heterogeneity, people overestimate on average the abilities of others as they do their own ability. One driving force behind this result is (...) the refusal to process information about oneself: not only does this lead to overestimation of one’s own ability, but by means of social projection also to overestimation of others’ abilities. (shrink)

Theories of subjective probability are viewed as formal languages for analyzing evidence and expressing degrees of belief. This article focuses on two probability langauges, the Bayesian language and the language of belief functions (Shafer, 1976). We describe and compare the semantics (i.e., the meaning of the scale) and the syntax (i.e., the formal calculus) of these languages. We also investigate some of the designs for probability judgment afforded by the two languages.

While there is now considerable anxiety about whether the psychological theory presupposed by virtue ethics is empirically sustainable, analogous issues have received little attention in the virtue epistemology literature. This paper argues that virtue epistemology encounters challenges reminiscent of those recently encountered by virtue ethics: just as seemingly trivial variation in context provokes unsettling variation in patterns of moral behavior, trivial variation in context elicits unsettling variation in patterns of cognitive functioning. Insofar as reliability is a condition on epistemic virtue, (...) we have reason to doubt that human beings possess the cognitive materials required for epistemic virtue, and thereby reason to think that virtue epistemology is threatened by skepticism. We conclude that while virtue epistemology has resources for addressing this challenge, exploiting these resources forces tradeoffs between empirical and normative adequacy. (shrink)

Decision makers often rely on expert opinion when making forecasts under uncertainty. In doing so, they confront two methodological challenges: the elicitation problem, which requires them to extract meaningful information from experts; and the aggregation problem, which requires them to combine expert opinion by resolving disagreements. Linear averaging is a justifiably popular method for addressing aggregation, but its robust simplicity makes two requirements on elicitation. First, each expert must offer probabilistically coherent forecasts; second, each expert must respond to all our (...) queries. In practice, human judges (even experts) may be incoherent, and may prefer to assess only the subset of events about which they are comfortable offering an opinion. In this paper, a new methodology is developed for combining expert assessment of chance. The method retains the conceptual and computational simplicity of linear averaging, but generalizes the standard approach by relaxing the requirements on expert elicitation. The method also enjoys provable performance guarantees, and in experiments with real-world forecasting data is shown to offer both computational efficiency and competitive forecasting gains as compared to rival aggregation methods. This paper is relevant to the practice of decision analysis, for it enables an elicitation methodology in which judges have freedom to choose the events they assess. (shrink)

Can there be good reasons for judging one set of probabilistic assertions more reliable than a second? There are many candidates for measuring "goodness" of probabilistic forecasts. Here, I focus on one such aspirant: calibration. Calibration requires an alignment of announced probabilities and observed relative frequency, e.g., 50 percent of forecasts made with the announced probability of.5 occur, 70 percent of forecasts made with probability.7 occur, etc. To summarize the conclusions: (i) Surveys designed to display calibration curves, from which a (...) recalibration is to be calculated, are useless without due consideration for the interconnections between questions (forecasts) in the survey. (ii) Subject to feedback, calibration in the long run is otiose. It gives no ground for validating one coherent opinion over another as each coherent forecaster is (almost) sure of his own long-run calibration. (iii) Calibration in the short run is an inducement to hedge forecasts. A calibration score, in the short run, is improper. It gives the forecaster reason to feign violation of total evidence by enticing him to use the more predictable frequencies in a larger finite reference class than that directly relevant. (shrink)