This is a position paper concerning the role of empirical studies of human default reasoning in the formalization of AI theories of default reasoning. We note that AI motivates its theoretical enterprise by reference to human skill at default reasoning, but that the actual research does not make any use of this sort of information and instead relies on intuitions of individual investigators. We discuss two reasons theorists might not consider human performance relevant to formalizing default reasoning: (a) that intuitions (...) are sufficient to describe a model, and (b) that human performance in this arena is irrelevant to a competence model of the phenomenon. We provide arguments against both these reasons. We then bring forward three further considerations against the use of intuitions in this arena: (a) it leads to an unawareness of predicate ambiguity, (b) it presumes an understanding of ordinary language statements of typicality, and (c) it is similar to discredited views in other fields. We advocate empirical investigation of the range of human phenomena that intuitively embody default reasoning. Gathering such information would provide data with which to generate formal default theories and against which to test the claims of proposed theories. Our position is that such data are the very phenomena that default theories are supposed to explain. (shrink)

Two experiments examined the typicality structure of contrasting political categories. In Experiment 1, two separate groups of participants rated the typicality of 15 individuals, including political figures and media personalities, with respect to the categories Democrat or Republican. The relation between the two sets of ratings was negative, linear, and extremely strong, r = −.9957. Essentially, one category was treated as a mirror image of the other. Experiment 2 replicated this result, showing some boundary conditions, and extending the result to (...) liberal and conservative categories. The same method was applied to two other pairs of contrasting categories, healthy and junk foods, and male and female jobs. For those categories, the relation between contrasting pairs was weaker and there was less of a direct trade-off between typicality in one category versus typicality in its opposite. The results are discussed in terms of implications for political decision making and reasoning, and conceptual representation. (shrink)

Studies of categorical induction typically examine how belief in a premise (e.g., Falcons have an ulnar artery) projects on to a conclusion (e.g., Robins have an ulnar artery). We study induction in cases in which the premise is uncertain (e.g., There is an 80% chance that falcons have an ulnar artery). Jeffrey's rule is a normative model for updating beliefs in the face of uncertain evidence. In three studies we tested the descriptive validity of Jeffrey's rule and a related probability (...) theorem, the rule of total probability. Although these rules provided good approximations to mean judgments in some cases, the results from regression and correlation analyses suggest that participants focus on the parts of these rules that are associated with the highest overall probability. We relate our findings to rational models of judgment. (shrink)

We advance a model of human probability judgment and apply it to the design of an extrapolation algorithm. Such an algorithm examines a person's judgment about the likelihood of various statements and is then able to predict the same person's judgments about new statements. The algorithm is tested against judgments produced by thirty undergraduates asked to assign probabilities to statements about mammals.

Applying machine learning algorithms to automatically infer relationships between concepts from large-scale collections of documents presents a unique opportunity to investigate at scale how human semantic knowledge is organized, how people use it to make fundamental judgments (“How similar are cats and bears?”), and how these judgments depend on the features that describe concepts (e.g., size, furriness). However, efforts to date have exhibited a substantial discrepancy between algorithm predictions and human empirical judgments. Here, we introduce a novel approach to generating (...) embeddings for this purpose motivated by the idea that semantic context plays a critical role in human judgment. We leverage this idea by constraining the topic or domain from which documents used for generating embeddings are drawn (e.g., referring to the natural world vs. transportation apparatus). Specifically, we trained state-of-the-art machine learning algorithms using contextually-constrained text corpora (domain-specific subsets of Wikipedia articles, 50+ million words each) and showed that this procedure greatly improved predictions of empirical similarity judgments and feature ratings of contextually relevant concepts. Furthermore, we describe a novel, computationally tractable method for improving predictions of contextually-unconstrained embedding models based on dimensionality reduction of their internal representation to a small number of contextually relevant semantic features. By improving the correspondence between predictions derived automatically by machine learning methods using vast amounts of data and more limited, but direct empirical measurements of human judgments, our approach may help leverage the availability of online corpora to better understand the structure of human semantic representations and how people make judgments based on those. (shrink)

Both scientists and children make important structural discoveries, yet their computational underpinnings are not well understood. Structure discovery has previously been formalized as probabilistic inference about the right structural form—where form could be a tree, ring, chain, grid, etc.. Although this approach can learn intuitive organizations, including a tree for animals and a ring for the color circle, it assumes a strong inductive bias that considers only these particular forms, and each form is explicitly provided as initial knowledge. Here we (...) introduce a new computational model of how organizing structure can be discovered, utilizing a broad hypothesis space with a preference for sparse connectivity. Given that the inductive bias is more general, the model's initial knowledge shows little qualitative resemblance to some of the discoveries it supports. As a consequence, the model can also learn complex structures for domains that lack intuitive description, as well as predict human property induction judgments without explicit structural forms. By allowing form to emerge from sparsity, our approach clarifies how both the richness and flexibility of human conceptual organization can coexist. (shrink)