In this article we evaluate the BankXX program from several perspectives. BankXX is a case-based legal argument program that retrieves cases and other legal knowledge pertinent to a legal argument through a combination of heuristic search and knowledge-based indexing. The program is described in detail in a companion article in Artificial Intelligence and Law 4: 1--71, 1996. Three perspectives are used to evaluate BankXX:(1) classical information retrieval measures of precision and recall applied against a hand-coded baseline; (2) knowledge-representation and case-based (...) reasoning, where the baseline is provided by the functionality of a well-known case-based argument program, HYPO (Ashley, 1990); and (3) search, in which the performance of BankXX run with various parameter settings, for instance, resource limits, is compared. In this article we report on an extensive series of experiments performed to evaluate the program. We also describe two additional experiments concerning(1) the program's search behavior; and (2) the use of a modified form of precision and recall based on case similarity. Finally we offer some general conclusions that might be drawn from these particular experiments. (shrink)

A theory of analogy must describe how the meaning of an analogy is derived from the meanings of its parts. In the structure‐mapping theory, the interpretation rules are characterized as implicit rules for mapping knowledge about a base domain into a target domain. Two important features of the theory are (a) the rules depend only on syntactic properties of the knowledge representation, and not on the specific content of the domains; and (b) the theoretical framework allows analogies to be distinguished (...) cleanly from literal similarity statements, applications of abstractions, and other kinds of comparisons.Two mapping principles are described: (a) Relations between objects, rather than attributes of objects, are mapped from base to target; and (b) The particular relations mapped are determined by systematicity, as defined by the existence of higher‐order relations. (shrink)

We present a model of similarity‐based retrieval that attempts to capture three seemingly contradictory psychological phenomena: (a) structural commonalities are weighed more heavily than surface commonalities in similarity judgments for items in working memory; (b) in retrieval, superficial similarity is more important than structural similarity; and yet (c) purely structural (analogical) remindings e sometimes experienced. Our model, MAC/FAC, explains these phenomena in terms of a two‐stage process. The first stage uses a computationally cheap, non‐structural matcher to filter candidate long‐term memory (...) items. It uses content vectors, a redundant encoding of structured representations whose dot product estimates how well the corresponding structural representations will match. The second stage uses SME (structure‐mapping engine) to compute structural matches on the handful of items found by the first stage. We show the utility of the MAC/FAC model through a series of computational experiments: (a) We demonstrate that MAC/FAC can model patterns of access found in psychological data; (b) we argue via sensitivity analyses that these simulation results rely on the theory; and (c) we compare the performance of MAC/FAC with ARCS, an alternate model of similarity‐based retrieval, and demonstrate that MAC/FAC explains the data better than ARCS. Finally, we discuss limitations and possible extensions of the model, relationships with other recent retrieval models, and place MAC/FAC in the context of other recent work on the nature of similarity. (shrink)

The BankXX system models the process of perusing and gathering information for argument as a heuristic best-first search for relevant cases, theories, and other domain-specific information. As BankXX searches its heterogeneous and highly interconnected network of domain knowledge, information is incrementally analyzed and amalgamated into a dozen desirable ingredients for argument (called argument pieces), such as citations to cases, applications of legal theories, and references to prototypical factual scenarios. At the conclusion of the search, BankXX outputs the set of argument (...) pieces filled with harvested material relevant to the input problem situation.This research explores the appropriateness of the search paradigm as a framework for harvesting and mining information needed to make legal arguments. In this article, we describe how legal research fits the heuristic search framework and detail how this model is used in BankXX. We describe the BankXX program with emphasis on its representation of legal knowledge and legal argument. We describe the heuristic search mechanism and evaluation functions that drive the program. We give an extended example of the processing of BankXX on the facts of an actual legal case in BankXX's application domain — the good faith question of Chapter 13 personal bankruptcy law. We discuss closely related research on legal knowledge representation and retrieval and the use of search for case retrieval or tasks related to argument creation. Finally we review what we believe are the contributions of this research to the understanding of the diverse disciplines it addresses. (shrink)