We argue that existing learning algorithms are often poorly equipped to solve problems involving a certain type of important and widespread regularity that we call “type-2 regularity.” The solution in these cases is to trade achieved representation against computational search. We investigate several ways in which such a trade-off may be pursued including simple incremental learning, modular connectionism, and the developmental hypothesis of “representational redescription.”.

Some regularities enjoy only an attenuated existence in a body of training data. These are regularities whose statistical visibility depends on some systematic recoding of the data. The space of possible recodings is, however, infinitely large – it is the space of applicable Turing machines. As a result, mappings that pivot on such attenuated regularities cannot, in general, be found by brute-force search. The class of problems that present such mappings we call the class of “type-2 problems.” Type-1 problems, by (...) contrast, present tractable problems of search insofar as the relevant regularities can be found by sampling the input data as originally coded. Type-2 problems, we suggest, present neither rare nor pathological cases. They are rife in biologically realistic settings and in domains ranging from simple animat behaviors to language acquisition. Not only are such problems rife – they are standardly solved! This presents a puzzle. How, given the statistical intractability of these type-2 cases, does nature turn the trick? One answer, which we do not pursue, is to suppose that evolution gifts us with exactly the right set of recoding biases so as to reduce specific type-2 problems to type-1 mappings. Such a heavy-duty nativism is no doubt sometimes plausible. But we believe there are other, more general mechanisms also at work. Such mechanisms provide general strategies for managing problems of type-2 complexity. Several such mechanisms are investigated. At the heart of each is a fundamental ploy – namely, the maximal exploitation of states of representation already achieved by prior, simpler learning so as to reduce the amount of subsequent computational search. Such exploitation both characterizes and helps make unitary sense of a diverse range of mechanisms. These include simple incremental learning, modular connectionism, and the developmental hypothesis of “representational redescription”. In addition, the most distinctive features of human cognition – language and culture – may themselves be viewed as adaptations enabling this representation/computation trade-off to be pursued on an even grander scale. (shrink)

Simon’s notion of bounded rationality is deeply intertwined with his activity as a cognitive psychologist and founder of so-called cognitivism, a mainstream approach in cognitive psychology until the 1980s. Cognitivism, understood as ‘symbolic information processing,’ provided the first cognitive psychology foundation to bounded rationality. Has bounded rationality since then fully followed the development of cognitive psychology beyond symbolic information processing in the post-Simonian era? To answer this question, this paper focuses on Simon’s opposition during the 1990s to a new view (...) of cognition called situated cognition, which has since put into question the entire view in cognitive psychology of humans as symbolic information processors. This paper then reads the cognitivism/situated cognition debate through the lens of current bounded rationality research in economics, in order to inquire into whether it has tackled the issues in that controversy; to envisage possible new foundations for a cognitive psychology-based bounded rationality. (shrink)

Vera & Simon (1993a) have argued that the theories and methods known as situated action or situativity theory are compatible with the assumptions and methodology of the physical symbol systems hypothesis and do not require a new approach to the study of cognition. When the central criterion of computational universality is added to the loose definition of a symbol system which Vera and Simon provide, it becomes apparent that there are important incompatibilities between the two approaches such that situativity theory (...) cannot be subsumed within the symbol systems approach. Symbol systems and situativity theoretic approaches are, and should be seen to be, competing approaches to the study of cognition. (shrink)