Reviews evidence which suggests that there may be little or no direct introspective access to higher order cognitive processes. Ss are sometimes unaware of the existence of a stimulus that importantly influenced a response, unaware of the existence of the response, and unaware that the stimulus has affected the response. It is proposed that when people attempt to report on their cognitive processes, that is, on the processes mediating the effects of a stimulus on a response, they do not do (...) so on the basis of any true introspection. Instead, their reports are based on a priori, implicit causal theories, or judgments about the extent to which a particular stimulus is a plausible cause of a given response. This suggests that though people may not be able to observe directly their cognitive processes, they will sometimes be able to report accurately about them. Accurate reports will occur when influential stimuli are salient and are plausible causes of the responses they produce, and will not occur when stimuli are not salient or are not plausible causes. (shrink)

In this paper, we present a competency model for complex problem solving by building on the categories of Knowledge, Skills, Abilities, and Other components. We highlight domain-general and domain-specific components in each of these categories, review established conceptualizations of CPS, and present a new model of CPS competency that is meant to provide a starting point for systematic research on training and assessment. The model highlights the idea that complex problems differ with regard to the KSAO components they demand from (...) a problem solver and that performance in one problem does not necessarily predict performance in a different problem. Implications for research on the training and assessment of CPS competency are discussed, and a selection of well-established tests for various components of the KSAO model is provided. (shrink)

In dynamic system control, cognitive mechanisms and abilities underlying performance may vary depending on the nature of the task. We therefore investigated the effects of system structure and its interaction with cognitive abilities on system control performance. A sample of 127 university students completed a series of different system control tasks that were manipulated in terms of system size and recurrent feedback, either with or without a cognitive load manipulation. Cognitive abilities assessed included reasoning ability, working memory capacity, and cognitive (...) reflection. System size and recurrent feedback affected overall performance as expected. Overall, the results support that cognitive ability is a good predictor of performance in dynamic system control tasks but predictiveness is reduced when the system structure contains recurrent feedback. We discuss this finding from a cognitive processing perspective as well as its implications for individual differences research in dynamic systems. (shrink)

In this paper, we bring together research on complex problem solving with that on motivational psychology about goal setting. Complex problems require motivational effort because of their inherent difficulties. Goal Setting Theory has shown with simple tasks that high, specific performance goals lead to better performance outcome than do-your-best goals. However, in complex tasks, learning goals have proven more effective than performance goals. Based on the Zurich Resource Model, so-called motto-goals should activate a person’s resources through positive affect. It was (...) found that motto-goals are effective with unpleasant duties. Therefore, we tested the hypothesis that motto-goals outperform learning and performance goals in the case of complex problems. A total of N = 123 subject participated in the experiment. In dependence of their goal condition, subjects developed a personal motto, learning, or performance goal. This goal was adapted for the computer-simulated complex scenario Tailorshop, where subjects worked as managers in a small fictional company. Other than expected, there was no main effect of goal condition for the management performance. An unexpected gender effect revealed better performance for men than women, pointing to a potential stereotype threat. As hypothesized, motto goals led to higher positive and lower negative affect than the other two goal types. Even though positive affect decreased and negative affect increased in all three groups during Tailorshop completion, participants with motto goals reported the lowest rates of negative affect. Exploratory analyses investigated the role of affect in complex problem solving via mediational analyses and the influence of goal type on perceived goal attainment. (shrink)

The assumption that positive affect leads to a better performance in simple cognitive tasks has become well established. We address the question whether positive and negative emotions differentially influence performance in complex problem-solving in the same way. Emotions were induced by positive or negative feedback in 74 participants who had to manage a computer-simulated complex problem-solving scenario. Results show that overall scenario performance is not affected, but positive and negative emotions elicit distinguishable problem-solving strategies: Participants with negative emotions are more (...) focused on the seeking and use of information. We discuss methodological requirements for investigating emotion influences in complex and dynamic cognitive tasks. (shrink)

In the last 20 years, a stream of research emerged under the label of „complex problem solving“ (CPS). This research was intended to describe the way people deal with complex, dynamic, and intransparent situations. Complex computer-simulated scenarios were as stimulus material in psychological experiments. This line of research lead to subtle insights into the way how people deal with complexity and uncertainty. Besides these knowledge-rich, realistic, intransparent, complex, dynamic scenarios with many variables, a second line of research used more simple, (...) knowledge-lean scenarios with a low number of variables („minimal complex systems“, MCS) that have been proposed recently in problem-solving research for the purpose of educational assessment. In both cases, the idea behind the use of microworlds is to increase validity of problem solving tasks by presenting interactive environments that can be explored and controlled by participants while pursuing certain action goals. The main argument presented here is: both types of systems - CPS and MCS – can only be dealt with successfully if causal dependencies between input and output variables are identified and used for system control. System knowledge is necessary for control and intervention. But CPS and MCS differ in their way of how causal dependencies are identified and how the mental model is constructed; therefore, they cannot be compared directly to each other with respect to the cognitive processes that are necessary for solving the tasks. Knowledge-poor MCS tasks address only a small fraction of the cognitive processes and structures needed for knowledge-rich CPS situations. (shrink)

In this paper we propose the multiple complex systems approach for assessing domain-general complex problem-solving skills and its processes knowledge acquisition and knowledge application. After defining the construct and the formal frameworks for describing complex problems, we emphasise some of the measurement issues inherent in assessing CPS skills with single tasks. With examples of the MicroDYN test and the MicroFIN test, we show how to adequately score problem-solving skills by using multiple tasks. We discuss implications for problem-solving research and the (...) assessment of CPS skills in general. (shrink)

The assumption that positive affect leads to a better performance in simple cognitive tasks has become well established. We address the question whether positive and negative emotions differentially influence performance in complex problem-solving in the same way. Emotions were induced by positive or negative feedback in 74 participants who had to manage a computer-simulated complex problem-solving scenario. Results show that overall scenario performance is not affected, but positive and negative emotions elicit distinguishable problem-solving strategies: Participants with negative emotions are more (...) focused on the seeking and use of information. We discuss methodological requirements for investigating emotion influences in complex and dynamic cognitive tasks. (shrink)

In this rejoinder, we respond to two commentaries on the study by Greiff, S.; Stadler, M.; Sonnleitner, P.; Wolff, C.; Martin, R. Sometimes less is more: Comparing the validity of complex problem solving measures. Intelligence 2015, 50, 100–113. The study was the first to address the important comparison between a classical measure of complex problem solving (CPS) and the more recent multiple complex systems (MCS) approach regarding their validity. In the study, we investigated the relations between one classical microworld as (...) the initially developed method (here, the Tailorshop) with three more recently developed multiple complex systems (MCS; here, MicroDYN, Genetics Lab, and MicroFIN) tests. We found that the MCS tests showed higher levels of convergent validity with each other than with the Tailorshop even after reasoning was controlled for, thus empirically distinguishing between the two approaches. The commentary by Kretzschmar and the commentary by Funke, Fischer, and Holt expressed several concerns with how our study was conducted, our data was analyzed, and our results were interpreted. Whereas we acknowledge and agree with some of the more general statements made in these commentaries, we respectfully disagree with others, or we consider them to be at least partially in contrast with the existing literature and the currently available empirical evidence. (shrink)

This study examined whether experience with different computer-based complex problem situations would improve complex problem solving competence in an unknown problem situation. Therefore, university students took part in a control group study. They were trained in five different complex problem situations for up to 7 hr, and their performance was tested in a sixth complex problem situation. The data analyses revealed that the training influenced the CPS process of knowledge acquisition. However, the CPS process of knowledge application was not impacted (...) by experience with other problem situations. Implications for the concept of CPS as a trainable competence as well as the training of CPS in general are discussed. (shrink)

The question “How can humans learn efficiently to make decisions in a complex, dynamic, and uncertain environment” is still a very open question. We investigate what effects arise when feedback is given in a computer-simulated microworld that is controlled by participants. This has a direct impact on training simulators that are already in standard use in many professions, e.g., for flight simulators for pilots, and a potential impact on a better understanding of human decision making in general. Our study is (...) based on a benchmark microworld with an economic framing, the IWR Tailorshop. N=94 participants played four rounds of the microworld, each 10 months, via a web interface. We propose a new approach to quantify performance and learning, which is based on a mathematical model of the microworld and optimization. Six participant groups receive different kinds of feedback in a training phase, then results in a performance phase without feedback are analyzed. As a main result, feedback of optimal solutions in training rounds improved model knowledge, early learning, and performance, especially when this information is encoded in a graphical representation. (shrink)

Empirical methods of self-description, think aloud protocols and introspection have been extensively criticized or neglected in behaviorist and cognitivist psychology. Their methodological value has been fundamentally questioned since there apparently is no suficient proof for their validity. However, the major arguments against self-description can be critically reviewed by theoretical psychology. This way, these methods’ empirical value can be redeemed. Furthermore, self-descriptive methods can be updated by the use of contemporary media technology. In order to support the promising perspectives for future (...) empirical research in the field of cognitive psychology, Live Streaming is proposed as a viable data source. Introducing this new paradigm, this paper presents some of the formal constituents and accessible contents of Live Streaming, and relates them to established forms of empirical research. By its structure and established usage, Live Streaming bears remarkable resemblances to the traditional methods of self-description, yet it also adds fruitful new features of use. On the basis of its qualities, the possible benefits that appear to be feasible in comparison with the traditional methods of self-description are elaborated, such as Live Streaming’s ecological validity. Ultimately, controversial theoretical concepts, such as those in phenomenology and cultural-historical psychology, are adopted to sketch further potential benefits of the utility of Live Streaming in current psychology debates. (shrink)

The current study had three goals: to investigate strategies, tactics, and errors as predictors of success and failure under uncertainty following the dynamic decision making and complex problem solving framework; to use observation and to examine its reliability and potential as a data collection method when using microworlds; and to investigate the applicability and validity of a microworld developed in the West, to an Asian sample. One hundred three participants in the Philippines took the role of fire chief in the (...) microworld WINFIRE. Their strategies, tactics, and errors were observed and coded by experimenters as they worked individually on the simulation twice. Results showed that DDM strategies, tactics, and errors predicted success and failure in WINFIRE, and strategies and tactics that led to success increased while errors decreased over time; strategies, tactics, and errors can be validly assessed through observation by experimenters, specifically that two types of decision makers were identified: the active, flexible, and big picture planners and the slow or cautious, and singlefocused decision makers; these findings together with participants’ survey ratings speak for the applicability of the microworld in an East Asian sample and for its validity. Findings are potentially relevant for experts and for training programs, highlighting the benefits of virtual environments during DDM. (shrink)