Influencing students’ educational achievements first requires understanding the underlying processes that lead to variation in students’ performance. Researchers are therefore increasingly interested in analyzing the differences in behavior displayed in educational assessments rather than merely assessing their outcomes. Such analyses provide valuable information on the differences between successful and unsuccessful students and help to design appropriate interventions. Complex problem solving (CPS) tasks have proven to provide particularly rich process data as they allow for a multitude of behaviors several of which (...) can lead to a successful performance. So far, this data has often been analyzed on a rather aggregated level looking at an average number of actions or predefined strategies. with only a few articles investigating the specific actions performed. In this paper, we illustrate and discuss a more detailed report the results of an exploratory analysis of CPS process data based on the specific actions performed.log-files that is aimed at distinguishing between students that applied the correct strategy to a problem but failed to solve it and those applying the strategy successfully. In that, the sequence of behavior displayed is reduced to interpretable parts (n-grams) that allow searching for meaningful differences between the two groups of students. This level of analysis allows finding previously undefined or unknown patterns within the data and increases our understanding of the processes underlying successful problem -solving behavior even further. (shrink)

The assumption that positive affect leads to a better performance in simple cognitive tasks has become well established. We address the question whether positive and negative emotions differentially influence performance in complex problem-solving in the same way. Emotions were induced by positive or negative feedback in 74 participants who had to manage a computer-simulated complex problem-solving scenario. Results show that overall scenario performance is not affected, but positive and negative emotions elicit distinguishable problem-solving strategies: Participants with negative emotions are more (...) focused on the seeking and use of information. We discuss methodological requirements for investigating emotion influences in complex and dynamic cognitive tasks. (shrink)

The assumption that positive affect leads to a better performance in simple cognitive tasks has become well established. We address the question whether positive and negative emotions differentially influence performance in complex problem-solving in the same way. Emotions were induced by positive or negative feedback in 74 participants who had to manage a computer-simulated complex problem-solving scenario. Results show that overall scenario performance is not affected, but positive and negative emotions elicit distinguishable problem-solving strategies: Participants with negative emotions are more (...) focused on the seeking and use of information. We discuss methodological requirements for investigating emotion influences in complex and dynamic cognitive tasks. (shrink)

Multiple goals balancing is an important but not yet fully validated dimension of complex problem solving (CPS). The present study used process data to explore how solvers clarify goals, set priorities, and balance conflicting goals. We extracted behavioral indicators of goal pursuit from the log data of 3,201 students on the third subtask of the “Ticket” task in the PISA 2012 CPS test. Cluster analysis was used to identify 10 groups that varied in goal pursuit behavior. Logistics and least-squares regression (...) analysis were used to explore how goal pursuit affected task scores and CPS proficiency. The results showed that competent solvers clarified goals and weighed priorities more effectively. They also made trade-offs between conflicting goals. The importance of theoretically-driven log data analysis and coping strategies in the face of multiple goals conflict scenarios was discussed. (shrink)

The present study discusses findings that replicate and extend the original work of Burns and Vollmeyer (2002), which showed that performance in problem solving tasks was more accurate when people were engaged in a non-specific goal than in a specific goal. The main innovation here was to examine the goal specificity effect under both observation-based and conventional action-based learning conditions. The findings show that goal specificity affects the accuracy of problem solving in the same way, both when the learning stage (...) of the task is observationbased and when it is action-based. Additionally, the findings show that, when instructions do not promote goal specificity, observation-based problem solving is as effective as action-based problem solving. (shrink)

Three general issues emerge from the preceding papers: a confusion between judgement and related activities such as decision making, problem solving, and attitudes; differences in the underlying assumptions about the nature of judgement; and different approaches for testing the adequacy of theories human judgement. The implications of these issues for studying human judgement processes and for future research priorities in this area are briefly discussed.

This study examined whether experience with different computer-based complex problem situations would improve complex problem solving competence in an unknown problem situation. Therefore, university students took part in a control group study. They were trained in five different complex problem situations for up to 7 hr, and their performance was tested in a sixth complex problem situation. The data analyses revealed that the training influenced the CPS process of knowledge acquisition. However, the CPS process of knowledge application was not impacted (...) by experience with other problem situations. Implications for the concept of CPS as a trainable competence as well as the training of CPS in general are discussed. (shrink)

Complex problem solving can be interpreted as the number of psychological mechanisms that allow us to reach our targets in difficult situations, that can be classified as complex, dynamic, non-transparent, interconnected, and multilayered, and also polytelic. The previous results demonstrated associations between the personality dimensions neuroticism, conscientiousness, and extraversion and problem-solving performance. However, there are no studies dealing with personality disorders in connection with CPS skills. Therefore, the current study examines a clinical sample consisting of people with personality and/or depressive (...) disorders. As we have data for all the potential personality disorders and also data from each patient regarding to potential depression, we meet the whole range from healthy to impaired for each personality disorder and for depression. We make use of a unique operationalization: CPS was surveyed in a simulation game, making use of the microworld approach. This study was designed to investigate the hypothesis that personality traits are related to CPS performance. Results show that schizotypal, histrionic, dependent, and depressive persons are less likely to successfully solve problems, while persons having the additional behavioral characteristics of resilience, action orientation, and motivation for creation are more likely to successfully solve complex problems. (shrink)

In dynamic system control, cognitive mechanisms and abilities underlying performance may vary depending on the nature of the task. We therefore investigated the effects of system structure and its interaction with cognitive abilities on system control performance. A sample of 127 university students completed a series of different system control tasks that were manipulated in terms of system size and recurrent feedback, either with or without a cognitive load manipulation. Cognitive abilities assessed included reasoning ability, working memory capacity, and cognitive (...) reflection. System size and recurrent feedback affected overall performance as expected. Overall, the results support that cognitive ability is a good predictor of performance in dynamic system control tasks but predictiveness is reduced when the system structure contains recurrent feedback. We discuss this finding from a cognitive processing perspective as well as its implications for individual differences research in dynamic systems. (shrink)

The previous papers raise a number of issues. How should we develop task typologies both to separate judgement from related cognitive tasks and to classify tasks within the judgement domain? Are there grounds for selecting between models of judgement when empirical tests fail to do so? What techniques can be used to find out more about the cognitive processes underlying judgement behaviour? I discuss these issues and give a brief assessment of the current state of play in this rapidly changing (...) area. (shrink)

In this paper we propose the multiple complex systems approach for assessing domain-general complex problem-solving skills and its processes knowledge acquisition and knowledge application. After defining the construct and the formal frameworks for describing complex problems, we emphasise some of the measurement issues inherent in assessing CPS skills with single tasks. With examples of the MicroDYN test and the MicroFIN test, we show how to adequately score problem-solving skills by using multiple tasks. We discuss implications for problem-solving research and the (...) assessment of CPS skills in general. (shrink)

Many situations require people to acquire knowledge about, and learn how to control, complex dynamic systems of inter-connected variables. Numerous studies have found that most problem solvers are unable to acquire complete knowledge of the underlying structure of a system through an unguided exploration of the system variables; additional instruction or guidance is required. This paper examines whether providing structural information following an unguided exploration also improves control performance, and the extent to which any improvements are moderated by problem solvers’ (...) fluid intelligence. A sample of 98 individuals attempted to discover the underlying structure of a computer-simulated complex dynamic system. After initially controlling the system with their limited knowledge, half of the sample received guidance concerning the actual underlying structure of the system. All participants then controlled the system again. In contrast to the results of previous studies, guidance did result in immediate improvements in control performance. Fluid intelligence moderated the extent to which problem solvers benefited from structural information. These results indicate that guidance in the form of structural information is critical in promoting knowledge and control over complex and dynamic systems. (shrink)

In the last 20 years, a stream of research emerged under the label of „complex problem solving“ (CPS). This research was intended to describe the way people deal with complex, dynamic, and intransparent situations. Complex computer-simulated scenarios were as stimulus material in psychological experiments. This line of research lead to subtle insights into the way how people deal with complexity and uncertainty. Besides these knowledge-rich, realistic, intransparent, complex, dynamic scenarios with many variables, a second line of research used more simple, (...) knowledge-lean scenarios with a low number of variables („minimal complex systems“, MCS) that have been proposed recently in problem-solving research for the purpose of educational assessment. In both cases, the idea behind the use of microworlds is to increase validity of problem solving tasks by presenting interactive environments that can be explored and controlled by participants while pursuing certain action goals. The main argument presented here is: both types of systems - CPS and MCS – can only be dealt with successfully if causal dependencies between input and output variables are identified and used for system control. System knowledge is necessary for control and intervention. But CPS and MCS differ in their way of how causal dependencies are identified and how the mental model is constructed; therefore, they cannot be compared directly to each other with respect to the cognitive processes that are necessary for solving the tasks. Knowledge-poor MCS tasks address only a small fraction of the cognitive processes and structures needed for knowledge-rich CPS situations. (shrink)

In this paper, we present a competency model for complex problem solving by building on the categories of Knowledge, Skills, Abilities, and Other components. We highlight domain-general and domain-specific components in each of these categories, review established conceptualizations of CPS, and present a new model of CPS competency that is meant to provide a starting point for systematic research on training and assessment. The model highlights the idea that complex problems differ with regard to the KSAO components they demand from (...) a problem solver and that performance in one problem does not necessarily predict performance in a different problem. Implications for research on the training and assessment of CPS competency are discussed, and a selection of well-established tests for various components of the KSAO model is provided. (shrink)

Real causal systems are complicated. Despite this, causal learning research has traditionally emphasized how causal relations can be induced on the basis of idealized events, i.e. those that have been mapped to binary variables and abstracted from time. For example, participants may be asked to assess the efficacy of a headache-relief pill on the basis of multiple patients who take the pill (or not) and find their headache relieved (or not). In contrast, the current study examines learning via interactions with (...) continuous dynamic systems, systems that include continuous variables that interact over time (and that can be continuously observed in real time by the learner). To explore such systems, we develop a new framework that represents a causal system as a network of stationary Gauss--Markov (``\OU'') processes and show how such \textit{OU networks} can express complex dynamic phenomena such as feedback loops and oscillations. To assess adult's abilities to learn such systems, we conducted an experiment in which participants were asked to identify the causal relationships of a number of OU networks, potentially carrying out multiple, temporally-extended interventions. We compared their judgments to a normative model for learning OU networks as well as a range of alternative and heuristic learning models from the literature. We found that, although participants exhibited substantial learning of such systems, they committed certain systematic errors. These successes and failures were best accounted for by a model that describes people as focusing on pairs of variables, rather than evaluating the evidence with respect to the full space of possible structural models. We argue that our approach provides both a principled framework for exploring the space of dynamic learning environments as well as new algorithmic insights into how people interact successfully with a continuous causal world. (shrink)

The assumption that positive affect leads to a better performance in simple cognitive tasks has become well established. We address the question whether positive and negative emotions differentially influence performance in complex problem-solving in the same way. Emotions were induced by positive or negative feedback in 74 participants who had to manage a computer-simulated complex problem-solving scenario. Results show that overall scenario performance is not affected, but positive and negative emotions elicit distinguishable problem-solving strategies: Participants with negative emotions are more (...) focused on the seeking and use of information. We discuss methodological requirements for investigating emotion influences in complex and dynamic cognitive tasks. (shrink)