Certain characteristics make machine learning a powerful tool for processing large amounts of data, and also particularly unsuitable for explanatory purposes. There are worries that its increasing use in science may sideline the explanatory goals of research. We analyze the key characteristics of ML that might have implications for the future directions in scientific research: epistemic opacity and the ‘theory-agnostic’ modeling. These characteristics are further analyzed in a comparison of ML with the traditional statistical methods, in order to demonstrate what (...) it is specifically that makes ML methodology substantially unsuitable for reaching explanations. The analysis is given broader philosophical context by connecting it with the views on the role of prediction and explanation in science, their relationship, and the value of explanation. We proceed to show, first, that ML disrupts the relationship between prediction and explanation commonly understood as a functional relationship. Then we show that the value of explanation is not exhausted in purely predictive functions, but rather has a ubiquitously recognized value for both science and everyday life. We then invoke two hypothetical scenarios with different degrees of automatization of science, which help test our intuitions on the role of explanation in science. The main question we address is whether ML will reorient or otherwise impact our standard explanatory practice. We conclude with a prognosis that ML would diversify science into purely predictively oriented research based on ML-like techniques and, on the other hand, remaining faithful to anthropocentric research focused on the search for explanation. (shrink)

Some claim that digital phenotyping will revolutionize understanding of human psychology and experience and significantly promote human wellbeing. This paper investigates the nature of digital phenotyping in relation to its alleged promise. Unlike most of the literature to date on philosophy and digital phenotyping, which has focused on its ethical aspects, this paper focuses on its epistemic and methodological aspects. The paper advances a tetra-taxonomy involving four scenario types in which knowledge may be acquired from human “digitypes” by digital phenotyping. (...) These scenarios comprise two causal relations and a correlative and constitutive relation that can exist between information generated by digital systems/devices on the one hand and psychological or behavioral phenomena on the other. The paper describes several modes of inference involved in deriving knowledge within these scenarios. After this epistemic mapping, the paper analyzes the possible knowledge potential and limitations of digital phenotyping. It finds that digital phenotyping holds promise of delivering insight into conditions and states as well producing potentially new psychological categories. It also argues that care must be taken that digital phenotyping does not make unwarranted conclusions and is aware of potentially distorting effects in digital sensing and measurement. If digital phenotyping is to truly revolutionize knowledge of human life, it must deliver on a range of fronts, including making accurate forecasts and diagnoses of states and behaviors, providing causal explanations of these phenomena, and revealing important constituents of human conditions, psychology, and experience. (shrink)

Bayesian confirmation theory is a leading theory to decide the confirmation/refutation of a hypothesis based on probability calculus. While it may be much discussed in philosophy of science, is it actually practiced in terms of hypothesis testing by scientists? Since the assignment of some of the probabilities in the theory is open to debate and the risk of making the wrong decision is unknown, many scientists do not use the theory in hypothesis testing. Instead, they use alternative statistical tests that (...) can measure the risk or the reliability in decision making, circumventing some of the theoretical problems in practice. Therefore, the theory is not very popular in hypothesis testing among scientists at present. However, there are some proponents of Bayesian hypothesis testing, and software packages are made available to accelerate utilization by scientists. Time will tell whether Bayesian confirmation theory can become both a leading theory and a widely practiced method. In addition, this theory can be used to model the belief of scientists when testing hypotheses. (shrink)

Self-regulated learning is critical for learning across tasks, domains, and contexts. Despite its importance, research shows that not all learners are equally skilled at accurately and dynamically monitoring and regulating their self-regulatory processes. Therefore, learning technologies, such as intelligent tutoring systems, have been designed to measure and foster SRL. This paper presents an overview of over 10 years of research on SRL with MetaTutor, a hypermedia-based ITS designed to scaffold college students’ SRL while they learn about the human circulatory system. (...) MetaTutor’s architecture and instructional features are designed based on models of SRL, empirical evidence on human and computerized tutoring principles of multimedia learning, Artificial Intelligence in educational systems for metacognition and SRL, and research on SRL from our team and that of other researchers. We present MetaTutor followed by a synthesis of key research findings on the effectiveness of various versions of the system on learning outcomes. First, we focus on findings from self-reports, learning outcomes, and multimodal data and their contributions to our understanding of SRL with an ITS. Second, we elaborate on the role of embedded pedagogical agents as external regulators designed to scaffold learners’ cognitive and metacognitive SRL strategy use. Third, we highlight and elaborate on the contributions of multimodal data in measuring and understanding the role of cognitive, affective, metacognitive, and motivational processes. Additionally, we unpack some of the challenges these data pose for designing real-time instructional interventions that scaffold SRL. Fourth, we present existing theoretical, methodological, and analytical challenges and briefly discuss lessons learned and open challenges. (shrink)

The replicability of findings in experimental psychology can be improved by distinguishing sharply between hypothesis-generating research and hypothesis-testing research. This distinction can be achieved by preregistration, a method that has recently attracted widespread attention. Although preregistration is fair in the sense that it inoculates researchers against hindsight bias and confirmation bias, preregistration does not allow researchers to analyze the data flexibly without the analysis being demoted to exploratory. To alleviate this concern we discuss how researchers may conduct blinded analyses. As (...) with preregistration, blinded analyses break the feedback loop between the analysis plan and analysis outcome, thereby preventing cherry-picking and significance seeking. However, blinded analyses retain the flexibility to account for unexpected peculiarities in the data. We discuss different methods of blinding, offer recommendations for blinding of popular experimental designs, and introduce the design for an online blinding protocol. (shrink)

Neuroscientists have in recent years turned to building models that aim to generate predictions rather than explanations. This “predictive turn” has swept across domains including law, marketing, and neuropsychiatry. Yet the norms of prediction remain undertheorized relative to those of explanation. I examine two styles of predictive modeling and show how they exemplify the normative dynamics at work in prediction. I propose an account of how predictive models, conceived of as technological devices for aiding decision-making, can come to be adequate (...) for purposes that are defined by both their guiding research questions and their larger social context of application. (shrink)

Over the past decades psychological theories have made significant headway into economics, culminating in the 2002 (partially) and 2017 Nobel prizes awarded for work in the field of Behavioral Economics. Many of the insights imported from psychology into economics share a common trait: the presumption that decision makers use shortcuts that lead to deviations from rational behaviour (the Heuristics-and-Biases program). Many economists seem unaware that this viewpoint has long been contested in cognitive psychology. Proponents of an alternative program (the Ecological-Rationality (...) program) argue that heuristics need not be irrational, particularly when judged relative to characteristics of the environment. We sketch out the historical context of the antagonism between these two research programs and then review more recent work in the Ecological-Rationality tradition. While the heuristics-and-biases program is now well-established in (mainstream neo-classical) economics via Behavioral Economics, we show there is considerable scope for the Ecological-Rationality program to interact with economics. In fact, we argue that there are many existing, yet overlooked, bridges between the two, based on independently derived research in economics that can be construed as being aligned with the tradition of the Ecological-Rationality program. We close the paper with a discussion of the open challenges and difficulties of integrating the Ecological Rationality program with economics. (shrink)

Research on self-determination theory emphasizes the importance of the internalization of motivation as a crucial factor for determining the quality of motivation. Hence, intrinsic motivation is deemed as an important predictor of learning. Research on epistemic beliefs, on the other hand, focuses on the nature of knowledge, and learning with more sophisticated epistemic beliefs associated with more adaptive outcomes. While learning and achievement are multiply determined, a more comprehensive theoretical model that takes into account both motivational quality and epistemic beliefs (...) is needed. Hence, this study aims to examine the role of intrinsic and instrumental motivation alongside epistemic beliefs in predicting students’ achievement in science. Data were drawn from the PISA 2015 survey. We focused on four of the top-performing societies. Two were Eastern societies – Singapore and Hong Kong, and the other two were Western societies: Canada and Finland. We found both common and specific patterns among the four societies. Regarding the common patterns, we found that intrinsic motivation and epistemic beliefs had direct positive effects on science achievement. As for the regionally-specific findings, instrumental motivation positively predicted achievement only in Western societies, but not in Eastern societies. The interaction effect between motivation and epistemic beliefs also demonstrated different patterns across the four societies. Implications for the role of motivation and epistemic beliefs in optimizing student learning and achievement are discussed. (shrink)

We comment on the proposition “that lower temperatures and especially greater seasonal variation in temperature call for individuals and societies to adopt … a greater degree of self-control” (Van Lange et al., sect. 3, para. 4) for which we cannot find empirical support in a large data set with data-driven analyses. After providing greater nuance in our theoretical review, we suggest that Van Lange et al. revisit their model with an eye toward the social determinants of self-control.

BackgroundSelf-report multiple choice questionnaires have been widely utilized to quantitatively measure one’s personality and psychological constructs. Despite several strengths, self-report multiple choice questionnaires have considerable limitations in nature. With the rise of machine learning and Natural language processing, researchers in the field of psychology are widely adopting NLP to assess psychological construct to predict human behaviors. However, there is a lack of connections between the work being performed in computer science and that of psychology due to small data sets and (...) unvalidated modeling practices.AimsThe current article introduces the study method and procedure of phase II which includes the interview questions for the five-factor model of personality developed in phase I. This study aims to develop the interview and open-ended questions for the FFM-based personality assessments, specifically designed with experts in the field of clinical and personality psychology, and to collect the personality-related text data using the interview questions and self-report measures on personality and psychological distress. The purpose of the study includes examining the relationship between natural language data obtained from the interview questions, measuring the FFM personality constructs, and psychological distress to demonstrate the validity of the natural language-based personality prediction.MethodsPhase I study was conducted to fifty-nine native Korean adults to acquire the personality-related text data from the interview and open-ended questions based on the FFM of personality. The interview questions were revised and finalized with the feedback from the external expert committee, consisting of personality and clinical psychologists. Based on the established interview questions, a total of 300 Korean adults will be recruited using a convenience sampling method via online survey. The text data collected from interviews will be analyzed using the natural language processing. The results of the online survey including demographic data, depression, anxiety, and personality inventories will be analyzed together in the model to predict individuals’ FFM of personality and the level of psychological distress. (shrink)

In this article, we extend the Bayesian nonparametric regression method Gaussian Process Regression to the analysis of longitudinal panel data. We call this new approach Gaussian Process Panel Modeling (GPPM). GPPM provides great flexibility because of the large number of models it can represent. It allows classical statistical inference as well as machine learning inspired predictive modeling. GPPM offers frequentist and Bayesian inference without the need to resort to Markov chain Monte Carlo-based approximations, which makes the approach exact and fast. (...) GPPMs are defined using the kernel-language, which can express many traditional modeling approaches for longitudinal data, such as linear structural equation models, multilevel models, or state-space models but also various commonly used machine learning approaches. As a result, GPPM is uniquely able to represent hybrid models combining traditional parametric longitudinal models and nonparametric machine learning models. In the present paper, we introduce GPPM and illustrate its utility through theoretical arguments as well as simulated and empirical data. (shrink)

Extended redundancy analysis is a statistical method that relates multiple sets of predictors to response variables. In ERA, the conventional approach of model evaluation tends to overestimate the performance of a model since the performance is assessed using the same sample used for model development. To avoid the overly optimistic assessment, we introduce a new model evaluation approach for ERA, which utilizes computer-intensive resampling methods to assess how well a model performs on unseen data. Specifically, we suggest several new model (...) evaluation metrics for ERA that compute a model’s performance on out-of-sample data, i.e., data not used for model development. Although considerable work has been done in machine learning and statistics to examine the utility of cross-validation and bootstrap variants for assessing such out-of-sample predictive performance, to date, no research has been carried out in the context of ERA. We use simulated and real data examples to compare the proposed model evaluation approach with the conventional one. Results show the conventional approach always favor more complex ERA models, thereby failing to prevent the problem of overfitting in model selection. Conversely, the proposed approach can select the true ERA model among many mis-specified models. (shrink)

Modern technologies have enabled the development of dynamic game- and simulation-based assessments to measure psychological constructs. This has highlighted their potential for supplementing other assessment modalities, such as self-report. This study describes the development, design, and preliminary validation of a simulation-based assessment methodology to measure psychological resilience—an important construct for multiple life domains. The design was guided by theories of resilience, and principles of evidence-centered design and stealth assessment. The system analyzed log files from a simulated task to derive individual (...) trajectories in response to stressors. Using slope analyses, these trajectories were indicative of four types of responses to stressors: thriving, recovery, surviving, and succumbing. Using Machine Learning, the trajectories were predictive of self-reported resilience with high accuracy, supporting construct validity of the simulation-based assessment. These findings add to the growing evidence supporting the utility of gamified assessment of psychological constructs. Importantly, these findings address theoretical debates about the construct of resilience, adding to its theory, supporting the combination of the “trait” and “process” approaches to its operationalization. (shrink)

The experimental research paradigm lies at the core of empirical psychology. New data analytical and computational tools continually enrich its methodological arsenal, while the paradigm’s mission remains the testing of theoretical predictions and causal explanations. Predictions regarding experimental results necessarily point to the future. Once the data are collected, the causal inferences refer to a hypothesis now lying in the past. The experimental paradigm is not designed to permit strong inferences about particular incidents that occurred before predictions were made. In (...) contrast, historical research and scholarship in other humanities focus on this backward direction of inference. The disconnect between forward-looking experimental psychology and backward-looking historical (i.e., narrative) psychology is a challenge in the postmodern era, which can be addressed. To illustrate this possibility, I discuss three historical case studies in light of theory and research in contemporary psychology. (shrink)

In light of the recent credibility crisis in psychology, this paper argues for a greater emphasis on theorizing in scientific research. Although reliable experimental evidence, preregistration, methodological rigor, and new computational frameworks for modeling are important, scientific progress also relies on properly functioning theories. However, the current understanding of the role of theorizing in psychology is lacking, which may lead to future crises. Theories should not be viewed as mere speculations or simple inductive generalizations. To address this issue, the author (...) introduces a framework called “cognitive metascience,” which studies the processes and results of evaluating scientific practice. This study should proceed both qualitatively, as in traditional science and technology studies and cognitive science, and quantitatively, by analyzing scientific discourse using language technology. By analyzing theories as cognitive artifacts that support cognitive tasks, this paper aims to shed more light on their nature. This perspective reveals that multiple distinct theories serve entirely different roles, and studying these roles, along with their epistemic vices and virtues, can provide insight into how theorizing should proceed. The author urges a change in research culture to appreciate the variety of distinct theories and to systematically advance scientific progress. . (shrink)

Transcranial direct current stimulation is a safe and efficient intervention for treating major depressive disorder. However, research has suggested heterogeneity of response between patients. The emerging field of precision psychiatry aims to use statistical modeling of multi-modal data to tailor treatment to the single patient. To this end, more in-depth analysis of randomized controlled trials will be relevant due to limited availability of other large datasets with high phenotypic detail and to develop tools for personalization within counterfactually controlled environments to (...) distinguish specific vs. non-specific patterns in treatment data. Previous research has aimed at identifying patient-related factors associated with better response. However, most analyses have operated on the group-level, ignoring natural clusters within the patients' constituting factors, their individual trajectories of symptom improvement, and their presented symptoms. Furthermore, group-based modeling strategies were limited to explanatory approaches using in-sample hypothesis-testing, that are ill-suited to prognosticate outcomes of single patients. This dissertation provides a methodological framework for reevaluation of existing clinical trial data to provide future investigations with more differentiated units of analysis and to complement explanatory approaches with predictive modeling strategies enabling prediction of single-patient outcomes. Using data from a landmark 3-arm clinical trial paradigmatic for a rigorously controlled experimental design the dissertation provides three blueprint studies for modeling heterogeneity of tDCS response: Study 1 characterized response to tDCS by considering patient-individual dynamics of symptom change over the course of treatment. Distinct trajectories of tDCS response could be identified, representing patient subgroups with varying strength and speed of improvement. These results suggest development of individualized treatment protocols and exploration of prolonged treatment courses. Study 2 reevaluated the efficacy of tDCS, in distinct, naturally occurring clusters of depressive symptoms. Using unsupervised machine learning, a global depression measure was parsed into 4 distinct symptom clusters. Analysis of cluster-scores showed superiority of tDCS and escitalopram over placebo in core depressive symptoms, but only tDCS was superior in improving sleep and only escitalopram was superior in improving guilt/anxiety symptoms, suggesting treatment selection based on patients' symptom profiles. In Study 3 supervised ML algorithms were employed to predict response to tDCS. In this proof-of-concept approach, response could be predicted above chance on the single-patient level, but overall accuracy was modest. Features employed for model training were explored using interpretable ML methods. Trained algorithms were provided to the field for expansion as well as tests of generalizability and incremental utility. The presented studies illustrate how in-depth secondary analyses of clinical trial data can aid personalization of treatment. The provided methodological framework can be expanded and generalized to other contexts and interventions that show heterogeneity of treatment effects. Yet, the empirical studies also epitomize challenges precision psychiatry is faced with, including low data availability, low outcome granularity, and limited external validation opportunities. The dissertation concludes with a discussion of challenges and future directions resulting from infrastructural demands in data acquisition, data management, data sharing, and interdisciplinary collaboration. (shrink)

Different theories of emotions have been introduced since the 19th century. Even though a large number of apparent differences between these theories exist, there is a broad consensus today that emotions consist of multiple components such as cognition, physiology, motivation, and subjectively perceived feeling. Appraisal theories of emotions, such as the Component Process Model by Klaus Scherer, emphasize that the cognitive evaluation of a stimulus or event is the driving component of the emotion process. It is believed to cause changes (...) in all other components and hence to differentiate emotion states. To test the CPM and gain more insights into the multi-componential emotion process, the present thesis examines two emotion sub-processes – the link between the cognitive and the feeling component and the link between the cognitive and the physiological component – by using different predictive modeling approaches. In study 1, four theoretically informed models were implemented. The models use a weighted distance metric based on an emotion prototype approach to predict the perceived emotion of participants from self-reported cognitive appraisals. Moreover, they incorporate different weighting functions with weighting parameters that were either derived from theory or estimated from empirical data. The results substantiate the examined link based on the predictive performance of the models. In line with the CPM, the preferred model weighted the appraisal evaluations differently in the distance metric. However, the data-derived weighting parameters of this model deviate from theoretically proposed ones. Study 2 analyzed the link between cognition and physiology by predicting self-reported appraisal dimensions from a large set of physiological features using different linear and non-linear machine learning algorithms. Based on the predictive performance of the models, the study is able to confirm that most cognitive evaluations were interlinked with different physiological responses. The comparison of the different algorithms and the application of methods for interpretable machine learning showed that the relation between these two components is best represented by a non-linear model and that the studied link seems to vary among physiological signals and cognitive dimensions. Both studies substantiate the assumption that the cognitive appraisal process is interlinked with physiology and subjective feelings, accentuating the relevance of cognition in emotion as assumed in appraisal theory. They also demonstrate how computational emotion modeling can be applied in basic research on emotions. (shrink)

We challenge the unquestioning pursuit of the appearance of objectivity and ingrained designer-user dualism in human-centred design research and propose a resurrection of introspection as a valid approach to investigating subjective experiences. Through comparing epistemic perspectives and reviewing the histories of introspection in several disciplines, we liberate the research field of experience-driven design from a long-lasting doubt about and the disguised and unsystematic use of this method. To establish a foundation for the further development of introspective methods, we focus on (...) its most controversial type (i.e. researcher introspection) and discuss its strengths and weaknesses, preconditions of use, diverse ways to practise for different suitable experiencedriven design research purposes, and useful techniques and tools. (shrink)

Extreme Response Style describes individual differences in selecting extreme response options in Likert scale items, which are stable over time and across different psychological constructs. This thesis contains three empirical studies on the detection, avoidance, and compensation of ERS: In the first study, we introduce a new method to detect ERS which uses an ERS index from heterogeneous items as covariate in partial credit trees. This approach combines the objectivity of ERS indices from heterogeneous items with the threshold interpretation of (...) ERS known from analyses with the ordinal mixed-Rasch model. We analyzed personality facets of 11714 subjects from the German nonclinical normative sample of the Revised NEO Personality Inventory, and 3835 participants of the longitudinal panel of the GESIS - Leibniz-Institute for the Social Sciences, who filled out the Positive and Negative Affect Schedule, and the Questionnaire of Spatial Strategies. ERS was detected in all analyzed scales. The resulting pattern suggests that ERS reflects a stable trait with a continuous structure. In the second study, we investigate whether data from items with dichotomous response formats are unaffected by ERS, as has been assumed in the literature. In a paper and pencil questionnaire, 429 German psychology students completed the Shyness scale from the Revised Minnesota Multiphasic Personality Inventory and the Achievement Orientation scale from the Revised Freiburger Persöhnlichkeitsinventar. ERS was assessed by an ERS index from heterogeneous items, a binary ERS measure based on the classification of an ordinal mixed-Rasch model, and a binary self-report measure of ERS. ERS measures were used as covariates in Rasch trees and DIF Lasso models of the dichotomous scales. We did not find any effect of ERS on dichotomous item responses. Adopting dichotomous response formats seems to be a reasonable strategy to avoid ERS. In the third study, we test whether instructions to give more or less extreme responses depending on participants’ individual response tendencies, can counterbalance the impact of ERS. In an online questionnaire, 788 German subjects completed the Impulsivity and Order facets of the NEO-PI-R three times under different ERS instructions. In the first round, a standard instruction was used. Participants in the experimental group received instructions for more or less extreme responses in the second and third round, while subjects in the control group responded under neutral instructions. ERS was measured by an ERS index from heterogeneous items and a self-report measure of ERS. Binary ERS classifications were used to create artificial datasets in which participants received an instruction which should either compensate or aggravate their individual response tendencies. Predictive performance of Random Forest models, in which self-reported impulsive and orderly behaviors were predicted by the item responses, was compared between the compensation, aggravation, and control settings. No differences in predictive performance were observed between the settings. Likewise, PC tree analyses suggest that ERS was still present in the compensation setting. Including ERS measures as predictors did not increase predictive performance when items were answered under standard instructions. Our findings are in line with simulation studies suggesting that ERS has a small impact on applied psychological measurement. Future research on ERS could improve psychological measurements by considering continuous models of ERS. In light of recent calls to turn psychology into a more predictive science, investigating the impact of ERS on criterion validity should also have high priority. (shrink)