No scientific conclusion follows automatically from a statistically non-significant result, yet people routinely use non-significant results to guide conclusions about the status of theories (or the effectiveness of practices). To know whether a non-significant result counts against a theory, or if it just indicates data insensitivity, researchers must use one of: power, intervals (such as confidence or credibility intervals), or else an indicator of the relative evidence for one theory over another, such as a Bayes factor. I argue Bayes factors (...) allow theory to be linked to data in a way that overcomes the weaknesses of the other approaches. Specifically, Bayes factors use the data themselves to determine their sensitivity in distinguishing theories (unlike power), and they make use of those aspects of a theory’s predictions that are often easiest to specify (unlike power and intervals, which require specifying the minimal interesting value in order to address theory). Bayes factors provide a coherent approach to determining whether non-significant results support a null hypothesis over a theory, or whether the data are just insensitive. They allow accepting and rejecting the null hypothesis to be put on an equal footing. Concrete examples are provided to indicate the range of application of a simple online Bayes calculator, which reveal both the strengths and weaknesses of Bayes factors. (shrink)

The idea of replication is based on the premise that there are permanent laws to be replicated and verified, and the scientific method is adequate for doing so. Scientific truth, however, is not absolute but relative to time and context, and the method used. Time and context are inextricably interwoven, in that time creates different contexts and contexts (e.g., Christmas Day vs. New Year’s Day) create different experiences of time, rendering psychological phenomena inherently variable. This means that internal and external (...) conditions fluctuate and are different in a replication study vs. the original. Thus, a replication experiment is just another empirical investigation that has no special status in the establishment of scientific truth. It is not the final arbiter of whether or not something exists. In their pursuit of homogeneous external conditions, replications have ignored the homogeneity of internal conditions. There is not a single replication reported in the literature that would have shown participants’ feelings and thoughts—both conscious and nonconscious—to be identical to those of the original participants. Experimental instructions can create varying ratios of conscious over nonconscious processing from one study to another. Ironically, every replication is a failure at the fundamentals of human psychology. While patterns can be discovered, they are not permanent or unchangeable laws of human behavior to be proven by the pinpoint statistical verification through replication. As scientific knowledge in physics is temporary and incomplete, should it be any surprise that science can only provide “temporary winners” for psychological knowledge of human behavior? (shrink)

Based on an analysis of ten popular introductions to social psychology, we will show that Karl Popper's philosophy of ‘critical rationalism’ so far has had little to no traceable influence on the epistemology and practice of social psychology. If Popper is quoted or mentioned in the textbooks at all, the guiding principle of ‘falsificationism’ is reduced to a mere ‘falsifiability’ and some central elements of critical rationalism are left out – those that are incompatible with positivism and inductivism. Echoing earlier (...) attempts to introduce Popper to social psychology by Paul Meehl and Tom Pettigrew, we will argue that a discussing Popper's ideas in more depth could help social psychology to move forward in view of the ‘crisis of confidence’ that has emerged recently in view of the ‘Stapel affair’ and the reports of failures to replicate social psychological experiments in high-powered replication attempts. (shrink)

Philosophers and scientists agree that an important characteristic of theories is their internal coherence. I propose that there is a particular type of internal coherence, termed “unit coherence,” that has received insufficient attention from psychologists. When psychologists theorize about the mechanisms that bring about human behavior, the units in which the variables are expressed need to be consistent throughout the theory; this is unit coherence. The theory of reasoned action is an example of a unit incoherent theory. I explain why (...) this is so and also show how the theory can be made to be unit coherent. I also include examples of unit coherent theories to demonstrate that they are possible to have in psychology. Although the concept of unit coherence is somewhat subtle, it has surprising and far-reaching consequences. (shrink)

The paper examines the claim that significance testing violates the Principle of Total Evidence. I argue that p-values violate PTE for two-sided tests but satisfy PTE for one-sided tests invoking a sufficient test statistic independent of the preferred theory of evidence. While the focus of the paper is to evaluate a particular claim about the relationship of significance testing and PTE, I clarify the reading of this methodological principle along the way.

ABSTRACT:This study examines whether the empirical evidence on the relationship between corporate social performance (CSP) and corporate financial performance (CFP) differs depending on the publication outlet in which that evidence appears. This moderator meta-analysis, based on a total sample size of 33,878 observations, suggests that published CSP-CFP findings have been shaped by differences in institutional logics in different subdisciplines of organization studies. In economics, finance, and accounting journals, the average correlations were only about half the magnitude of the findings published (...) in Social Issues in Management, Business Ethics, or Business and Society journals (mean corrected correlation coefficientof.22 vs..49, respectively). Specifically, economists did not find null or negative CSP-CFP correlations, and average findings published in general management outlets (=.41) were closer to Social Issues in Management, Business Ethics, and Business and Society results than to findings reported in economics, finance, and accounting journals. (shrink)

The phenomenon of base-rate neglect has elicited much debate. One arena of debate concerns how people make judgments under conditions of uncertainty. Another more controversial arena concerns human rationality. In this target article, we attempt to unpack the perspectives in the literature on both kinds of issues and evaluate their ability to explain existing data and their conceptual coherence. From this evaluation we conclude that the best account of the data should be framed in terms of a dual-process model of (...) judgment, which attributes base-rate neglect to associative judgment strategies that fail to adequately represent the set structure of the problem. Base-rate neglect is reduced when problems are presented in a format that affords accurate representation in terms of nested sets of individuals. (shrink)

(2011). Korn and Freidlin's Misunderstanding of the Null Hypothesis Significance Testing Procedure. The American Journal of Bioethics: Vol. 11, No. 3, pp. 15-16.

ABSTRACTPossibly, the replication crisis constitutes the most important problem in psychology. It calls into question whether psychology is a science. Existing conceptualizations of replicability depend on effect sizes; the larger the population effect size, the greater the probability of replication. This is problematic and contributes to the replication crisis. A different conceptualization, not dependent on population effect sizes, is desirable. The proposed solution features the closeness of sample means to their corresponding population means, in both the original and replication experiments. (...) If the researcher has specified the sampling precision desired, it is possible to calculate the probability of replication, prior to data collection, and without dependence on the population effect size or expected population effect size. In addition, it is not necessary to know population means or standard deviations, nor sample means or standard deviations, to employ the proposed a priori way of thinking about... (shrink)

Psychologists take two propositions for granted. Specifically, empirical verification of predictions derived from a theory support that the theory is more likely to be true and support that additional predictions derived from the theory have an increased probability of being sustained if subjected to empirical testing. In contrast, I argue that both propositions depend strongly on whether auxiliary assumptions are taken into account. When auxiliary assumptions are not taken into account, the first proposition is valid but the second is not. (...) When auxiliary assumptions are taken into account, the first proposition is not valid, and the second proposition encounters additional problems. I use Venn diagrams and Bayesian principles to demonstrate these conclusions. (shrink)

The terms nested sets, partitive frequencies, inside-outside view, and dual processes add little but confusion to our original analysis (Gigerenzer & Hoffrage 1995; 1999). The idea of nested set was introduced because of an oversight; it simply rephrases two of our equations. Representation in terms of chances, in contrast, is a novel contribution yet consistent with our computational analysis System 1.dual process theory” is: Unless the two processes are defined, this distinction can account post hoc for almost everything. In contrast, (...) an ecological view of cognition helps to explain how insight is elicited from the outside (the external representation of information) and, more generally, how cognitive strategies match with environmental structures. (shrink)

The current research applied a mid-level evolutionary theory that has been successfully employed across numerous animal species—life history theory—in an attempt to understand the Dark Triad personality trait cluster (narcissism, psychopathy, and Machiavellianism). In Study 1 (N = 246), a measure of life history strategy was correlated with psychopathy, but unexpectedly with neither Machiavellianism nor narcissism. Study 2 (N = 321) replicated this overall pattern of results using longer, traditional measures of the Dark Triad traits and alternative, future-discounting indicators of (...) life history strategy (a smaller-sooner, larger-later monetary dilemma and self-reported risk-taking behaviors). Additional findings suggested two sources of shared variance across the Dark Triad traits: confidence in predicting future outcomes and openness to short-term mating. (shrink)

This study investigates: the changes in three major health-related factors—physical activity, non-physical-activity health behavior, and depressive symptoms, and how changes in physical activity were associated with changes in one’s depressive symptoms among young adults, middle-aged adults, and older adults while controlling non-physical-activity health behavior and sociodemographic characteristics among young, middle-aged, and older adults before and after the COVID-19 outbreak lockdown in the United States. A total of 695 participants completed an online questionnaire via MTurk, and participants were asked to recall (...) their physical activity, depressive symptoms, and non-physical-activity health behavior status in January and May of 2020. The IPAQ-SF was used to evaluate individuals’ physical activity, while the CES-D-10 was used to assess depressive symptoms. Covariates included non-physical-activity health behavior and sociodemographic factors. A Bayesian significance testing of changes was used to examine significant changes in physical activity, non-physical-activity behavior, and depressive symptoms in each age group while Bayesian regression analysis was employed to examine how the changes in physical activity were associated with respondents’ depressive symptoms while controlling for individual NHB and sociodemographic characteristics. The results showed that the participants tended to maintain their physical activity levels after the lockdown despite significant increases in sitting time among young and older adults. Decreases in moderate physical activity frequency were associated with a higher level of depressive symptoms. Although young and middle-aged cohorts experienced fewer differences in depressive symptoms compared to their counterparts in the older group, we found no significant heterogeneity effects in the relationships of interest across all age groups. Considering different influences of physical activity on depressive symptoms depending on different levels of activity and ages, more randomized clinical trials with program-based intervention studies should be conducted with different physical activity programs for different age populations. (shrink)

Barbey & Sloman (B&S) present five models that account for performance in Bayesian inference tasks, and argue that the data disconfirm four of them but support one model. Contrary to B&S, I argue that the cited data fail to provide strong confirmation or disconfirmation for any of the models.