Since the advent of the twenty-first century, science has experienced a crisis pertaining to the replicability of quantitative research findings, which has become known as the ‘replication crisis’. The replication crisis has particularly afflicted research in the behavioural sciences, and psychology in particular. Given the relevance of psychology to education, it is unsurprising that the replication crisis also presents an issue for quantitative educational research, thus potentially compromising its practical usefulness. This article outlines the replication crisis in psychology, and highlights (...) its significance for quantitative educational research. Greater methodological rigour, aimed at addressing methodological deficiencies in the conduct of scientific research, has been suggested as a response to the replication crisis. Following a review of various calls for greater methodological rigour, the current article argues that, whilst methodological deficiencies may be a contributory factor, there is potentially a more fundamental reason for the replication crisis. The case is made that the non-separability of a measured attribute from the measurement instrument, and the irreducible uncertainty in unmeasured attributes, may be the principal reasons for the replication crisis in psychology and education. Implications of such an explanation for the crisis are articulated in relation to the evidence-based policy agenda in education. (shrink)

In this paper, we provide a re-interpretation of qualitative and quantitative modeling from a representationalist perspective. In this view, both approaches attempt to construct abstract representations of empirical relational structures. Whereas quantitative research uses variable-based models that abstract from individual cases, qualitative research favors case-based models that abstract from individual characteristics. Variable-based models are usually stated in the form of quantified sentences. This syntactic structure implies that sentences about individual cases are derived using deductive reasoning. In contrast, case-based models are (...) usually stated using context-dependent existential sentences. This syntactic structure implies that sentences about other cases are justifiable by inductive reasoning. We apply this representationalist perspective to the problems of generalization and replication. Using the analytical framework of modal logic, we argue that the modes of reasoning are often not only applied to the context that has been studied empirically, but also on a between-contexts level. Consequently, quantitative researchers mostly adhere to a top-down strategy of generalization, whereas qualitative researchers usually follow a bottom-up strategy of generalization. Depending on which strategy is employed, the role of replication attempts is very different. In deductive reasoning, replication attempts serve as empirical tests of the underlying theory. Therefore, failed replications imply a faulty theory. From an inductive perspective, however, replication attempts serve to explore the scope of the theory. Consequently, failed replications do not question the theory per se, but help to shape its boundary conditions. We conclude that quantitative research may benefit from a bottom-up generalization strategy as it is employed in most qualitative research programs. Inductive reasoning forces us to think about the boundary conditions of our theories and provides a framework for generalization beyond statistical testing. In this perspective, failed replications are just as informative as successful replications, because they help to explore the scope of our theories. (shrink)

Jury science is fraught with difficulty. Since legal and institutional hurdles render it all but impossible to study live criminal jury deliberation, researchers make use of various indirect methods to evaluate jury performance. But each of these methods are open to methodological criticism and, strikingly, some of the highest-profile jury research programmes in recent years have reached opposing conclusions. Uncertainty about jury performance is an obstacle for legal reform—ongoing debates about the ‘justice gap’ for complainants of sexual offences has rendered (...) these problems stark. This paper proposes a way to advance the debate. (shrink)

Metascientists have studied questionable research practices in science. The present article considers the parallel concept of questionable metascience practices (QMPs). A QMP is a research practice, assumption, or perspective that has been questioned by several commentators as being potentially problematic for metascience and/or the science reform movement. The present article reviews ten QMPs that relate to criticism, replication, bias, generalization, and the characterization of science. Specifically, the following QMPs are considered: (1) rejecting or ignoring self-criticism; (2) a fast ‘n’ bropen (...) scientific criticism style; (3) overplaying the role of replication in science; (4) assuming a replication rate is “too low” without specifying an “acceptable” rate; (5) an unacknowledged metabias towards explaining the replication crisis in terms of researcher bias; (6) assuming that researcher bias can be reduced; (7) devaluing exploratory results as being more “tentative” than confirmatory results; (8) presuming that questionable research practices are problematic research practices; (9) focusing on knowledge accumulation; and (10) focusing on specific scientific methods. It is stressed that only some metascientists engage in some QMPs some of the time, and that these QMPs may not always be problematic. Research is required to estimate the prevalence and impact of QMPs. In the meantime, QMPs should be viewed as invitations to ask questions about how we go about doing better metascience. (shrink)

Modern empirical psychology, as a reductionist, materialist, and positivist science, has to a great extent replaced philosophical psychology – or more precisely philosophical anthropology– in our contemporary world, and this has caused modern psychology to lose sight of what was most interesting in pre-modern psychology, namely the attempt to situate the human person in his experience of reality in the lifeworld (lebenswelt). This has resulted in the practice of psychology becoming detached from the realities of lived experience as its view (...) of human nature becomes increasingly narrow, rigid and scientistic. This is evidenced by the current “replication crisis” in modern psychology, which has severely impacted the credibility of modern psychology as a field of enquiry. This crisis arose with an increasing methodological standardisation that is being pursued at the expense of interrogating the scientistic presuppositions that ground the study of modern psychology. (shrink)

To resolve the hard problem of consciousness, we propose a systems-level theory of syner- gistic processes to account for sentience, consciousness, and mind. Subjectivity arises from interactive network processes within and between nature’s entangled, relational, and iter- ative elements. Searches for the physical cause or locus of consciousness and subjectivity are misguided given that consciousness emerges from processes with no single source. To account for the evolution and phenomenology of mind, our synergistic network account of consciousness dispenses with mind–body dualism, (...) it weakens boundaries between the material and the non-material, between the internal and the external, and emphasises synergistic processes and functionalities over outcome. The theory challenges psychology to abandon its cause–effect categorisations and implied mind–body dualism in favour of functional systems-level analyses to better account for relational processes and functions which unfold synergistically within and sustain nature’s complex networks of entangled elements. (shrink)