One of the critical issues in the philosophy of science is to understand scientific knowledge. This paper proposes a novel approach to the study of reflection on science, called “cognitive metascience”. In particular, it offers a new understanding of scientific knowledge as constituted by various kinds of scientific representations, framed as cognitive artifacts. It introduces a novel functional taxonomy of cognitive artifacts prevalent in scientific practice, covering a huge diversity of their formats, vehicles, and functions. As a consequence, toolboxes, conceptual (...) frameworks, theories, models, and individual hypotheses can be understood as artifacts supporting our cognitive performance. It is also shown that by empirically studying how artifacts function, we may discover hitherto undiscussed virtues and vices of these scientific representations. This paper relies on the use of language technology to analyze scientific discourse empirically, which allows us to uncover the metascientific views of researchers. This, in turn, can become part of normative considerations concerning virtues and vices of cognitive artifacts. (shrink)

Replicability is usually considered to be one of the cornerstones of science; however, the growing recognition of nonreplicable experiments and studies in scientific journals—a phenomenon that has been called ‘replicability crisis’—has spurred a debate on the meaning, function, and significance of replicability in science. Amid this discussion, it has become clear that replicability is not a monolithic concept; what is still controversial is exactly how the distinction between different kinds of replicability should be laid out terminologically and conceptually, and to (...) what extent it bears on the more general debate on the centrality of replicability in science. This paper’s goals are to clarify the different uses of the terms related to replicability and, more importantly, to conceptually specify the kinds of replicability and their respective epistemic functions. (shrink)

Most discussions of the reproducibility crisis focus on its epistemic aspect: the fact that the scientific community fails to follow some norms of scientific investigation, which leads to high rates of irreproducibility via a high rate of false positive findings. The purpose of this paper is to argue that there is a heretofore underappreciated and understudied dimension to the reproducibility crisis in experimental psychology and neuroscience that may prove to be at least as important as the epistemic dimension. This is (...) the communication dimension. The link between communication and reproducibility is immediate: independent investigators would not be able to recreate an experiment whose design or implementation were inadequately described. I exploit evidence of a replicability and reproducibility crisis in computational science, as well as research into quality of reporting to support the claim that a widespread failure to adhere to reporting standards, especially the norm of descriptive completeness, is an important contributing factor in the current reproducibility crisis in experimental psychology and neuroscience. (shrink)

In light of the recent credibility crisis in psychology, this paper argues for a greater emphasis on theorizing in scientific research. Although reliable experimental evidence, preregistration, methodological rigor, and new computational frameworks for modeling are important, scientific progress also relies on properly functioning theories. However, the current understanding of the role of theorizing in psychology is lacking, which may lead to future crises. Theories should not be viewed as mere speculations or simple inductive generalizations. To address this issue, the author (...) introduces a framework called “cognitive metascience,” which studies the processes and results of evaluating scientific practice. This study should proceed both qualitatively, as in traditional science and technology studies and cognitive science, and quantitatively, by analyzing scientific discourse using language technology. By analyzing theories as cognitive artifacts that support cognitive tasks, this paper aims to shed more light on their nature. This perspective reveals that multiple distinct theories serve entirely different roles, and studying these roles, along with their epistemic vices and virtues, can provide insight into how theorizing should proceed. The author urges a change in research culture to appreciate the variety of distinct theories and to systematically advance scientific progress. . (shrink)