Prepublication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behaviour of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing prepublication peer review can be evaluated with a reasonable level of confidence based on presently available evidence, they are either positive or neutral. (...) We conclude that on present evidence abolishing peer review weakly dominates the status quo. (shrink)

Scholars belong to multiple communities of credit simultaneously. When these communities disagree about a scholarly achievement’s credit assignment, this raises a puzzle for decision and game theor...

Given the importance of scientific research in shaping our perception of the world, and our senses of what policies will and won’t succeed in altering that world, it is of great practical, political, and moral importance that we carry out scientific research with integrity. The phenomenon of scientific fraud stands in the way of that, as scientists may knowingly enter claims they take to be false into the scientific literature, often knowingly doing so in defiance of norms they profess allegiance (...) to. In this chapter we take a look at some of the causes of scientific fraud, and how it might be manifested in large-scale research teams and situations of anonymous authorship. We find that such cases make trouble for what might seem like intuitive answers to the question “who should be held responsible for this fraud?”, and we argue that in such cases it would be better to hold the entire community responsible for seeing to it that there is less fraud. (shrink)

To arrive at their final evaluation of a manuscript or grant proposal, reviewers must convert a submission’s strengths and weaknesses for heterogeneous peer review criteria into a single metric of quality or merit. I identify this process of commensuration as the locus for a new kind of peer review bias. Commensuration bias illuminates how the systematic prioritization of some peer review criteria over others permits and facilitates problematic patterns of publication and funding in science. Commensuration bias also foregrounds a range (...) of structural strategies for realigning peer review practices and institutions with the aims of science. (shrink)

On the surface, developing a social psychology of science seems compelling as a way to understand how individual social cognition – in aggregate – contributes towards individual and group behavior within scientific communities (Kitcher, 2002). However, in cases where the functional input-output profile of psychological processes cannot be mapped directly onto the observed behavior of working scientists, it becomes clear that the relationship between psychological claims and normative philosophy of science should be refined. For example, a robust body of social (...) psychological research demonstrates implicit gender bias in the evaluation of others (e.g., Steinpreis, Anders, & Ritzke, 1999). Many expected implicit bias to be a major cause of women’s underrepresentation in math intensive fields of science; however, quantitative sociological research of hiring and manuscript and grant evaluation has discovered no gender disparity in outcomes (Ceci & Williams, 2011). Why might this be so? This paper will discuss methodological challenges that go beyond classic problems of external validity in extrapolating psychological effects and explanations to scientific communities. These problems include more complex external validity issues raised by the introduction of multi-process models of cognition (e.g., implicit versus explicit social cognition) as well as the reflexive role that folk and experimental theories of social psychology play in guiding the behavior of scientists at the individual and community level. (shrink)

Advocates of the self-corrective thesis argue that scientific method will refute false theories and find closer approximations to the truth in the long run. I discuss a contemporary interpretation of this thesis in terms of frequentist statistics in the context of the behavioral sciences. First, I identify experimental replications and systematic aggregation of evidence (meta-analysis) as the self-corrective mechanism. Then, I present a computer simulation study of scientific communities that implement this mechanism to argue that frequentist statistics may converge upon (...) a correct estimate or not depending on the social structure of the community that uses it. Based on this study, I argue that methodological explanations of the “replicability crisis” in psychology are limited and propose an alternative explanation in terms of biases. Finally, I conclude suggesting that scientific self-correction should be understood as an interaction effect between inference methods and social structures. (shrink)