The demand for greater public accountability is changing the nature of ex ante peer review at public science agencies worldwide. Based on a four year research project, this essay examines these changes through an analysis of the process of grant proposal review at two US public science agencies, the National Institutes of Health (NIH) and the National Science Foundation (NSF). Weaving historical and conceptual narratives with analytical accounts, we describe the ways in which these two agencies struggle with the question (...) of incorporating considerations of societal impact into the process of peer review. We use this comparative analysis to draw two main conclusions. First, evaluation of broader societal impacts is not different in kind from evaluation of intellectual merit. Second, the scientific community may actually bolster its autonomy by taking a broader range of considerations into its peer review processes. (shrink)

Born in the 17th century, journal peer review is an extremely diverse technology, constantly torn between two often incompatible goals: the validation of manuscripts conceived as a collective industrial-like reproducible process performed to assert scientific statements, and the dissemination of articles considered as a means to spur scientific discussion, raising controversies, and civically challenging a state of knowledge. Such a situation is particularly conducive to clarifying the processes of valuation and evaluation in journal peer review. In this article, such processes (...) are considered as specific tests in order to emphasize the uncertain properties of pre-tests manuscripts. On the one hand, evaluation tests are examined at the core of the validation of manuscripts, such as defining the coordination of judging instances or controlling the modalities of inter-knowledge between reviewers and authors. They are also studied regarding the dissemination of articles, notably through the contemporary conception of a continuing evaluation test termed “post publication peer review”. On the other hand, valuation tests are both part of the validation of manuscripts, such as the weighting of different judgments of the same manuscript and the tensions that these hierarchies cause, and of the dissemination of articles, such as attention metrics recording the uses of articles. The conclusion sketches out how the articulation of these different tests has recently empowered readers as a new key judging instance for dissemination and for validation, potentially transforming the definition of peers, and thus the whole process of journal peer review. (shrink)

The nature of authority is shifting in online scholarly communication. This examination of the history and future of peer review argues that effective online communication requires the development of an open, community?oriented, post?publication system of peer?to?peer review, transforming peer review from a process focused on gatekeeping to one concerned with filtering the wealth of scholarly material made available via the Internet.

Capitalist dynamics in knowledge production are not limited to situations in which economic interests influence researchers’ practices. Building on laboratory studies and the French “pragmatic” tradition in sociology, this article proposes an approach to tackle more pervasive capitalist logics at work in contemporary research and their consequences. It uses the term epistemic capitalism to denote the accumulation of capital, as worth made durable, through the act of doing research, in and beyond academia. In doing so, it conceptualizes capitalism primarily not (...) as a system of circulation and accumulation of monetary value but rather as a cultural way of producing, attributing, and accumulating specific forms of worth, which need not be monetary. Empirically, the article studies variants in epistemic capitalism by addressing the differing role of the accumulation of different forms of capital and the regimes connected to it in two institutional settings in Austria, academic life science laboratories and biotechnology start-up companies. Concluding, it argues that analytically dissociating the concept of capitalism from its link to economic value allows a finer-grained cultural analysis of the importance and effects of processes of accumulation in contemporary research. It ends with discussing the normative implications of these findings for debates about the commercialization of academia. (shrink)

Peer review of grant applications, it has been suggested, might be distorted by what is popularly termed old boyism, cronyism, or particularism. We argue that the existing debate emphasizes the more uninteresting aspects of the peer review system and that the operation of old boyism, as currently understood would have little effect on the overall direction of science. We identify a phenomenon of cognitive particularism, which we consider to be more important than the institutional cronyism analyzed in previous studies. We (...) illustrate with material drawn from observation of grant-awarding commit tees of the Science and Engineering Research CounciL In the concluding discussion, we explore some of the possible implications for the peer review system. (shrink)