Peer review of grant applications, it has been suggested, might be distorted by what is popularly termed old boyism, cronyism, or particularism. We argue that the existing debate emphasizes the more uninteresting aspects of the peer review system and that the operation of old boyism, as currently understood would have little effect on the overall direction of science. We identify a phenomenon of cognitive particularism, which we consider to be more important than the institutional cronyism analyzed in previous studies. We (...) illustrate with material drawn from observation of grant-awarding commit tees of the Science and Engineering Research CounciL In the concluding discussion, we explore some of the possible implications for the peer review system. (shrink)

Research agendas and academic evaluation are inevitably linked. By means of economic incentives, promotion, research funding, and reputation academic evaluation is a powerful influence on the production of knowledge; moreover, it is often conceived as a universal instrument without consideration of the context in which it is applied. Evaluation systems are social constructions in dispute, being the current focus of international debates regarding criteria, indicators, and their associated methods. A universalist type of productivity indicators is gaining centrality in academic evaluation (...) with profound effects on the content of research that is conducted everywhere. Specifically, evaluation systems based on this type of indicators are sending negative signals to scientists willing to conduct research on contextualized agendas, particularly those negotiated with non scientists. On the basis of theoretical and empirical studies documented on the specialized literature and extensive personal engagement with university research policy in Uruguay, we argue that the consolidation of evaluation practices of alleged universal validity deteriorates and discourages a type of research which is undeniably important in developing contexts. (shrink)

To arrive at their final evaluation of a manuscript or grant proposal, reviewers must convert a submission’s strengths and weaknesses for heterogeneous peer review criteria into a single metric of quality or merit. I identify this process of commensuration as the locus for a new kind of peer review bias. Commensuration bias illuminates how the systematic prioritization of some peer review criteria over others permits and facilitates problematic patterns of publication and funding in science. Commensuration bias also foregrounds a range (...) of structural strategies for realigning peer review practices and institutions with the aims of science. (shrink)

Epistemological differences fuel continuous and frequently divisive debates in the social sciences and the humanities. Sociologists have yet to consider how such differences affect peer evaluation. The empirical literature has studied distributive fairness, but neglected how epistemological differences affect perception of fairness in decision making. The normative literature suggests that evaluators should overcome their epistemological differences by ‘‘translating’’ their preferred standards into general criteria of evaluation. However, little is known about how procedural fairness actually operates. Drawing on eighty-one interviews with (...) panelists serving on five multidisciplinary fellowship competitions in the social sciences and the humanities, we show that Evaluators generally draw on four epistemological styles to make arguments in favor of and against proposals. These are the constructivist, comprehensive, positivist, and utilitarian styles; and Peer reviewers define a fair decision-making process as one in which panelists engage in ‘‘cognitive contextualization,’’ that is, use epistemological styles most appropriate to the field or discipline of the proposal under review. (shrink)

This book is the first to provide a sustained, coherent analysis of Foucault's work as a whole. To demonstrate the sense in which Foucault's work is beyond structuralism and hermeneutics, the authors unfold a careful, analytical exposition of his oeuvre. They argue that during the of Foucault's work became a sustained and largely successful effort to develop a new method - "interpretative analytics" - capable of explaining both the logic of structuralism's claim to be an objective science and the apparent (...) validity of the hermeneutical counterclaim that the human sciences can proceed only by understanding the deepest meaning of the subject and his tradition. (shrink)

Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the normative implications (...) of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to countenance the complexity of social interactions among actors involved directly and indirectly in peer review. (shrink)