A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.

Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the normative implications (...) of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to countenance the complexity of social interactions among actors involved directly and indirectly in peer review. (shrink)

In this book Bruno Latour brings together these different approaches to provide a lively and challenging analysis of science, demonstrating how social context..

A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published (...) research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices.With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated. (shrink)

A vital and underappreciated dimension of social interaction is the way individuals justify their actions to others, instinctively drawing on their experience to appeal to principles they hope will command respect. Individuals, however, often misread situations, and many disagreements can be explained by people appealing, knowingly and unknowingly, to different principles. On Justification is the first English translation of Luc Boltanski and Laurent Thévenot's ambitious theoretical examination of these phenomena, a book that has already had a huge impact on French (...) sociology and is likely to have a similar influence in the English-speaking world. In this foundational work of post-Bourdieu sociology, the authors examine a wide range of situations where people justify their actions. The authors argue that justifications fall into six main logics exemplified by six authors: civic (Rousseau), market (Adam Smith), industrial (Saint-Simon), domestic (Bossuet), inspiration (Augustine), and fame (Hobbes). The authors show how these justifications conflict, as people compete to legitimize their views of a situation. On Justification is likely to spark important debates across the social sciences. (shrink)

Do scientists agree? It is not only unrealistic to suppose that they do, but probably just as unrealistic to think that they ought to. Agreement is for what is already established scientific history. The current and vital ongoing aspect of science consists of an active and often heated interaction of data, ideas and minds, in a process one might call "creative disagreement." The "scientific method" is largely derived from a reconstruction based on selective hindsight. What actually goes on has much (...) less the flavor of a systematic method than of trial and error, conjecture, chance, competition and even dialectic. (shrink)

Shapiro traces the genesis of the fact, a modern concept that originated not in natural science but in legal discourse. She follows the concept's evolution and diffusion across a variety of disciplines in early modern England.