Exploring the role of values in scientific inquiry, Hugh Lacey examines the nature and meaning of values, and looks at challenges to the view, posed by postmodernists, feminists, radical ecologists, Third-World advocates and religious fundamentalists, that science is value free. He also focuses on discussions of 'development', especially in Third World countries. This paperback edition includes a new preface.

While much literature has sprouted on peer review, this is the first book-length, wide-ranging study that utilizes methods and resources of contemporary philosophy. It covers the tension between peer review and the liberal notion that truth emerges when ideas proliferate in the marketplace of ideas; arguments for and against blind review of submissions; the alleged conservatism of peer review; the anomalous nature of book reviewing; the status of non-peer-reviewed publications; and the future of peer review.

A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published (...) research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices.With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated. (shrink)

An empirically sensitive formulation of the norms of transformative criticism must recognize that even public and shared standards of evaluation can be implemented in ways that unintentionally perpetuate and reproduce forms of social bias that are epistemically detrimental. Helen Longino’s theory can explain and redress such social bias by treating peer evaluations as hypotheses based on data and by requiring a kind of perspectival diversity that bears, not on the content of the community’s knowledge claims, but on the beliefs and (...) norms of the culture of the knowledge community itself. To illustrate how socializing cognition can bias evaluations, we focus on peer-review practices, with some discussion of peer-review practices in philosophy. Data include responses to surveys by editors from general philosophy journals, as well as analyses of reviews and editorial decisions for the 2007 Cognitive Science Society Conference. (shrink)

In A Social History of Truth, Shapin engages these universal questions through an elegant recreation of a crucial period in the history of early modern science: ...

Scientists and 'anti-scientists' alike need a more realistic image of science. The traditional mode of research, academic science, is not just a 'method': it is a distinctive culture, whose members win esteem and employment by making public their findings. Fierce competition for credibility is strictly regulated by established practices such as peer review. Highly specialized international communities of independent experts form spontaneously and generate the type of knowledge we call 'scientific' - systematic, theoretical, empirically-tested, quantitative, and so on. Ziman shows (...) that these familiar 'philosophical' features of scientific knowledge are inseparable from the ordinary cognitive capabilities and peculiar social relationships of its producers. This wide-angled close-up of the natural and human sciences recognizes their unique value, whilst revealing the limits of their rationality, reliability, and universal applicability. It also shows how, for better or worse, the new 'post-academic' research culture of teamwork, accountability, etc. is changing these supposedly eternal philosophical characteristics. (shrink)

During the last three decades, reflections on the growth of scientific knowledge have inspired historians, sociologists, and some philosophers to contend that scientific objectivity is a myth. In this book, Kitcher attempts to resurrect the notions of objectivity and progress in science by identifying both the limitations of idealized treatments of growth of knowledge and the overreactions to philosophical idealizations. Recognizing that science is done not by logically omniscient subjects working in isolation, but by people with a variety of personal (...) and social interests, who cooperate and compete with one another, he argues that, nonetheless, we may conceive the growth of science as a process in which both our vision of nature and our ways of learning more about nature improve. Offering a detailed picture of the advancement of science, he sets a new agenda for the philosophy of science and for other "science studies" disciplines. (shrink)

A new, social epistemology of science that addresses practical as well as theoretical concerns.

In Science as Social Knowledge, Helen Longino offers a contextual analysis of evidential relevance. She claims that this "contextual empiricism" reconciles the objectivity of science with the claim that science is socially constructed. I argue that while her account does offer key insights into the role that values play in science, her claim that science is nonetheless objective is problematic.

This is an important book precisely because there is none other quite like it.

Psychometrically oriented researchers construe low inter-rater reliability measures for expert peer reviewers as damning for the practice of peer review. I argue that this perspective overlooks different forms of normatively appropriate disagreement among reviewers. Of special interest are Kuhnian questions about the extent to which variance in reviewer ratings can be accounted for by normatively appropriate disagreements about how to interpret and apply evaluative criteria within disciplines during times of normal science. Until these empirical-cum-philosophical analyses are done, it will remain (...) unclear the extent to which low inter-rater reliability measures represent reasonable disagreement rather than arbitrary differences between reviewers. (shrink)

Epistemological differences fuel continuous and frequently divisive debates in the social sciences and the humanities. Sociologists have yet to consider how such differences affect peer evaluation. The empirical literature has studied distributive fairness, but neglected how epistemological differences affect perception of fairness in decision making. The normative literature suggests that evaluators should overcome their epistemological differences by ‘‘translating’’ their preferred standards into general criteria of evaluation. However, little is known about how procedural fairness actually operates. Drawing on eighty-one interviews with (...) panelists serving on five multidisciplinary fellowship competitions in the social sciences and the humanities, we show that Evaluators generally draw on four epistemological styles to make arguments in favor of and against proposals. These are the constructivist, comprehensive, positivist, and utilitarian styles; and Peer reviewers define a fair decision-making process as one in which panelists engage in ‘‘cognitive contextualization,’’ that is, use epistemological styles most appropriate to the field or discipline of the proposal under review. (shrink)

Peer review of grant applications, it has been suggested, might be distorted by what is popularly termed old boyism, cronyism, or particularism. We argue that the existing debate emphasizes the more uninteresting aspects of the peer review system and that the operation of old boyism, as currently understood would have little effect on the overall direction of science. We identify a phenomenon of cognitive particularism, which we consider to be more important than the institutional cronyism analyzed in previous studies. We (...) illustrate with material drawn from observation of grant-awarding commit tees of the Science and Engineering Research CounciL In the concluding discussion, we explore some of the possible implications for the peer review system. (shrink)

Researchers display confirmation bias when they persevere by revising procedures until obtaining a theory-predicted result. This strategy produces findings that are overgeneralized in avoidable ways, and this in turn binders successful applications. (The 40-year history of an attitude-change phenomenon.

Published research findings are sometimes refuted by subsequent evidence, says Ioannidis, with ensuing confusion and disappointment.

Background: Editorial peer review is universally used but little studied. We examined the relationship between external reviewers' recommendations and the editorial outcome of manuscripts undergoing external peer-review at the Journal of General Internal Medicine. Methodology/Principal Findings: We examined reviewer recommendations and editors' decisions at JGIM between 2004 and 2008. For manuscripts undergoing peer review, we calculated chance-corrected agreement among reviewers on recommendations to reject versus accept or revise. Using mixed effects logistic regression models, we estimated intra-class correlation coefficients at the (...) reviewer and manuscript level. Finally, we examined the probability of rejection in relation to reviewer agreement and disagreement. The 2264 manuscripts sent for external review during the study period received 5881 reviews provided by 2916 reviewers; 28% of reviews recommended rejection. Chance corrected agreement on rejection among reviewers was 0.11. In mixed effects models adjusting for study year and manuscript type, the reviewer-level ICC was 0.23 and the manuscript-level ICC was 0.17. The editors' overall rejection rate was 48%: 88% when all reviewers for a manuscript agreed on rejection and 20% when all reviewers agreed that the manuscript should not be rejected. Conclusions/Significance: Reviewers at JGIM agreed on recommendations to reject vs. accept/revise at levels barely beyond chance, yet editors placed considerable weight on reviewers' recommendations. Efforts are needed to improve the reliability of the peer-review process while helping editors understand the limitations of reviewers' recommendations. © 2010 Kravitz et al. (shrink)

Do scientists agree? It is not only unrealistic to suppose that they do, but probably just as unrealistic to think that they ought to. Agreement is for what is already established scientific history. The current and vital ongoing aspect of science consists of an active and often heated interaction of data, ideas and minds, in a process one might call "creative disagreement." The "scientific method" is largely derived from a reconstruction based on selective hindsight. What actually goes on has much (...) less the flavor of a systematic method than of trial and error, conjecture, chance, competition and even dialectic. (shrink)