A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published (...) research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices.With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated. (shrink)

A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.

In this paper I will be implicitly defending the following thesis: An individual X obtains knowledge of scientific claim p in virtue of being a member of a community A that regards claim p as knowledge. The thesis states is that a claim p only becomes scientific knowledge once it's been through a process of validation by a scientific community. This is meant to be contrasted with the claim that individuals first obtain scientific knowledge perception or inference, and then transmit (...) it to their colleagues, without the community playing any epistemological role. The strategy that I will follow is the following. In the first section I will consider the claim “that collaboration plays a causal role in advancing scientists' epistemic goals, and that its growing popularity is a consequence of its effectiveness in aiding communities of scientists to realize their epistemic goals”. I will conclude that the claim is rather weak in the sense that it only justifies certain sections of scientific practice and does not establish that in principle scientific knowledge is produced in the manner described above. An attempt to strengthen the thesis will be made through the presentation of evidence that all through history in what is widely recognised as scientific activity the scientist is never alone, even if they are the single author of a scientific work. I will draw on certain insights from Latour's study in the making of scientific knowledge to support the thesis that the individual scientist is necessarily surrounded by allies. This attempt will consist of two parts, the first being what the exploration of what I term the intra-laboratory aspect of scientific activity, and the second being the public forum aspect. I will conclude that the latter aspect is the aspect which supports the claim that the production of scientific knowledge is in principle social, that is that the appropriate unit of epistemological analysis of the production of scientific knowledge is the scientific community rather than the individual scientist. Finally, I will promote the thesis that community agreement is constitutive of knowledge, presenting and arguing for the communitarian account of scientific knowledge. I will briefly argue against an individualistic conception of knowledge acquisition, based on the model of the solitary Cartesian thinker and the notion that knowing something involves being in a certain mental state, and then briefly talk about belief as the property of plural subjects before I move on to present the communitarian model of knowledge acquisition. (shrink)