This paper reviews the published empirical evidence concerning journal peer review consisting of 68 papers, all but three published since 1975. Peer review improves quality, but its use to screen papers has met with limited success. Current procedures to assure quality and fairness seem to discourage scientific advancement, especially important innovations, because findings that conflict with current beliefs are often judged to have defects. Editors can use procedures to encourage the publication of papers with innovative findings such as invited papers, (...) early-acceptance procedures, author nominations of reviewers, structured rating sheets, open peer review, results-blind review, and, in particular, electronic publication. Some journals are currently using these procedures. The basic principle behind the proposals is to change the decision from whether to publish a paper to how to publish it. (shrink)

_Conjectures and Refutations_ is one of Karl Popper's most wide-ranging and popular works, notable not only for its acute insight into the way scientific knowledge grows, but also for applying those insights to politics and to history. It provides one of the clearest and most accessible statements of the fundamental idea that guided his work: not only our knowledge, but our aims and our standards, grow through an unending process of trial and error.

One of the most dramatic techniques for constructing accounts of "undiscovery" or incompetence in science involves the manipulative deception—in some accounts, the "hoaxing"—of the putatively incompetent researcher, ostensibly as an experiment to evaluate his or her methodology and the soundness of his or her knowledge claims. In this article, the author examines five cases in which such deceptions have been employed, noting the patterns of argument that typically follow these deceptions and the factors that seem to determine the power of (...) these arguments. The author suggests that the "hoaxer" in such cases is most likely to be successful when his or her views about the targeted researcher's methodology and knowledge claims are orthodox with respect to his or her intended audience. (shrink)

The failure to consider the Sokal affair in light of other, related episodes has contributed to a wholesale misreading of its significance. The episode has often been offered as evidence for the bankruptcy of a broad and diverse collection offields, variously referred to as cultural studies of science, sociology of science, history of science, and science and technology studies. However, when viewed in context, the Sokal affair illustrates pre cisely why social scientific and humanistic studies of science are necessary. To (...) develop this argument, the author explicitly compares Alan Sokal's experiment with a similar experiment, performed by William M. Epstein and published in this very journal. Comparing the research questions, methods, ethics, and reception of these two experi ments not only reveals the limitations of Sokal's critique but also shows that Sokal has unwittingly endorsed one of the central lines of research in science and technology studies. (shrink)

This article reports the results of a study of confirmational response bias among social work journals. A contrived research paper with positive findings and its negative mirror image were submitted to two different groups of social work journals and to two comparison groups of journals outside social work. The quantitative results, suggesting bias, are tentative; but the qualitative findings based upon an analysis of the referee comments are clear and consistent. Few referees from prestigious or nonprestcgrous social work journals prepared (...) reviews that were knowledgeable, scientifically astute, or objective. The best reviews came from journals outside of soccal work or from journals that are accepted as social work journals but originate with other disciplines. (shrink)

The displacement of the idea that facts and evidence matter by the idea that everything boils down to subjective interests and perspectives is -- second only to American political campaigns -- the most prominent and pernicious manifestation of anti-intellectualism in our time.