Scientific research and ethics -- Ethical theory and decision making -- Data acquisition and management -- Mentoring and professional relationship -- Collaboration in research -- Authorship -- Publication and peer review -- Misconduct in research -- Intellectual property -- Conflicts of interest and scientific objectivity -- The use of animals in research -- The use of human subjects in research -- The use of vulnerable subjects in research -- Genetics, cloning, and stem cell research -- International research.

This paper reviews the published empirical evidence concerning journal peer review consisting of 68 papers, all but three published since 1975. Peer review improves quality, but its use to screen papers has met with limited success. Current procedures to assure quality and fairness seem to discourage scientific advancement, especially important innovations, because findings that conflict with current beliefs are often judged to have defects. Editors can use procedures to encourage the publication of papers with innovative findings such as invited papers, (...) early-acceptance procedures, author nominations of reviewers, structured rating sheets, open peer review, results-blind review, and, in particular, electronic publication. Some journals are currently using these procedures. The basic principle behind the proposals is to change the decision from whether to publish a paper to how to publish it. (shrink)

A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published (...) research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices.With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated. (shrink)

A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.

Psychometrically oriented researchers construe low inter-rater reliability measures for expert peer reviewers as damning for the practice of peer review. I argue that this perspective overlooks different forms of normatively appropriate disagreement among reviewers. Of special interest are Kuhnian questions about the extent to which variance in reviewer ratings can be accounted for by normatively appropriate disagreements about how to interpret and apply evaluative criteria within disciplines during times of normal science. Until these empirical-cum-philosophical analyses are done, it will remain (...) unclear the extent to which low inter-rater reliability measures represent reasonable disagreement rather than arbitrary differences between reviewers. (shrink)

Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the normative implications (...) of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to countenance the complexity of social interactions among actors involved directly and indirectly in peer review. (shrink)

This article systematically reviews published studies of the association of pharmaceutical industry funding and clinical trial results, as well a few closely related studies. It reviews two earlier results, and surveys the recent literature. Results are clear: Pharmaceutical company sponsorship is strongly associated with results that favor the sponsors' interests.

This article reports the results of an anonymous survey of researchers at a government research institution concerning their perceptions about ethical problems with journal peer review. Incompetent review was the most common ethical problem reported by the respondents, with 61.8% (SE = 3.3%) claiming to have experienced this at some point during peer review. Bias (50.5%, SE = 3.4%) was the next most common problem. About 22.7% (SE = 2.8%) of respondents said that a reviewer had required them to include (...) unnecessary references to his/her publication(s), 17.7% (SE = 2.6%) said that comments from reviewers had included personal attacks, and 9.6% (SE = 2.0%) stated that reviewers had delayed publication to publish a paper on the same topic. Two of the most serious violations of peer review ethics, breach of confidentiality (6.8%, SE = 1.7%) and using ideas, data, or methods without permission (5%, SE = 1.5%) were perceived less often than the other problems. We recommend that other investigators follow up on our exploratory research with additional studies on the ethics of peer review. (shrink)

"Difficult to put down.... I have studied these issues for the better part of a decade and learned from this book not only about new cases but also about the intersection of law, science, and government."—Daryl E. Chubin, author of Peerless Science: Peer Review in United States Science Policy "Thoughtful, clear, and very well written... will be the basis of how the issues are defined, what the options and their problems are, and what other features lurk on the horizon."—Lawrence Badash, (...) University of California, Santa Barbara. (shrink)

Editors have a responsibility to retract seriously flawed articles from their journals. However, there appears to be little consistency in journals’ policies or procedures for this. In a qualitative study, we therefore interviewed editors of science journals using semi-structured interviews to investigate their experience of retracting articles. We identified potential barriers to retraction, difficulties in the process and also sources of support and encouragement. Our findings have been used as the basis for guidelines developed by the Committee on Publication Ethics.

Background: Editorial peer review is universally used but little studied. We examined the relationship between external reviewers' recommendations and the editorial outcome of manuscripts undergoing external peer-review at the Journal of General Internal Medicine. Methodology/Principal Findings: We examined reviewer recommendations and editors' decisions at JGIM between 2004 and 2008. For manuscripts undergoing peer review, we calculated chance-corrected agreement among reviewers on recommendations to reject versus accept or revise. Using mixed effects logistic regression models, we estimated intra-class correlation coefficients at the (...) reviewer and manuscript level. Finally, we examined the probability of rejection in relation to reviewer agreement and disagreement. The 2264 manuscripts sent for external review during the study period received 5881 reviews provided by 2916 reviewers; 28% of reviews recommended rejection. Chance corrected agreement on rejection among reviewers was 0.11. In mixed effects models adjusting for study year and manuscript type, the reviewer-level ICC was 0.23 and the manuscript-level ICC was 0.17. The editors' overall rejection rate was 48%: 88% when all reviewers for a manuscript agreed on rejection and 20% when all reviewers agreed that the manuscript should not be rejected. Conclusions/Significance: Reviewers at JGIM agreed on recommendations to reject vs. accept/revise at levels barely beyond chance, yet editors placed considerable weight on reviewers' recommendations. Efforts are needed to improve the reliability of the peer-review process while helping editors understand the limitations of reviewers' recommendations. © 2010 Kravitz et al. (shrink)