This paper reviews the published empirical evidence concerning journal peer review consisting of 68 papers, all but three published since 1975. Peer review improves quality, but its use to screen papers has met with limited success. Current procedures to assure quality and fairness seem to discourage scientific advancement, especially important innovations, because findings that conflict with current beliefs are often judged to have defects. Editors can use procedures to encourage the publication of papers with innovative findings such as invited papers, (...) early-acceptance procedures, author nominations of reviewers, structured rating sheets, open peer review, results-blind review, and, in particular, electronic publication. Some journals are currently using these procedures. The basic principle behind the proposals is to change the decision from whether to publish a paper to how to publish it. (shrink)

A quarter-century ago, during a period when belief in the robustness of classical tests on means was practically a professional shibboleth, a series of large, carefully controlled, and well-validated experiments and sampling studies (supplemented and supported by extensive mathematical derivations) dramatically showed that highly publicized claims of robustness were insufficiently qualified and that extreme nonrobustness could occur under perfectly reasonable experimental and testing conditions. When these findings were published in technical reports, they tended either to be ignored or to be (...) so misrepresented and distorted by those who cited them as to make them appear to support, rather than question, the very claims of robustness they tended to refute. Attempts to publish these iconoclastic results in many of the most renowned professional journals were met with rejection based upon reviewer comments so illogical or fatuous as to be interpreted only as an indication of either contrived obstructionism or pathetic professional incompetence. Eventual acceptance by a few refereed journals could, in every case, be interpreted as a political-type fluke. (shrink)

This research examined the experiences of authors of academic journal articles in the educational sector of all eight universities in Finland. The ethical principles of peer review and best and worst review processes were in focus. Data were gathered by electronic questionnaire, which was completed by 121 respondents who represented well the heterogeneity of the staff in the educational sector. Out of nine ethical principles honesty, constructiveness, and impartiality were appreciated but promptness, balance, and diplomacy were criticized. According to two (...) open questions, a third of authors praised and blamed reviewers as experts and non-experts. The accuracy of feedback was more often present in the best rather than in the worst experienced review processes. Journals’ editors and their decision-making called forth more negative than positive accounts. The results were discussed in the context of ethical codes for reviewers and researchers’ findings regarding the ethical responsibilities to promote good science with thorough, appropriate, and honest feedback and feedforward. (shrink)