Peer reviews of 84 organic farming grant applications from Sweden were analyzed to determine whether the reviewers’ affiliation to one of two types of agriculture (i.e., organic and conventional) influenced their reviews. Fifteen reviewers were divided into three groups: (1) scientists with experience in organic farming research; (2) scientists with no experience in organic farming research; and (3) users of organic farming research. The two groups of scientists assessed the societal relevance and scientific quality of the grant applications based on (...) three criteria (i.e., presentation, methodology, qualifications), whereas the user group only assessed societal relevance. The analysis showed that the two groups of scientists provided very different reviews. Scientist reviewers with experience in organic farming research agreed more with the user group on research relevance than did scientist reviewers without such experience, and the assessment of relevance was closely correlated to the assessment of scientific quality within both scientific groups. As both scientific groups did not clearly distinguish between societal relevance and scientific quality, the idea of an objective science is challenged. The contextual values associated with the norms of good agriculture were not clearly distinguished from the constitutive values of science associated with the traditional norms of good science. This raises the question of whether organic and conventional grant applications should be mixed for review regardless of the reviewers. (shrink)

The documented low levels of reliability of the peer review process present a serious challenge to editors who must often base their publication decisions on conflicting referee recommendations. The purpose of this article is to discuss this process and examine ways to produce a more reliable and useful peer review system.

Rl In response to Somit & Peterson's call for multiple journal manuscript submissions, and consistent with Cicchetti (1991a and 1991b), counterarguments are presented. The policy for multiple submissions is difficult to defend scientifically ana would place an unwarranted burden on both reviewers and journal editors. As such the policy is again rejected. R2 As earlier hypothesized, referee agreement on manuscripts submitted to a major journal in chemistry was significantly higher for acceptance than for rejection. This is consistent with the high (...) acceptance rate (>70%) of this journal. There is significantly more referee agreement on rejection than on acceptance for manuscripts submitted to major journals, in behavioral science and medicine. These journals have high rejection rates (often >70%). (shrink)