When the New Organon appeared in 1620, part of a six-part programme of scientific inquiry entitled 'The Great Renewal of Learning', Francis Bacon was at the high point of his political career, and his ambitious work was groundbreaking in its attempt to give formal philosophical shape to a new and rapidly emerging experimentally-based science. Bacon combines theoretical scientific epistemology with examples from applied science, examining phenomena as various as magnetism, gravity, and the ebb and flow of the tides, and anticipating (...) later experimental work by Robert Boyle and others. His work challenges the entire edifice of the philosophy and learning of his time, and has left its mark on all subsequent philosophical discussions of scientific method. This volume presents a new translation of the text into modern English by Michael Silverthorne, and an introduction by Lisa Jardine that sets the work in the context of Bacon's scientific and philosophical activities. (shrink)

Francis Bacon, lawyer, statesman, and philosopher, remains one of the most effectual thinkers in European intellectual history. We can trace his influence from Kant in the 1700s to Darwin a century later. The Advancement of Learning , first published in 1605, contains an unprecedented and thorough systematization of the whole range of human knowledge. Bacon’s argument that the sciences should move away from divine philosophy and embrace empirical observation would forever change the way philosophers and natural scientists interpret their world.

Thomas S. Kuhn's classic book is now available with a new index.

This paper addresses some critical issues in the applicability of quantitative performance measures (including bibliometric, economic, and co-occurrence measures) to the assessment of basic research. The strengths and weaknesses of metrics applied as research performance measures are examined. It is concluded that metrics have a useful role to play in the evaluation of research. Each metric employed, whether bibliometric, economic, co-occurrence, or others, brings a new dimension of potential insight to the complex problem of research assessment. However, when used in (...) a stand-alone mode, metrics can be easily misinterpreted and result in misleading conclusions. Metrics should be an integral part of a more comprehensive approach to research evaluation, in which the leading role is assumed by expert peer review. Under this structure, the reviewers must be broadly constituted, so that the technical correctness of the research approaches being taken (doing the job right) can be evaluated by technical experts and the correctness of the research target (doing the right job) can be evaluated as well. This comprehensive approach insures quality in the implementation of existing research paradigms and allows the introduction of revolutionary new research paradigms as well. (shrink)

The principles and practices of research peer review are described. While the principles are fundamentally generic and apply to peer review across the full spectrum of performing institutions as well as to manuscript/proposal/program peer review, the focus of this paper is peer review of proposed and ongoing programs in federal agencies. The paper describes desireable characteristics and important intangible factors in successful peer review. Also presented is a heuristic protocol for the conduct of successful peer review research evaluations and impact (...) assessments. Problems with peer review are then outlined, followed by examples of peer review of proposed and existing programs in selected federal agencies. Some peer review variants, such as the Science Court, are described, and then research requirements to improve peer review are discussed. (shrink)

Scientific editors’ policies, including peer review, are based mainly on tradition and belief. Do they actually achieve their desired effects, the selection of the best manuscripts and improvement of those published? Editorial decisions have important consequences—to investigators, the scientific community, and all who might benefit from correct information or be harmed by misleading research results. These decisions should be judged not just by intentions of reviewers and editors but also by the actual consequences of their actions. A small but growing (...) number of studies has put editorial policies to a strong scientific test. In a randomized, controlled trial. blinding reviewers to author and institution was usually successful and improved the quality of reviews. Two studies have shown that, contrary to conventional wisdom, reviewers early in their careers give better reviews than senior reviewers. Many studies have shown low agreement between reviewers but there is disagreement about whether this represents a failing of peer review or the expected and valuable effect of choosing reviewers with complementary expertise. In a study of whether manuscripts are improved by peer review and editing, articles published in Annals of Internal Medicine were improved in 33 of 34 dimensions of reporting quality, but published articles still had room for improvement. Because of the central place of peer review in the scientific community and the resources it requires, more studies are needed to define what it does and does not accomplish. (shrink)

Scientific editors’ policies, including peer review, are based mainly on tradition and belief. Do they actually achieve their desired effects, the selection of the best manuscripts and improvement of those published? Editorial decisions have important consequences—to investigators, the scientific community, and all who might benefit from correct information or be harmed by misleading research results. These decisions should be judged not just by intentions of reviewers and editors but also by the actual consequences of their actions. A small but growing (...) number of studies has put editorial policies to a strong scientific test. In a randomized, controlled trial. blinding reviewers to author and institution was usually successful and improved the quality of reviews. Two studies have shown that, contrary to conventional wisdom, reviewers early in their careers give better reviews than senior reviewers. Many studies have shown low agreement between reviewers but there is disagreement about whether this represents a failing of peer review or the expected and valuable effect of choosing reviewers with complementary expertise. In a study of whether manuscripts are improved by peer review and editing, articles published in Annals of Internal Medicine were improved in 33 of 34 dimensions of reporting quality, but published articles still had room for improvement. Because of the central place of peer review in the scientific community and the resources it requires, more studies are needed to define what it does and does not accomplish. (shrink)

The documented low levels of reliability of the peer review process present a serious challenge to editors who must often base their publication decisions on conflicting referee recommendations. The purpose of this article is to discuss this process and examine ways to produce a more reliable and useful peer review system.

The major challenge facing today’s biomedical researchers is the increasing competition for available funds. The competitive review process, through which the National Institutes of Health (NIH) awards grants, is built upon review by a committee of expert scientists. The NIH is firmly committed to ensuring that its peer review system is fair and objective.

This paper reviews the published empirical evidence concerning journal peer review consisting of 68 papers, all but three published since 1975. Peer review improves quality, but its use to screen papers has met with limited success. Current procedures to assure quality and fairness seem to discourage scientific advancement, especially important innovations, because findings that conflict with current beliefs are often judged to have defects. Editors can use procedures to encourage the publication of papers with innovative findings such as invited papers, (...) early-acceptance procedures, author nominations of reviewers, structured rating sheets, open peer review, results-blind review, and, in particular, electronic publication. Some journals are currently using these procedures. The basic principle behind the proposals is to change the decision from whether to publish a paper to how to publish it. (shrink)

A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published (...) research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices.With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated. (shrink)