This article aims to discuss the contemporary activities and roles that scientific associations play in science and society. It is based on a comprehensive study of scientific associations in Portugal, relying on a multi-method, quantitative and qualitative approach. After a brief review of the literature on associations in the social studies of science, we provide an outline of the expanding field of scientific associations in Portugal. We then proceed to present and discuss the five main roles of associations identified through (...) the research: communication among peers, promotion of research, science dissemination, representation of professional interests and policy advice. We conclude that the external roles of associations have become more important than the internal ones. Whereas the internationalisation of science has moved the communication, collaboration and competition between researchers into the transnational sphere, the links that associations forge between science and other social spheres are still deeply rooted in national settings and much dependant on specific configurations and practices by government, business and other social actors. (shrink)

In this paper, we analyze the role of science and technology studies journal editors in organizing and maintaining the peer review economy. We specifically conceptualize peer review as a gift economy running on perpetually renewed experiences of mutual indebtedness among members of an intellectual community. While the peer review system is conventionally presented as self-regulating, we draw attention to its vulnerabilities and to the essential curating function of editors. Aside from inherent complexities, there are various shifts in the broader political–economic (...) and sociotechnical organization of scholarly publishing that have recently made it more difficult for editors to organize robust cycles of gift exchange. This includes the increasing importance of journal metrics and associated changes in authorship practices; the growth and differentiation of the STS journal landscape; and changes in publishing funding models and the structure of the publishing market through which interactions among authors, editors, and reviewers are reconfigured. To maintain a functioning peer review economy in the face of numerous pressures, editors must balance contradictory imperatives: the need to triage intellectual production and rely on established cycles of gift exchange for efficiency, and the need to expand cycles of gift exchange to ensure the sustainability and diversity of the peer review economy. (shrink)

“Quality” refers nominatively to a standard of performance. Quality is the central idea that differentiates speech protected by academic freedom (the right to worthwhile utterances) from constitutionally protected speech (the right to say anything at all). Extant documents and discussions state that professional peers determine quality based on norms of a field. But professional peers deem utterances and activities as consonant with quality only in reference to criteria that establish meaning of the term. In the absence of articulation, these criteria (...) are ambiguous. Consequently, there exists recurrent confusion about what faculty members have a defensible right to say and do. This article develops an ontology of quality in reference to higher education teaching, a component of academic careers generally not subject to extensive peer review and where instructors thereby exercise considerable autonomy. The ontology identifies three criteria that bound quality: constraint, context, and amplitude. Boundedness exists only insofar as boundaries are controlled. The article examines two types of problems in professional control that affect quality: slippage and overreach. Both are instances of organizational deviance and abrogation of professional ethics. It is argued that the patterns threaten the structural integrity and public confidence of faculty, fields, and higher education institutions. (shrink)

Charles Perrow used the term normal accidents to characterize a type of catastrophic failure that resulted when complex, tightly coupled production systems encountered a certain kind of anomalous event. These were events in which systems failures interacted with one another in a way that could not be anticipated, and could not be easily understood and corrected. Systems of the production of expert knowledge are increasingly becoming tightly coupled. Unlike classical science, which operated with a long time horizon, many current forms (...) of expert knowledge are directed at immediate solutions to complex problems. These are prone to breakdowns like the kind discussed by Perrow. The example of the Homestake mine experiment shows that even in modern physics complex systems can produce knowledge failures that last for decades. The concept of knowledge risk is introduced, and used to characterize the risk of failure in such systems of knowledge production. (shrink)

Prediction markets, which trade contracts based on the results of predictions, have been remarkably successful in predicting the results of political events. A number of proposals have been made to extend prediction markets to scientific questions, and some small-scale science prediction markets have been implemented. Advocates for science prediction markets argue that they could alleviate problems in science such as bias in peer review and epistemically unjustified consensus. I argue that bias in peer review and epistemically unjustified consensuses are genuine (...) problems in science, and that current attempts to solve them fail in practical circumstances. Prediction markets do show some promise for answering scientific questions while avoiding the pitfalls of consensus and peer review. However, there are strong reasons to believe that science prediction markets will not perform nearly so well as election prediction markets, and implementing prediction markets on a large scale could have significant unintended consequences for the organization of scientific research. The promise of prediction markets to solve current problems in assessing scientific claims is therefore largely illusory. (shrink)

Large differences in scientific productivity between male and female researchers have not yet been explained satisfactorily. This study finds that child care and lack of research collaboration are the two factors that cause significant gender differences in scientific publishing. Women with young children and women who do not collaborate in research with other scientists are clearly less productive than both their male and female colleagues.

This article analyzes the transformation of Minerva from an intellectual towards a scholarly journal by making use of bibliometric methods. The aim is to provide some empirical insights that help to understand what properties of the journal changed in the course of this transformation process. Minerva was one of the first journals that reflected on science and its role in society and science policy in particular. Analyzing the development of the journal sheds light on the emergence of science (policy) studies (...) and on Minerva’s role as a forerunner in this field. In a first step, the methods will be described. The second section provides some empirical results of the publication output of Minerva and its relations to other journals in the field. The empirical findings are put into a broader perspective in the concluding third section. (shrink)

This paper presents a framework that editors, peer reviewers, and authors can use to identify and resolve efficiently disputes that arise during peer review in scientific journals. The framework is called a scientific dialectical brief. In this framework, differences among authors and reviewers are formatted into specific assertions and the support each party provides for its position. A literature review suggests that scientists use five main types of support; empirical data, reasoning, speculation, feelings, and status. It is suggested that the (...) scientific dialectical brief format can streamline the review process by facilitating rapid differentiation between stronger and weaker support, so that valuable time can be focused on the better-substantiated claims. The paper concludes with some suggestions for implementation. (shrink)

This paper presents a framework that editors, peer reviewers, and authors can use to identify and resolve efficiently disputes that arise during peer review in scientific journals. The framework is called a scientific dialectical brief. In this framework, differences among authors and reviewers are formatted into specific assertions and the support each party provides for its position. A literature review suggests that scientists use five main types of support; empirical data, reasoning, speculation, feelings, and status. It is suggested that the (...) scientific dialectical brief format can streamline the review process by facilitating rapid differentiation between stronger and weaker support, so that valuable time can be focused on the better-substantiated claims. The paper concludes with some suggestions for implementation. (shrink)

Peer review is a topic of considerable concern to many researchers, and there is a correspondingly large body of research on the topic. This issue of Science and Engineering Ethics presents recent work on peer review that is both grounded in empirical science and is applicable to policy decisions. This research raises two basic questions; (a) how does current peer review operate, and (b) how can it be improved? Topics addressed include descriptions of how peer review is used in Federal (...) agencies. whether peer review leads to better manuscripts, demographic characteristics of authors or reviewers (status or institutional affiliation), blinding of reviewers, authors, or results, reliability and consistency of reviews, accepting a paper before the study is done, simultaneous submission, and use of dispute resolution procedures such as scientific dialectical and pleading protocols. (shrink)

This paper presents a framework that editors, peer reviewers, and authors can use to identify and resolve efficiently disputes that arise during peer review in scientific journals. The framework is called a scientific dialectical brief. In this framework, differences among authors and reviewers are formatted into specific assertions and the support each party provides for its position. A literature review suggests that scientists use five main types of support; empirical data, reasoning, speculation, feelings, and status. It is suggested that the (...) scientific dialectical brief format can streamline the review process by facilitating rapid differentiation between stronger and weaker support, so that valuable time can be focused on the better-substantiated claims. The paper concludes with some suggestions for implementation. (shrink)

Two analogies are at the foundation of editions of writings of scientists, technologists and physicians. Both are exemplified in the collection of ‘works’, texts of printed finished versions of contributions. The literary analogy is that of authorship, of the creation of a significant assemblage of words and other symbols. Assemblages of monographs and articles of a scientist are functionally no different than comparable arrays of the writings of theologians, philosophers, poets, novelists and historians.

The conventional history of modern scientific peer review begins with the censorship practices of the Royal Society of London in the 1660s. This article traces one strand of the “prehistory” of peer review in the writings of John Amos Comenius and other members of the Hartlib circle, a precursor group to the Royal Society of London. These reformers appear to have first envisioned peer review as a technique for theologians, only later proposing to apply it to philosophy. The importance of (...) peer review was as a technique that would permit a community of theologians or philosophers to resolve disputes internally rather than publicly, since public disputation would (they believed) sow doubt, error, and confusion, and disrupt the social order. (shrink)

Born in the 17th century, journal peer review is an extremely diverse technology, constantly torn between two often incompatible goals: the validation of manuscripts conceived as a collective industrial-like reproducible process performed to assert scientific statements, and the dissemination of articles considered as a means to spur scientific discussion, raising controversies, and civically challenging a state of knowledge. Such a situation is particularly conducive to clarifying the processes of valuation and evaluation in journal peer review. In this article, such processes (...) are considered as specific tests in order to emphasize the uncertain properties of pre-tests manuscripts. On the one hand, evaluation tests are examined at the core of the validation of manuscripts, such as defining the coordination of judging instances or controlling the modalities of inter-knowledge between reviewers and authors. They are also studied regarding the dissemination of articles, notably through the contemporary conception of a continuing evaluation test termed “post publication peer review”. On the other hand, valuation tests are both part of the validation of manuscripts, such as the weighting of different judgments of the same manuscript and the tensions that these hierarchies cause, and of the dissemination of articles, such as attention metrics recording the uses of articles. The conclusion sketches out how the articulation of these different tests has recently empowered readers as a new key judging instance for dissemination and for validation, potentially transforming the definition of peers, and thus the whole process of journal peer review. (shrink)

Rooted in altruism theory, the purpose of the double-blind academic journal peer-review process is to: assess the quality of scientific research, minimize the potential for nepotism, and; advance the standards of research through high-quality, constructive feedback. However, considering the limited, if any, public recognition and monetary incentives that referees receive for reviewing manuscripts, academics are often reluctant to squander their limited time toward peer reviewing manuscripts. If they do accept such invitations, referees, at times, do not invest the appropriate time (...) needed and, as a result, scantily review manuscripts, which adversely affects the quality of the review. In addition, given that authors’ identities are not blind to journal editors, there is the potential for bias toward well-established academics from highly-ranked institutions. As a result of these issues, the aims of the academic journal review process are currently not being fulfilled. To rectify these issues, several recommendations, namely: single-blind the editors, pay reviewers, standardize the review process, increase the acceptance standards at academic conferences, and provide constructive feedback, are offered. (shrink)

The main aim of this article is to start a discussion of social pattern, a term that is commonly used in sociology but not specified or defined. The key question can be phrased as follows: Is it possible to transform the notion of social pattern from its current status in sociology as a proto-concept into a fully worked out concept? And if so, how can this be done? To provide material for the discussion we begin by introducing a few different (...) types of patterns that are currently being used (patterns in nature, cultural patterns, statistical patterns, and computationally generated patterns). This is followed by a suggestion for what a strictlysociologicalconcept of social pattern may look like. A useful and theoretically solid concept of social pattern can in our view be constructed by basing it on Weber’s concept of social action. This means thatboththe behavior of the actorsandthe meaning these invest their behavior with must be taken into account. The article ends with a brief discussion of how to use the concept of social patterns in an effective way and what may endanger such a use. (shrink)

For over 50 years, Minerva has been one of the leading independent journals in the study of ‘science, learning and policy’. Its pages have much to say about the origins and conduct of the ‘intellectual Cold War’, the defence of academic freedom, the emergence of modernization theory, and pioneering strategies in the social studies of science. This paper revisits Minerva through the life and times of its founding Editor, Edward Shils, and traces his influence on its early years – from (...) its association with the Congress for Cultural Freedom in the 1950s, to the higher reaches of research policy in the 1990s. At the close of his life, Minerva continued to espouse Shils’ commitment to what he saw as the fundamental Enlightenment traditions of consensus, civility, and community. In the 21st century, with the achievements of science producing rapid change in every walk of life, his legacy so far retains an established place in the history of scholarship. Whether that legacy will endure – and if so, what role Minerva will play in its defence – remain key questions for the coming generation. (shrink)

Scholars belong to multiple communities of credit simultaneously. When these communities disagree about a scholarly achievement’s credit assignment, this raises a puzzle for decision and game theor...

Under the traditional system of peer-reviewed publication, the degree of prestige conferred to authors by successful publication is tied to the degree of the intellectual rigor of its peer review process: ambitious scientists do well professionally by doing well epistemically. As a result, we should expect journal editors, in their dual role as epistemic evaluators and prestige-allocators, to have the power to motivate improved author behavior through the tightening of publication requirements. Contrary to this expectation, I will argue that the (...) publication bias literature in academic medicine demonstrates that editor interventions have had limited effectiveness in improving the health of the publication and trial registration record, suggesting that much stronger interventions are needed. (shrink)

An empirically sensitive formulation of the norms of transformative criticism must recognize that even public and shared standards of evaluation can be implemented in ways that unintentionally perpetuate and reproduce forms of social bias that are epistemically detrimental. Helen Longino’s theory can explain and redress such social bias by treating peer evaluations as hypotheses based on data and by requiring a kind of perspectival diversity that bears, not on the content of the community’s knowledge claims, but on the beliefs and (...) norms of the culture of the knowledge community itself. To illustrate how socializing cognition can bias evaluations, we focus on peer-review practices, with some discussion of peer-review practices in philosophy. Data include responses to surveys by editors from general philosophy journals, as well as analyses of reviews and editorial decisions for the 2007 Cognitive Science Society Conference. (shrink)

Psychometrically oriented researchers construe low inter-rater reliability measures for expert peer reviewers as damning for the practice of peer review. I argue that this perspective overlooks different forms of normatively appropriate disagreement among reviewers. Of special interest are Kuhnian questions about the extent to which variance in reviewer ratings can be accounted for by normatively appropriate disagreements about how to interpret and apply evaluative criteria within disciplines during times of normal science. Until these empirical-cum-philosophical analyses are done, it will remain (...) unclear the extent to which low inter-rater reliability measures represent reasonable disagreement rather than arbitrary differences between reviewers. (shrink)