Scientific realists have suggested that changes in our scientific communities over the course of their history have rendered those communities progressively less vulnerable to the problem of unconcieved alternatives over time. I argue in response not only that the most fundamental historical transformations of the scientific enterprise have generated steadily mounting obstacles to revolutionary, transformative, or unorthodox scientific theorizing, but also that we have substantial independent evidence that the institutional apparatus of contemporary scientific inquiry fosters an exceedingly and increasingly theoretically (...) conservative form of that inquiry. I conclude that contemporary scientific communities are actually more vulnerable to the problem of unconceived alternatives than their historical predecessors, and I briefly suggest how we might seek to pursue scientific inquiry in a less theoretically conservative way. (shrink)

Prepublication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behaviour of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing prepublication peer review can be evaluated with a reasonable level of confidence based on presently available evidence, they are either positive or neutral. (...) We conclude that on present evidence abolishing peer review weakly dominates the status quo. (shrink)

Noesis is an Internet search engine dedicated to mapping the profession of philosophy online. In this paper, I recount the history of the project's development since 1998 and discuss the role it may play in representing philosophy optimally, adequately, fairly, and accessibly. Unlike many other representations of philosophy, Noesis is dynamic in the sense that it constantly changes and inclusive in the sense that it lets the profession speak for itself about what philosophy is, how it is practiced, and why (...) it is important. In this paper, I explain how Noesis is dynamic and inclusive. I close by suggesting why such a communitarian representation of the profession is both timely and necessary. (shrink)

This study seeks to examine the impact of alumni connections between the evaluators and evaluatees on the results of peer review ratings for the Korean national R&D project and selection success rate. Specifically, this study analyzed the evaluation results of 8,402 research proposal entries submitted between 2007 and 2011 for the “general researchers support project,” all in the Natural Science and Engineering areas and sponsored by the National Research Foundation of Korea. Each proposal entry was evaluated by three evaluators, and (...) approximately 39 percent of the proposals had at least one evaluator from the same university. The results of this study showed that evaluators have a tendency to give relatively high scores to research proposals submitted by the alumni of the same universities as their alma mater. Also, when an evaluator from the same university as an evaluatee was included in the evaluator group, the results of this study showed that the percentage of entry submissions was higher compared to the contrary. Such results show that in the process of peer review–based research proposal evaluations for national R&D projects, alumni connections have significant influence on evaluation results in South Korea. (shrink)

Recently, mainstream philosophy journals have tended to implement more and more stringent forms of peer review, probably in an attempt to prevent editorial decisions that are based on factors other than quality. Against this trend, we propose that journals should relax their standards of acceptance, as well as be less restrictive about whom is to decide what is admitted into the debate. We start by arguing, partly on the basis of the history of peer review in the journal Mind, that (...) past and current peer review practices attest to partisanship with respect to philosophical approach. Then, we explain that such partisanship conflicts with the standard aims of peer review, and that it is both epistemically and morally problematic. This assessment suggests that, if feasible, journals should treat all available and proposed standards of acceptance in philosophy as epistemically equal, and that philosophical work should be evaluated in terms of the novelty and significance of its contribution to developing thought in ways that are of value. Finally, we show, in a programmatic way, that improving the current situation is feasible, and can be done fairly easily. (shrink)

The nature of authority is shifting in online scholarly communication. This examination of the history and future of peer review argues that effective online communication requires the development of an open, community?oriented, post?publication system of peer?to?peer review, transforming peer review from a process focused on gatekeeping to one concerned with filtering the wealth of scholarly material made available via the Internet.

There is an increasing push by journals to ensure that data and products related to published papers are shared as part of a cultural move to promote transparency, reproducibility, and trust in the scientific literature. Yet few journals commit to evaluating their effectiveness in implementing reporting standards aimed at meeting those goals (1, 2). Similarly, though the vast majority of journals endorse peer review as an approach to ensure trust in the literature, few make their peer review data available to (...) evaluate effectiveness toward achieving concrete measures of quality, including consistency and completeness in meeting reporting standards. Remedying these apparent disconnects is critical for closing the gap between guidance recommendations and actual reporting behavior. We see this as a collective action problem requiring leadership and investment by publishers, who can be incentivized through mechanisms that allow them to manage reputational risk and through continued innovation in journal assessment. (shrink)

The notion of quality is analysed for its functional roots as a social heuristic for reusing others’ quality judgements and hence aiding choice. This is applied to the context of academic publishing, where the costs of publishing have greatly decreased, but the problem of finding the papers one wants has become harder. This paper suggests that instead of relying on generic quality judgements, such as those delivered by journal reviewers, that the maximum amount of judgemental information be preserved and then (...) made available to potential readers to help them find papers that meet their particular needs. The suggestion is that: multidimensional quality data be captured on review of papers, this information is stored on a database, and then used to filter papers according to the criteria set by the searcher—personalising the quality filter. In other words the quality judgements and subsequent use are maintained in a disaggregated form, maintaining the maximum informational context of the judgements for future use. The advantages, disadvantages, challenges and possible variations of this proposal are discussed. (shrink)