Claims that science should be more democratic than it is frequently arouse opposition. In this essay, I distinguish my own views about the democratization of science from the more ambitious theses defended by Paul Feyerabend. I argue that it is unlikely that the complexity of some scientific debates will allow for resolution according to the methodological principles of any formal confirmation theory, suggesting instead that major revolutions rest on conflicts of values. Yet these conflicts should not be dismissed as irresoluble.

Claims that science should be more democratic than it is frequently arouse opposition. In this essay, I distinguish my own views about the democratization of science from the more ambitious theses defended by Paul Feyerabend. I argue that it is unlikely that the complexity of some scientific debates will allow for resolution according to the methodological principles of any formal confirmation theory, suggesting instead that major revolutions rest on conflicts of values. Yet these conflicts should not be dismissed as irresoluble.

Because of its complexity, contemporary scientific research is almost always tackled by groups of scientists, each of which works in a different part of a given research domain. We believe that understanding scientific progress thus requires understanding this division of cognitive labor. To this end, we present a novel agent-based model of scientific research in which scientists divide their labor to explore an unknown epistemic landscape. Scientists aim to climb uphill in this landscape, where elevation represents the significance of the (...) results discovered by employing a research approach. We consider three different search strategies scientists can adopt for exploring the landscape. In the first, scientists work alone and do not let the discoveries of the community as a whole influence their actions. This is compared with two social research strategies, which we call the follower and maverick strategies. Followers are biased towards what others have already discovered, and we find that pure populations of these scientists do less well than scientists acting independently. However, pure populations of mavericks, who try to avoid research approaches that have already been taken, vastly outperform both of the other strategies. Finally, we show that in mixed populations, mavericks stimulate followers to greater levels of epistemic production, making polymorphic populations of mavericks and followers ideal in many research domains. (shrink)

Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the normative implications (...) of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to countenance the complexity of social interactions among actors involved directly and indirectly in peer review. (shrink)

Computer simulation of an epistemic landscape model, modified to include explicit representation of a centralized funding body, show the method of funding allocation has significant effects on communal trade-off between exploration and exploitation, with consequences for the community’s ability to generate significant truths. The results show this effect is contextual, and depends on the size of the landscape being explored, with funding that includes explicit random allocation performing significantly better than peer-review on large landscapes. The paper proposes a way of (...) incorporating external institutional factors in formal social epistemology, and offers a way of bringing such investigations to bear on current research policy questions. (shrink)

Exploratory inquiry has difficulty attracting research funding because funding agencies have little sense of how to detect good science in exploratory contexts. After documenting and explaining the focus on hypothesis testing among a variety of institutions responsible for distinguishing between good and bad science, I analyze the NIH grant review process. I argue that a good explanation for the focus on hypothesis testing—at least at the level of science funding agencies—is the fact that hypothesis-driven research is relatively easy to appraise. (...) I then explore one method by which we might gauge the epistemic merits of different styles of inquiry. (shrink)

In 2013 the Health Research Council of New Zealand began a stream of funding titled 'Explorer Grants', and in 2017 changes were introduced to the funding mechanisms of the Volkswagen Foundation 'Experiment!' and the New Zealand Science for Technological Innovation challenge 'Seed Projects'. All three funding streams aim at encouraging novel scientific ideas, and all now employ random selection by lottery as part of the grant selection process. The idea of funding science by lottery has emerged independently in several corners (...) of academia, including in philosophy of science. This paper reviews the conceptual and institutional landscape in which this policy proposal emerged, how different academic fields presented and supported arguments for the proposal, and how these have been reflected in actual policy. The paper presents an analytical synthesis of the arguments presented to date, notes how they support each other and shape policy recommendations in various ways, and where competing arguments highlight need for further analysis or more data. In addition, it provides lessons for how philosophers of science can engage in shaping science policy, and in particular highlights the importance of mixing complementary expertise: it takes a village to raise policy. (shrink)

At present, peer review is the most common method used by funding agencies to make decisions about resource allocation. But how reliable, efficient, and fair is it in practice? The ex ante evaluation of scientific novelty is a fundamentally uncertain endeavor; bias and chance are embedded in the final outcome. In the current study, I will examine some of the most central problems of peer review and highlight the possible benefits of using a lottery as an alternative decision-making mechanism. Lotteries (...) are driven by chance, not reason. The argument made in the study is that the epistemic landscape could benefit in several respects by using a lottery, thus avoiding all types of bias, disagreement, and other limitations associated with the peer review process. Funding agencies could form a pool of funding applicants who have minimal qualification levels and then select randomly within that pool. The benefits of a lottery would not only be that it saves time and resources, but also that it contributes to a more dynamic selection process and increases the epistemic diversity, fairness, and impartiality within academia. (shrink)

There are now several proposals for introducing random elements into the process of funding allocation for research, and some initial implementation of this policy by funding bodies. The proposals have been supported on efficiency grounds, with models, including social epistemology models, showing random allocation could increase the generation of significant truths in a community of scientists when compared to funding by peer review. The models in the literature are, however, fairly abstract. This paper introduces some of the considerations that are (...) required to build on the modelling work towards a fully-fledged policy proposal, including issues of cost and fairness. (shrink)

With this special issue, we would like to promote research on changes in the funding of the sciences, social sciences, and humanities. Since funding secures the livelihood of researchers and the means to do research, it is an indispensable condition for almost all research; as funding arrangements are undergoing dramatic changes, we think it timely to renew the science studies community’s efforts to understand the funding of research. Changes in the governance of science have garnered considerable attention from science studies (...) and higher education research; however, the impact of these changes on the conduct and content of research has not received sufficient attention, and theoretical insights into the connections between funding practices and research practices are few and far between. The aim of this special issue is to contribute to our theoretical understanding of the changing nature of research funding and its impact on the production of scientific knowledge. More specifically, we are interested in the interplay between funding and research practices: What is the impact of institutionalised funding arrangements on the production of scientific knowledge? (shrink)

While some form of evaluation has always been employed in science (e.g. peer review, hiring), formal systems of evaluation of research and researchers have recently come to play a more prominent role in many countries because of the adoption of new models of governance. According to such models, the quality of the output of both researchers and their institutions is measured, and issues such as eligibility for tenure or the allocation of public funding to research institutions crucially depends on the (...) outcomes of such measures. However, concerns have been raised over the risk that such evaluation may be threatening epistemic pluralism by penalizing the existent heterodox schools of thought and discouraging the pursuit of new ones. It has been proposed that this may happen because of epistemic bias favouring mainstream research programmes. In this paper, I claim that (1) epistemic pluralism is desirable and should be preserved; (2) formal evaluation exercises may threaten epistemic pluralism because they may be affected by some form of epistemic bias; therefore, (3) to preserve epistemic pluralism, we need some strategy to actively dampen epistemic bias. (shrink)

BackgroundWinning funding for health and medical research usually involves a lengthy application process. With success rates under 20%, much of the time spent by 80% of applicants could have been better used on actual research. An alternative funding system that could save time is using democracy to award the most deserving researchers based on votes from the research community. We aimed to pilot how such a system could work and examine some potential biases.MethodsWe used an online survey with a convenience (...) sample of Australian researchers. Researchers were asked to name the 10 scientists currently working in Australia that they thought most deserved funding for future research. For comparison, we used recent winners from large national fellowship schemes that used traditional peer review.ResultsVoting took a median of 5 min (inter-quartile range 3 to 10 min). Extrapolating to a national voting scheme, we estimate 599 working days of voting time (95% CI 490 to 728), compared with 827 working days for the current peer review system for fellowships. The gender ratio in the votes was a more equal 45:55 (female to male) compared with 34:66 in recent fellowship winners, although this could be explained by Simpson’s paradox. Voters were biased towards their own institution, with an additional 1.6 votes per ballot (inter-quartile range 0.8 to 2.2) above the expected number. Respondents raised many concerns about the idea of using democracy to fund research, including vote rigging, lobbying and it becoming a popularity contest.ConclusionsThis is a preliminary study of using voting that does not investigate many of the concerns about how a voting system would work. We were able to show that voting would take less time than traditional peer review and would spread the workload over many more reviewers. Further studies of alternative funding systems are needed as well as a wide discussion with the research community about potential changes. (shrink)

Peer review of grant applications, it has been suggested, might be distorted by what is popularly termed old boyism, cronyism, or particularism. We argue that the existing debate emphasizes the more uninteresting aspects of the peer review system and that the operation of old boyism, as currently understood would have little effect on the overall direction of science. We identify a phenomenon of cognitive particularism, which we consider to be more important than the institutional cronyism analyzed in previous studies. We (...) illustrate with material drawn from observation of grant-awarding commit tees of the Science and Engineering Research CounciL In the concluding discussion, we explore some of the possible implications for the peer review system. (shrink)