Computer simulation of an epistemic landscape model, modified to include explicit representation of a centralized funding body, show the method of funding allocation has significant effects on communal trade-off between exploration and exploitation, with consequences for the community’s ability to generate significant truths. The results show this effect is contextual, and depends on the size of the landscape being explored, with funding that includes explicit random allocation performing significantly better than peer-review on large landscapes. The paper proposes a way of (...) incorporating external institutional factors in formal social epistemology, and offers a way of bringing such investigations to bear on current research policy questions. (shrink)

Computer simulation of an epistemic landscape model, modified to include explicit representation of a centralized funding body, show the method of funding allocation has significant effects on communal trade-off between exploration and exploitation, with consequences for the community’s ability to generate significant truths. The results show this effect is contextual, and depends on the size of the landscape being explored, with funding that includes explicit random allocation performing significantly better than peer review on large landscapes. The article proposes a way (...) of incorporating external institutional factors in formal social epistemology, and offers a way of bringing such investigations to bear on current research policy questions. 1Introduction2Theoretical Background3Model Description4Simulation Details 4.1Simulating the epistemic landscape4.2Simulating agents4.3Simulating communal knowledge4.4Simulating funding strategies4.5Simulating merit dynamics5Results and Discussion 5.1Experiment 1: The winner-takes-it-all mechanism only5.2Experiment 2: All dynamic mechanisms5.3Experiment 3: Adding a new funding mechanism 5.4Experiment 4: Varying the degree of myopia5.5Experiment 5: Variability of individual epistemic gain5.6Experiment 6: Likelihood of renewal6Discussion7Conclusion. (shrink)

These preprints were automatically compiled into a PDF from the collection of papers deposited in PhilSci-Archive in conjunction with the PSA 2018.

The main goal of publicly funded biomedical research is to generate social value through the creation and application of knowledge that can improve the well-being of current and future people. Prioritising research with the greatest potential social value is crucial for good stewardship of limited public resources and ensuring ethical involvement of research participants. At the National Institutes of Health (NIH), peer reviewers hold the expertise and responsibility for social value assessment and resulting prioritisation at the project level. However, previous (...) research has shown that peer reviewers place more emphasis on a study’s methods (‘Approach’) than on its potential social value (best approximated by the criterion of ‘Significance’). Lower weighting of Significance may be due to reviewers’ views on the relative importance of social value, their belief that social value is evaluated at other stages of the research priority-setting process or the lack of guidance on how to approach the challenging task of assessing expected social value. The NIH is currently revising its review criteria and how these criteria contribute to overall scores. To elevate the role of social value in priority setting, the agency should support empirical research on how peer reviewers approach the assessment of social value, provide more specific guidance for reviewing social value and experiment with alternative reviewer assignment strategies. These recommendations would help ensure that funding priorities align with the NIH’s mission and the obligation of taxpayer-funded research to contribute to the public good. (shrink)

Throughout scientific history, there have been cases of mainstream science dismissing novel ideas of less prominent researchers. Nowadays, many researchers with different social and academic backgrounds, origins and gender identities work together on topics of crucial importance. Still, it is questionable whether the privileged groups consider the views of underprivileged colleagues with sufficient attention. To profit from the diversity of thoughts, the scientific community first has to be open to minority viewpoints and epistemically include them in mainstream research. Moreover, the (...) idea of inclusive science poses stronger requirements than the paradigm of open science. We argue that the concept of integration of different opinions is insufficient because the process of integration assumes adjusting oneself to the majority view and fitting into the dominant paradigm while contributing only with smaller amendments. Epistemic inclusion, on the other hand, means dynamically changing the research paradigm during the interaction with diverse methods and hypotheses. The process of inclusion preserves marginalized views and increases epistemic justice. (shrink)

To anyone vaguely aware of Feyerabend, the title of this paper would appear as an oxymoron. For Feyerabend, it is often thought, science is an anarchic practice with no discernible structure. Against this trend, I elaborate the groundwork that Feyerabend has provided for the beginnings of an approach to organizing scientific research. Specifically, I argue that Feyerabend’s pluralism, once suitably modified, provides a plausible account of how to organize science. These modifications come from C.S. Peirce’s account of the economics of (...) theory pursuit, which has since been corroborated by empirical findings in the social sciences. I go on to contrast this approach with the conception of a ‘well-ordered science’ as outlined by Kitcher (Science, truth, and democracy, Oxford University Press, New York, 2001), Cartwright (Philos Sci 73(5):981–990, 2006), which rests on the assumption that we can predict the content of future research. I show how Feyerabend has already given us reasons to think that this model is much more limited than it is usually understood. I conclude by showing how models of resource allocation, specifically those of Kitcher (J Philos 87:5–22, 1990), Strevens (J Philos 100(2):55–79, 2003) and Weisberg and Muldoon (Philos Sci 76(2):225–252, 2009), unwittingly make use of this problematic assumption. I conclude by outlining a proposed model of resource allocation where funding is determined by lottery and briefly examining the extent to which it is compatible with the position defended in this paper. (shrink)

Feyerabend is infamous for his defense of pluralism, which he extends to every topic he discusses. Disagreement, a by-product of this pluralism, becomes a sign of flourishing critical communities. In Feyerabend’s political works, he extends this pluralism from science to democratic societies and incorporates his earlier work on scientific methodology into a procedure for designing just policy. However, a description and analysis of Feyerabend’s conception of disagreement is lacking. In this paper, I reconstruct and assess Feyerabend’s conception of disagreement, with (...) a particular emphasis on the role of experts, and its role in the formation of science policy. I go on to assess this argument in light of recent literature on manufactured disagreement on politically contentious science policy. I conclude by suggesting some prospects and problems for de-idealizing Feyerabend’s position on disagreement to see whether it may be plausibly implemented. (shrink)

At present, peer review is the most common method used by funding agencies to make decisions about resource allocation. But how reliable, efficient, and fair is it in practice? The ex ante evaluation of scientific novelty is a fundamentally uncertain endeavor; bias and chance are embedded in the final outcome. In the current study, I will examine some of the most central problems of peer review and highlight the possible benefits of using a lottery as an alternative decision-making mechanism. Lotteries (...) are driven by chance, not reason. The argument made in the study is that the epistemic landscape could benefit in several respects by using a lottery, thus avoiding all types of bias, disagreement, and other limitations associated with the peer review process. Funding agencies could form a pool of funding applicants who have minimal qualification levels and then select randomly within that pool. The benefits of a lottery would not only be that it saves time and resources, but also that it contributes to a more dynamic selection process and increases the epistemic diversity, fairness, and impartiality within academia. (shrink)

Naturalistic philosophers rely on literature search and review in a number of ways and for different purposes. Yet this article shows how processes of literature search and review are likely to be affected by widespread and systematic biases. A solution to this problem is offered here. Whilst the tradition of systematic reviews of literature from scientific disciplines has been neglected in philosophy, systematic reviews are important tools that minimize bias in literature search and review and allow for greater reproducibility and (...) transparency. If naturalistic philosophers wish to reduce bias in their research, they should then supplement their traditional tools for literature search and review by including systematic methodologies. (shrink)

This study examines the tension between the academic evaluation system and the development of unconventional research agendas. While scholars have studied the evaluation of research that crosses disciplinary boundaries from the perspective of peer reviewers, they have paid comparatively little attention to the experiences of the performers of unconventional science. This study asks how researchers develop unconventional research agendas to address a long-standing health problem and, in the process, make sense of the actions of a site visit committee organized to (...) advise a foundation funding the project. This study develops a process narrative on the development of a specific transdisciplinary team proposing to study premature birth. The findings show that when the performers and the evaluators of unconventional science developed competing understandings of the research agenda, transdisciplinary discovery became limited as a particular research topic became taboo. Yet, the study also reveals how the performers of unconventional science challenged the power of the site visit committee by making it seem as though they followed the committee’s decisions. These findings raise questions about the role of private foundations as funders of academic research and the suitability of traditional evaluation procedures for assessing transdisciplinary discovery. (shrink)

Scholars belong to multiple communities of credit simultaneously. When these communities disagree about a scholarly achievement’s credit assignment, this raises a puzzle for decision and game theor...

Notions of research quality are contextual in many respects: they vary between fields of research, between review contexts and between policy contexts. Yet, the role of these co-existing notions in research, and in research policy, is poorly understood. In this paper we offer a novel framework to study and understand research quality across three key dimensions. First, we distinguish between quality notions that originate in research fields and in research policy spaces. Second, drawing on existing studies, we identify three attributes (...) considered important for ‘good research’: its originality/novelty, plausibility/reliability, and value or usefulness. Third, we identify five different sites where notions of research quality emerge, are contested and institutionalised: researchers themselves, knowledge communities, research organisations, funding agencies and national policy arenas. We argue that the framework helps us understand processes and mechanisms through which ‘good research’ is recognised as well as tensions arising from the co-existence of conflicting quality notions. (shrink)

Peer reviewers at many funding agencies and scientific journals are asked to score submissions both on individual criteria and overall. The overall scores should be some kind of aggregate of the criteria scores. Carole Lee identifies this as a potential locus for bias to enter the peer review process, which she calls commensuration bias. Here I view the aggregation of scores through the lens of social choice theory. I argue that, when reviewing grant proposals, it is in many cases impossible (...) to avoid commensuration bias. (shrink)

The goal of this dissertation is to reconstruct, critically evaluate, and apply the pluralism of Paul Feyerabend. I conclude by suggesting future points of contact between Feyerabend’s pluralism and topics of interest in contemporary philosophy of science. I begin, in Chapter 1, by reconstructing Feyerabend’s critical philosophy. I show how his published works from 1948 until 1970 show a remarkably consistent argumentative strategy which becomes more refined and general as Feyerabend’s thought matures. Specifically, I argue that Feyerabend develops a persuasive (...) case against rationalism, or the thesis that there exist normative and exclusive rules of scientific rationality. In Chapter 2, I reconstruct Feyerabend’s pluralism and detail its relationship to his humanitarianism and epistemological anarchism. I understand Feyerabend’s pluralism as the combination of the principles of proliferation and tenacity. I show the evolution and justification of these principles from Feyerabend’s early papers until the late 1970s. In Chapter 3, I defend Feyerabend’s pluralism from its most prominent criticisms. I then clarify that Feyerabend’s pluralism amounts to a conception of the logic of theory pursuit and modify his view using insights from C.S. Peirce, Pierre Duhem, and Michael Polanyi. From Peirce, I show how economic, sociological, and value-laden features of theory pursuit may be used to constrain proliferation and tenacity. From Duhem and Polanyi, I try to show the proper role of tacit knowledge within a Feyerabendian framework. Finally, I show what implications Feyerabend’s pluralism has for models of distributing funds within scientific communities. I contend that it provides a more promising model that the ‘well-ordered science’ proposal advanced by numerous philosophers and social scientists. Specifically, I aim to understand what taking Feyerabend’s pluralism seriously entails for principles of balancing funding allocation decisions and the role of peer-review in evaluating the potential success of research proposals. I conclude by suggesting future lines of research for further analyzing and applying Feyerabend’s pluralism. (shrink)

Naturalistic philosophers rely on literature search and review in a number of ways and for different purposes. Yet this article shows how processes of literature search and review are likely to be affected by widespread and systematic biases. A solution to this problem is offered here. Whilst the tradition of systematic reviews of literature from scientific disciplines has been neglected in philosophy, systematic reviews are important tools that minimize bias in literature search and review and allow for greater reproducibility and (...) transparency. If naturalistic philosophers wish to reduce bias in their research, they should then supplement their traditional tools for literature search and review by including systematic methodologies. (shrink)

Project selection by funding bodies directly influences the division of cognitive labour in scientific communities. I present a novel adaptation of an existing agent-based model of scientific research, in which a central funding body selects from proposed projects located on an epistemic landscape. I simulate four different selection strategies: selection based on a god's-eye perspective of project significance, selection based on past success, selection based on past funding, and random selection. Results show the size of the landscape matters: on small (...) landscapes historical information leads to slightly better results than random selection, but on large landscapes random selection greatly outperforms historically-informed selection. (shrink)