A new, social epistemology of science that addresses practical as well as theoretical concerns.

"--Richard Grandy, Rice University "This is the first compelling diagnosis of what has gone awry in the raging 'science wars.

This is an important book precisely because there is none other quite like it.

An empirically sensitive formulation of the norms of transformative criticism must recognize that even public and shared standards of evaluation can be implemented in ways that unintentionally perpetuate and reproduce forms of social bias that are epistemically detrimental. Helen Longino’s theory can explain and redress such social bias by treating peer evaluations as hypotheses based on data and by requiring a kind of perspectival diversity that bears, not on the content of the community’s knowledge claims, but on the beliefs and (...) norms of the culture of the knowledge community itself. To illustrate how socializing cognition can bias evaluations, we focus on peer-review practices, with some discussion of peer-review practices in philosophy. Data include responses to surveys by editors from general philosophy journals, as well as analyses of reviews and editorial decisions for the 2007 Cognitive Science Society Conference. (shrink)

Psychometrically oriented researchers construe low inter-rater reliability measures for expert peer reviewers as damning for the practice of peer review. I argue that this perspective overlooks different forms of normatively appropriate disagreement among reviewers. Of special interest are Kuhnian questions about the extent to which variance in reviewer ratings can be accounted for by normatively appropriate disagreements about how to interpret and apply evaluative criteria within disciplines during times of normal science. Until these empirical-cum-philosophical analyses are done, it will remain (...) unclear the extent to which low inter-rater reliability measures represent reasonable disagreement rather than arbitrary differences between reviewers. (shrink)

Much attention has been focused in recent years on the ethical acceptability of physicians receiving gifts from drug companies. Professional guidelines recognize industry gifts as a conflict of interest and establish thresholds prohibiting the exchange of large gifts while expressly allowing for the exchange of small gifts such as pens, note pads, and coffee. Considerable evidence from the social sciences suggests that gifts of negligible value can influence the behavior of the recipient in ways the recipient does not always realize. (...) Policies and guidelines that rely on arbitrary value limits for gift-giving or receipt should be reevaluated. (shrink)

Published research findings are sometimes refuted by subsequent evidence, says Ioannidis, with ensuing confusion and disappointment.

The terms ``objectivity'''' and ``objective'''' are among the mostused yet ill-defined terms in the philosophy of science and epistemology. Common to all thevarious usages is the rhetorical force of ``I endorse this and you should too'''', orto put it more mildly, that one should trust the outcome of the objectivity-producing process.The persuasive endorsement and call to trust provide some conceptual coherenceto objectivity, but the reference to objectivity is hopefully not merely an attemptat persuasive endorsement. What, in addition to epistemological endorsement,does (...) objectivity carry with it? Drawing on recent historical and philosophical work,I articulate eight operationally accessible and distinct senses of objectivity.While there are links among these senses, providing cohesion to the concept, I argue thatnone of the eight senses is strictly reducible to the others, giving objectivity itsirreducible complexity. (shrink)

Epistemological differences fuel continuous and frequently divisive debates in the social sciences and the humanities. Sociologists have yet to consider how such differences affect peer evaluation. The empirical literature has studied distributive fairness, but neglected how epistemological differences affect perception of fairness in decision making. The normative literature suggests that evaluators should overcome their epistemological differences by ‘‘translating’’ their preferred standards into general criteria of evaluation. However, little is known about how procedural fairness actually operates. Drawing on eighty-one interviews with (...) panelists serving on five multidisciplinary fellowship competitions in the social sciences and the humanities, we show that Evaluators generally draw on four epistemological styles to make arguments in favor of and against proposals. These are the constructivist, comprehensive, positivist, and utilitarian styles; and Peer reviewers define a fair decision-making process as one in which panelists engage in ‘‘cognitive contextualization,’’ that is, use epistemological styles most appropriate to the field or discipline of the proposal under review. (shrink)

The nature of authority is shifting in online scholarly communication. This examination of the history and future of peer review argues that effective online communication requires the development of an open, community?oriented, post?publication system of peer?to?peer review, transforming peer review from a process focused on gatekeeping to one concerned with filtering the wealth of scholarly material made available via the Internet.

The low representation (< 30%) of women in philosophy in English-speaking countries has generated much discussion, both in academic circles and the public sphere. It is sometimes suggested (Haslanger 2009) that unconscious biases, acting at every level in the field, may be grounded in gendered schemas of philosophers and in the discipline more widely, and that actions to make philosophy a more welcoming place for women should address such schemas. However, existing data are too limited to fully warrant such an (...) explanation, which therefore will not satisfy those in favor of the status quo or those who argue against the need to address gender imbalance. In this paper, we propose measures to improve the profession that ought to be implemented without referring explicitly to this underrepresentation or to the climate for women and other underrepresented groups. Such recommendations are based on empirical research already carried out in other disciplines and do not rest on whether it is possible to identify the cause of this low representation. We argue that we need not wait for new or better data to ensure that fairer practices are enacted for women, other underrepresented groups, and everybody else, if only out of precaution. (shrink)

In Making Science, Cole shows how social variables and cognitive variables interact in the evaluation of frontier knowledge.

The philosophy of evidence-based medicine -- What is EBM? -- What is good evidence for a clinical decision? -- Ruling out plausible rival hypotheses and confounding factors : a method -- Resolving the paradox of effectiveness : when do observational studies offer the same degree of evidential support as randomized trials? -- Questioning double blinding as a universal methodological virtue of clinical trials : resolving the Philip's paradox -- Placebo controls : problematic and misleading baseline measures of effectiveness -- Questioning (...) the methodological superiority of "placebo" over "active" controlled trials -- Examining the paradox that traditional roles for mechanistic reasoning and expert -- Judgment have been up-ended by EBM -- A qualified defence of the EBM stance on mechanistic reasoning -- Knowledge that versus knowledge how : situating the EBM position on expert clinical judgment -- Moving EBM forward. (shrink)

Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the normative implications (...) of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to countenance the complexity of social interactions among actors involved directly and indirectly in peer review. (shrink)