The terms ``objectivity'''' and ``objective'''' are among the mostused yet ill-defined terms in the philosophy of science and epistemology. Common to all thevarious usages is the rhetorical force of ``I endorse this and you should too'''', orto put it more mildly, that one should trust the outcome of the objectivity-producing process.The persuasive endorsement and call to trust provide some conceptual coherenceto objectivity, but the reference to objectivity is hopefully not merely an attemptat persuasive endorsement. What, in addition to epistemological endorsement,does (...) objectivity carry with it? Drawing on recent historical and philosophical work,I articulate eight operationally accessible and distinct senses of objectivity.While there are links among these senses, providing cohesion to the concept, I argue thatnone of the eight senses is strictly reducible to the others, giving objectivity itsirreducible complexity. (shrink)

This is an important book precisely because there is none other quite like it.

Psychometrically oriented researchers construe low inter-rater reliability measures for expert peer reviewers as damning for the practice of peer review. I argue that this perspective overlooks different forms of normatively appropriate disagreement among reviewers. Of special interest are Kuhnian questions about the extent to which variance in reviewer ratings can be accounted for by normatively appropriate disagreements about how to interpret and apply evaluative criteria within disciplines during times of normal science. Until these empirical-cum-philosophical analyses are done, it will remain (...) unclear the extent to which low inter-rater reliability measures represent reasonable disagreement rather than arbitrary differences between reviewers. (shrink)

An empirically sensitive formulation of the norms of transformative criticism must recognize that even public and shared standards of evaluation can be implemented in ways that unintentionally perpetuate and reproduce forms of social bias that are epistemically detrimental. Helen Longino’s theory can explain and redress such social bias by treating peer evaluations as hypotheses based on data and by requiring a kind of perspectival diversity that bears, not on the content of the community’s knowledge claims, but on the beliefs and (...) norms of the culture of the knowledge community itself. To illustrate how socializing cognition can bias evaluations, we focus on peer-review practices, with some discussion of peer-review practices in philosophy. Data include responses to surveys by editors from general philosophy journals, as well as analyses of reviews and editorial decisions for the 2007 Cognitive Science Society Conference. (shrink)

The philosophy of evidence-based medicine -- What is EBM? -- What is good evidence for a clinical decision? -- Ruling out plausible rival hypotheses and confounding factors : a method -- Resolving the paradox of effectiveness : when do observational studies offer the same degree of evidential support as randomized trials? -- Questioning double blinding as a universal methodological virtue of clinical trials : resolving the Philip's paradox -- Placebo controls : problematic and misleading baseline measures of effectiveness -- Questioning (...) the methodological superiority of "placebo" over "active" controlled trials -- Examining the paradox that traditional roles for mechanistic reasoning and expert -- Judgment have been up-ended by EBM -- A qualified defence of the EBM stance on mechanistic reasoning -- Knowledge that versus knowledge how : situating the EBM position on expert clinical judgment -- Moving EBM forward. (shrink)

Epistemological differences fuel continuous and frequently divisive debates in the social sciences and the humanities. Sociologists have yet to consider how such differences affect peer evaluation. The empirical literature has studied distributive fairness, but neglected how epistemological differences affect perception of fairness in decision making. The normative literature suggests that evaluators should overcome their epistemological differences by ‘‘translating’’ their preferred standards into general criteria of evaluation. However, little is known about how procedural fairness actually operates. Drawing on eighty-one interviews with (...) panelists serving on five multidisciplinary fellowship competitions in the social sciences and the humanities, we show that Evaluators generally draw on four epistemological styles to make arguments in favor of and against proposals. These are the constructivist, comprehensive, positivist, and utilitarian styles; and Peer reviewers define a fair decision-making process as one in which panelists engage in ‘‘cognitive contextualization,’’ that is, use epistemological styles most appropriate to the field or discipline of the proposal under review. (shrink)

For the last forty years, two claims have been at the core of disputes about scientific change: that scientists reason rationally and that science is progressive. For most of this time discussions were polarized between philosophers, who defended traditional Enlightenment ideas about rationality and progress, and sociologists, who espoused relativism and constructivism. Recently, creative new ideas going beyond the polarized positions have come from the history of science, feminist criticism of science, psychology of science, and anthropology of science. Addressing the (...) traditional arguments as well as building on these new ideas, Miriam Solomon constructs a new epistemology of science. After discussions of the nature of empirical success and its relation to truth, Solomon offers a new, social account of scientific rationality. She shows that the pursuit of empirical success and truth can be consistent with both dissent and consensus, and that the distinction between dissent and consensus is of little epistemic significance. In building this social epistemology of science, she shows that scientific communities are not merely the locus of distributed expert knowledge and a resource for criticism but also the site of distributed decision making. Throughout, she illustrates her ideas with case studies from late-nineteenth- and twentieth-century physical and life sciences. Replacing the traditional focus on methods and heuristics to be applied by individual scientists, Solomon emphasizes science funding, administration, and policy. One of her goals is to have a positive influence on scientific decision making through practical social recommendations. (shrink)

Published research findings are sometimes refuted by subsequent evidence, says Ioannidis, with ensuing confusion and disappointment.

A new, social epistemology of science that addresses practical as well as theoretical concerns.

The nature of authority is shifting in online scholarly communication. This examination of the history and future of peer review argues that effective online communication requires the development of an open, community?oriented, post?publication system of peer?to?peer review, transforming peer review from a process focused on gatekeeping to one concerned with filtering the wealth of scholarly material made available via the Internet.

In Making Science, Cole shows how social variables and cognitive variables interact in the evaluation of frontier knowledge.

Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the normative implications (...) of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to countenance the complexity of social interactions among actors involved directly and indirectly in peer review. (shrink)

Much attention has been focused in recent years on the ethical acceptability of physicians receiving gifts from drug companies. Professional guidelines recognize industry gifts as a conflict of interest and establish thresholds prohibiting the exchange of large gifts while expressly allowing for the exchange of small gifts such as pens, note pads, and coffee. Considerable evidence from the social sciences suggests that gifts of negligible value can influence the behavior of the recipient in ways the recipient does not always realize. (...) Policies and guidelines that rely on arbitrary value limits for gift-giving or receipt should be reevaluated. (shrink)