An astonishing volume and diversity of evidence is available for many hypotheses in the biomedical and social sciences. Some of this evidence—usually from randomized controlled trials (RCTs)—is amalgamated by meta-analysis. Despite the ongoing debate regarding whether or not RCTs are the ‘gold-standard’ of evidence, it is usually meta-analysis which is considered the best source of evidence: meta-analysis is thought by many to be the platinum standard of evidence. However, I argue that meta-analysis falls far short of that standard. Different meta-analyses (...) of the same evidence can reach contradictory conclusions. Meta-analysis fails to provide objective grounds for intersubjective assessments of hypotheses because numerous decisions must be made when performing a meta-analysis which allow wide latitude for subjective idiosyncrasies to influence its outcome. I end by suggesting that an older tradition of evidence in medicine—the plurality of reasoning strategies appealed to by the epidemiologist Sir Bradford Hill—is a superior strategy for assessing a large volume and diversity of evidence. (shrink)

Measuring the effectiveness of medical interventions faces three epistemological challenges: the choice of good measuring instruments, the use of appropriate analytic measures, and the use of a reliable method of extrapolating measures from an experimental context to a more general context. In practice each of these challenges contributes to overestimating the effectiveness of medical interventions. These challenges suggest the need for corrective normative principles. The instruments employed in clinical research should measure patient-relevant and disease-specific parameters, and should not be sensitive (...) to parameters that are only indirectly relevant. Effectiveness always should be measured and reported in absolute terms (using measures such as 'absolute risk reduction'), and only sometimes should effectiveness also be measured and reported in relative terms (using measures such as 'relative risk reduction')-employment of relative measures promotes an informal fallacy akin to the base-rate fallacy, which can be exploited to exaggerate claims of effectiveness. Finally, extrapolating from research settings to clinical settings should more rigorously take into account possible ways in which the intervention in question can fail to be effective in a target population. (shrink)

Do we know what we're doing, and why? Psychological research seems to suggest not: reflection and self-awareness are surprisingly uncommon and inaccurate. John M. Doris presents a new account of agency and responsibility, which reconciles our understanding of ourselves as moral agents with empirical work on the unconscious mind.

Under the traditional system of peer-reviewed publication, the degree of prestige conferred to authors by successful publication is tied to the degree of the intellectual rigor of its peer review process: ambitious scientists do well professionally by doing well epistemically. As a result, we should expect journal editors, in their dual role as epistemic evaluators and prestige-allocators, to have the power to motivate improved author behavior through the tightening of publication requirements. Contrary to this expectation, I will argue that the (...) publication bias literature in academic medicine demonstrates that editor interventions have had limited effectiveness in improving the health of the publication and trial registration record, suggesting that much stronger interventions are needed. (shrink)

This is an important book precisely because there is none other quite like it.

In the latter half of the twentieth century, philosophers of science have argued (implicitly and explicitly) that epistemically rational individuals might compose epistemically irrational groups and that, conversely, epistemically rational groups might be composed of epistemically irrational individuals. We call the conjunction of these two claims the Independence Thesis, as they together imply that methodological prescriptions for scientific communities and those for individual scientists might be logically independent of one another. We develop a formal model of scientific inquiry, define four (...) criteria for individual and group epistemic rationality, and then prove that the four definitions diverge, in the sense that individuals will be judged rational when groups are not and vice versa. We conclude by explaining implications of the inconsistency thesis for (i) descriptive history and sociology of science and (ii) normative prescriptions for scientific communities. (shrink)

Feminist epistemology and philosophy of science studies the ways in which gender does and ought to influence our conceptions of knowledge, the knowing subject, and practices of inquiry and justification. It identifies ways in which dominant conceptions and practices of knowledge attribution, acquisition, and justification systematically disadvantage women and other subordinated groups, and strives to reform these conceptions and practices so that they serve the interests of these groups. Various practitioners of feminist epistemology and philosophy of science argue that dominant (...) knowledge practices disadvantage women by (1) excluding them from inquiry, (2) denying them epistemic authority, (3) denigrating their “feminine” cognitive styles and modes of knowledge, (4) producing theories of women that represent them as inferior, deviant, or significant only in the ways they serve male interests, (5) producing theories of social phenomena that render women's activities and interests, or gendered power relations, invisible, and (6) producing knowledge (science and technology) that is not useful for people in subordinate positions, or that reinforces gender and other social hierarchies. Feminist epistemologists trace these failures to flawed conceptions of knowledge, knowers, objectivity, and scientific methodology. They offer diverse accounts of how to overcome these failures. They also aim to (1) explain why the entry of women and feminist scholars into different academic disciplines, especially in biology and the social sciences, has generated new questions, theories, and methods, (2) show how gender has played a.. (shrink)

In a randomized clinical trial (RCT), a group of patients, initially assembled through a mixture of deliberation (involving explicit inclusion and exclusion criteria) and serendipity (which patients happen to walk into which doctor’s clinic while the trial is in progress), are divided by some random process into an experimental group (members of which will receive the therapy under test) and a control group (members of which will receive some other treatment – perhaps placebo, perhaps the currently standard treatment for the (...) condition at issue). In a ‘double blind’ trial neither the patient nor the clinician knows to which of the groups a particular patient belongs. The results of double blind randomized controlled trials are almost universally regarded as providing the ‘gold standard’ for evidence in medicine. Fairly extreme claims to this effect can be found in the literature. For example the statistician Tukey wrote (1977, p. 679) “almost the only source of reliable evidence [in medicine] … is that obtained from … carefully conducted randomised trials”. And the clinician Victor Herbert claimed (1977, p. 690) “..the only source of reliable evidence rising to the level of proof about the usefulness of any new therapy is that obtained from wellplanned and carefully conducted randomized, and, where possible, coded (double blind) clinical trials. [Other] studies may point in a direction, but cannot be evidence as lawyers use the term evidence to mean something probative … [that is] tending to prove or actually proving”. Finally, the still very influential movement in favour of ‘Evidence Based Medicine’ (EBM) that began at McMaster University in the 1980s was initially often regarded as endorsing the claim that only RCTs provide real scientifically telling evidence. (shrink)

During the last three decades, reflections on the growth of scientific knowledge have inspired historians, sociologists, and some philosophers to contend that scientific objectivity is a myth. In this book, Kitcher attempts to resurrect the notions of objectivity and progress in science by identifying both the limitations of idealized treatments of growth of knowledge and the overreactions to philosophical idealizations. Recognizing that science is done not by logically omniscient subjects working in isolation, but by people with a variety of personal (...) and social interests, who cooperate and compete with one another, he argues that, nonetheless, we may conceive the growth of science as a process in which both our vision of nature and our ways of learning more about nature improve. Offering a detailed picture of the advancement of science, he sets a new agenda for the philosophy of science and for other "science studies" disciplines. (shrink)

Because physical theories typically predict numerical values, an improvement in experimental precision reduces the tolerance range and hence increases corroborability. In most psychological research, improved power of a statistical design leads to a prior probability approaching 1/2 of finding a significant difference in the theoretically predicted direction. Hence the corroboration yielded by "success" is very weak, and becomes weaker with increased precision. "Statistical significance" plays a logical role in psychology precisely the reverse of its role in physics. This problem is (...) worsened by certain unhealthy tendencies prevalent among psychologists, such as a premium placed on experimental "cuteness" and a free reliance upon ad hoc explanations to avoid refutation. (shrink)

For the last forty years, two claims have been at the core of disputes about scientific change: that scientists reason rationally and that science is progressive. For most of this time discussions were polarized between philosophers, who defended traditional Enlightenment ideas about rationality and progress, and sociologists, who espoused relativism and constructivism. Recently, creative new ideas going beyond the polarized positions have come from the history of science, feminist criticism of science, psychology of science, and anthropology of science. Addressing the (...) traditional arguments as well as building on these new ideas, Miriam Solomon constructs a new epistemology of science. After discussions of the nature of empirical success and its relation to truth, Solomon offers a new, social account of scientific rationality. She shows that the pursuit of empirical success and truth can be consistent with both dissent and consensus, and that the distinction between dissent and consensus is of little epistemic significance. In building this social epistemology of science, she shows that scientific communities are not merely the locus of distributed expert knowledge and a resource for criticism but also the site of distributed decision making. Throughout, she illustrates her ideas with case studies from late-nineteenth- and twentieth-century physical and life sciences. Replacing the traditional focus on methods and heuristics to be applied by individual scientists, Solomon emphasizes science funding, administration, and policy. One of her goals is to have a positive influence on scientific decision making through practical social recommendations. (shrink)

Douglas proposes a new ideal in which values serve an essential function throughout scientific inquiry, but where the role values play is constrained at key points, protecting the integrity and objectivity of science.

Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the normative implications (...) of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to countenance the complexity of social interactions among actors involved directly and indirectly in peer review. (shrink)

We must restrict to mere probability not only statements of comparatively great uncertainty, like predictions about the weather, where we would cautiously ...

The use of power to infer null hypotheses from negative results has recently come under severe attack. In this article, I show that the power of a test can justify accepting the null hypothesis. This argument also gives us a new powerful reason for not treating p-values and power as measures of the strength of evidence.

Science's priority rule rewards those who are first to make a discovery, at the expense of all other scientists working towards the same goal, no matter how close they may be to making the same discovery. I propose an explanation of the priority rule that, better than previous explanations, accounts for the distinctive features of the rule. My explanation treats the priority system, and more generally, any scheme of rewards for scientific endeavor, as a device for achieving an allocation of (...) resources among different research programs that provides as much benefit as possible to society. I show that the priority system is especially well suited to finding an efficient allocation of resources in those situations, characteristic of scientific inquiry, in which any success in an endeavor subsequent to the first success brings little additional benefit to society. (shrink)

A new, social epistemology of science that addresses practical as well as theoretical concerns.

Scientific realists have suggested that changes in our scientific communities over the course of their history have rendered those communities progressively less vulnerable to the problem of unconcieved alternatives over time. I argue in response not only that the most fundamental historical transformations of the scientific enterprise have generated steadily mounting obstacles to revolutionary, transformative, or unorthodox scientific theorizing, but also that we have substantial independent evidence that the institutional apparatus of contemporary scientific inquiry fosters an exceedingly and increasingly theoretically (...) conservative form of that inquiry. I conclude that contemporary scientific communities are actually more vulnerable to the problem of unconceived alternatives than their historical predecessors, and I briefly suggest how we might seek to pursue scientific inquiry in a less theoretically conservative way. (shrink)

Published research findings are sometimes refuted by subsequent evidence, says Ioannidis, with ensuing confusion and disappointment.

The work of Tversky, Kahneman and others suggests that people often make use of cognitive heuristics such as availability, salience and representativeness in their reasoning and decision making. Through use of a historical example--the recent plate tectonics revolution in geology--I argue that such heuristics play a crucial role in scientific decision making also. I suggest how these heuristics are to be considered, along with noncognitive factors (such as motivation and social structures) when drawing historical and epistemological conclusions. The normative perspective (...) is community-wide, contextual, and instrumental. (shrink)