The replication (replicability, reproducibility) crisis in social psychology and clinical medicine arises from the fact that many apparently well-confirmed experimental results are subsequently overturned by studies that aim to replicate the original study. The culprit is widely held to be poor science: questionable research practices, failure to publish negative results, bad incentives, and even fraud. In this article I argue that the high rate of failed replications is consistent with high-quality science. We would expect this outcome if the field of (...) science in question produces a high proportion of false hypotheses prior to testing. If most of the hypotheses under test are false, then there will be many false hypotheses that are apparently supported by the outcomes of well conducted experiments and null hypothesis significance tests with a type-I error rate (α) of 5%. Failure to recognize this is to commit the fallacy of ignoring the base rate. I argue that this is a plausible diagnosis of the replication crisis and examine what lessons we thereby learn for the future conduct of science. (shrink)

Recent scholarship in philosophy of science and technology has shown that scientific and technological decision making are laden with values, including values of a social, political, and/or ethical character. This paper examines the role of value judgments in the design of machine-learning systems generally and in recidivism-prediction algorithms specifically. Drawing on work on inductive and epistemic risk, the paper argues that ML systems are value laden in ways similar to human decision making, because the development and design of ML systems (...) requires human decisions that involve tradeoffs that reflect values. In many cases, these decisions have significant—and, in some cases, disparate—downstream impacts on human lives. After examining an influential court decision regarding the use of proprietary recidivism-prediction algorithms in criminal sentencing, Wisconsin v. Loomis, the paper provides three recommendations for the use of ML in penal systems. (shrink)

With twentieth- and twenty-first-century philosophy of science’s unfolding acceptance of the nature of scientific inquiry being value-laden, the persistent worry has been that there are no means for legitimate negotiation of the social or non-epistemic values that enter into science. The rejection of the value-free ideal in science has thereby been coupled with the spectres of indiscriminate relativism and bias in scientific inquiry. I challenge this view in the context of recently expressed concerns regarding Canada's death of evidence controversy. The (...) worry, raised by Stathis Psillos, is that as constructivist accounts of science demoted the previously secure status of evidence for drawing justified conclusions in science, we were left with no rational delineation between the right and wrong values for science. The implication for the death of evidence controversy is that we may have no rational grounds for claiming that the Canadian Government is wrong to interfere with scientific enterprise. But he does offer another avenue for reaching the conclusion that the wrong social values are directing the current stifling of some sectors of Canadian science. Psillos draws from standpoint epistemologies to devise a salient defence of ‘valuing evidence’ as a universalizable social value. That is, government bodies ought to enable scientific research via adequate funding as well as political non-interference. In this paper, I counter that non-epistemic values can be rationally evaluated and that standpoint epistemology’s universalizable standpoint provides an inadequate framework for negotiating social values in science. Regarding, I draw from the evidence-based medicine debate in philosophy of medicine and from feminist empiricist investigations into the science–values relationship in order to make the argument for empirically driven value arbitration. If social values can be rationally chosen in the context of justification, then we can have grounds for charging the Canadian leadership with being ‘at war with science’. I further argue that my recommended empiricist methodology is preferable to Psillos’s search for universalizable perspectives for negotiating social values in science because the latter method permits little more than the trivial conclusion that evidence is valuable to science. (shrink)

This paper is meant to link the philosophical debate concerning the underdetermination of theories by evidence with a rather significant socio-political issue that has been taking place in Canada over the past few years: the so-called ‘death of evidence’ controversy. It places this debate within a broader philosophical framework by discussing the connection between evidence and theory; by bringing out the role of epistemic values in the so-called scientific method; and by examining the role of social values in science. While (...) it should be admitted that social values play an important role in science, the key question for anyone who advocates this view is: what and whose values? The way it is answered makes an important epistemic difference to how the relation between evidence and theory is appraised. I first review various arguments for the claim that evidence underdetermines theory and shows their presuppositions and limitations, using conceptual analysis and historical examples. After broaching the relation between evidence and method in science by highlighting the need to incorporate epistemic values into the scientific method, my discussion focuses on recent arguments for the role of social values in science. Finally, I address the implications of the approach outlined for the current ‘death of evidence’ debate in Canada. (shrink)

“Female sexual dysfunction” is the type of contested disease that has sparked concern about the role of the pharmaceutical industry in medical science. Many policies have been proposed to manage industry influence without carefully evaluating whether the proposed policies would be successful. We consider a proposal for incorporating citizen stakeholders into scientific research and show, via a detailed case study of the pharmaceutical regulation of flibanserin, that such programs can be co-opted. In closing, we use Holman’s asymmetric arms race framework (...) as a tool for evaluating policies in industry-funded science. (shrink)

The role of values in the scientific process is widely debated. But evidence and values cannot be neatly separated. Instead, values infuse the entire scientific process, starting with the choice of research questions. Research avenues are selected based on prior beliefs about the workings of the world. In fact, informally assigned prior probabilities and normalizing constants play an essential role in distinguishing causes from correlations and ignoring irrelevant associations that would otherwise be suggested by raw data. But since these initial (...) probabilities often have a heterogeneous pedigree – often not all that went into their formation is known or knowable – it becomes difficult to cleanly separate evidence from values. At the same time, this reality does not undermine science’s unique status in generating knowledge. This is because it is not the evidence per se generated by science, but rather the method by which science generates it, that justifies science’s privileged role. (shrink)

The ‘death of evidence’ issue in Canada raises the spectre of politicized science, and thus the question of what role social values may have in science and how this meshes with objectivity and evidence. I first criticize philosophical accounts that have to separate different steps of research to restrict the influence of social and other non-epistemic values. A prominent account that social values may play a role even in the context of theory acceptance is the argument from inductive risk. It (...) maintains that the more severe the social consequences of erroneously accepting a theory would be, the more evidence is needed before the theory may be accepted. However, an implication of this position is that increasing evidence makes the impact of social values converge to zero; and I argue for a stronger role for social values. On this position, social values may determine a theory’s conditions of adequacy, which among other things can include co.. (shrink)

Both scientists and philosophers of science have recently emphasized the importance of promoting transparency in science. For scientists, transparency is a way to promote reproducibility, progress, and trust in research. For philosophers of science, transparency can help address the value-ladenness of scientific research in a responsible way. Nevertheless, the concept of transparency is a complex one. Scientists can be transparent about many different things, for many different reasons, on behalf of many different stakeholders. This paper proposes a taxonomy that clarifies (...) the major dimensions along which approaches to transparency can vary. By doing so, it provides several insights that philosophers and other science studies scholars can pursue. In particular, it helps address common objections to pursuing transparency in science, it clarifies major forms of transparency, and it suggests avenues for further research on this topic. (shrink)

The article examines the process of niche standardization in medical research as an example of democratizing strategies implemented in industry-funded science. I argue that niche standardization can lead to undesirable epistemic and ethical consequences, if the various goals of research are not properly aligned. I examine two examples: the case of Sarafem, approved for the treatment of premenstrual dysphoric disorder in women, and the case of BiDil, approved for exclusive use in African Americans for the treatment of congestive heart failure. (...) I conclude by cautioning against the unqualified support of democratizing strategies in industry-funded research. (shrink)

The life sciences are said to be in the midst of a replication crisis because a majority of published results are irreproducible, and scientists rarely replicate existing data. Here I argue that point 2 of this assessment is flawed because there is a hitherto unidentified form of replication in the experimental life sciences, which I call ‘microreplications’. Using a case study from biochemistry, I illustrate how MRs depend on a key element of experimentation, namely, experimental controls. I end by reflecting (...) on what MRs mean for the broader debate about the replication crisis. (shrink)

Exploratory studies employing volunteer subjects are especially vulnerable to race and class bias. This article illustrates how inattention to race and class as critical dimensions in women's lives can produce biased research samples and lead to false conclusions. It analyzes the race and class background of 200 women who volunteered to participate in an in-depth study of Black and White professional, managerial, and administrative women. Despite a multiplicity of methods used to solicit subjects, White women raised in middle-class families who (...) worked in male-dominated occupations were the most likely to volunteer, and White women were more than twice as likely to respond to media solicitations or letters. To recruit most Black subjects and address their concerns about participation required more labor-intensive strategies involving personal contact. The article discusses reasons for differential volunteering and ways to integrate race and class into qualitative research on women. (shrink)