In the biomedical, behavioural and social sciences, the leading method used to estimate causal effects is commonly randomised controlled trials (RCTs) that are generally viewed as both the source and justification of the most valid evidence. In studying the foundation and theory behind RCTs, the existing literature analyses important single issues and biases in isolation that influence causal outcomes in trials (such as randomisation, statistical probabilities and placebos). The common account of biased causal inference is described in a general way (...) in terms of probabilistic imbalances between trial groups. This paper expands the common account of causal bias by distinguishing between the range of biases arising between trial groups but also within one of the groups or across the entire sample during trial design, implementation and analysis. This is done by providing concrete examples from highly influential RCT studies. In going beyond the existing RCT literature, the paper provides a broader, practice-based account of causal bias that specifies the between-group, within-group and across-group biases that affect the estimated causal results of trials – impacting both the effect size and statistical significance. Within this expanded framework, we can better identify the range of different types of biases we face in practice and address the central question about the overall validity of the RCT method and its causal claims. A study can face several smaller biases (related simultaneously to a smaller sample, smaller estimated effect, greater unblinding etc.) that generally add up to greater aggregate bias. Though difficult to measure precisely, it is important to assess and provide information in studies on how much different sources of bias, combined, can explain the estimated causal effect. The RCT method is thereby often the best we have to inform our policy decisions – and the evidence is strengthened when combined with multiple studies and other methods. Yet there is room for continually improving trials and identifying ways to reduce biases they face and to increase their overall validity. Implications are discussed. (shrink)

Contemporary debates about scientific institutions and practice feature many proposed reforms. Most of these require increased efforts from scientists. But how do scientists’ incentives for effort interact? How can scientific institutions encourage scientists to invest effort in research? We explore these questions using a game-theoretic model of publication markets. We employ a base game between authors and reviewers, before assessing some of its tendencies by means of analysis and simulations. We compare how the effort expenditures of these groups interact in (...) our model under a variety of settings, such as double-blind and open review systems. We make a number of findings, including that open review can increase the effort of authors in a range of circumstances and that these effects can manifest in a policy-relevant period of time. However, we find that open review’s impact on authors’ efforts is sensitive to the strength of several other influences. (shrink)

Agent-based models (ABMs) are one type of simulation model used in the context of the COVID-19 pandemic. In contrast to equation-based models, ABMs are algorithms that use individual agents and attribute changing characteristics to each one, multiple times during multiple iterations over time. This paper focuses on three philosophical aspects of ABMs as models of causal mechanisms, as generators of emergent phenomena, and as providers of explanation. Based on my discussion, I conclude that while ABMs cannot help much with causal (...) inference, they can be viewed as etio-prognostic explanations of illness occurrence and outcome. (shrink)

This book develops an integrated perspective on the practices and politics of making knowledge work in inclusive development and innovation. While debates about development and innovation commonly appeal to the authority of academic researchers, many current approaches emphasize the plurality of actors with relevant expertise for addressing livelihood challenges. Adopting an action-oriented and reflexive approach, this volume explores the variety of ways in which knowledge works, paying particular attention to dilemmas and controversies. The six parts of the book address the (...) complex interplay of knowledge and politics, starting with the need for knowledge integration in the first part and decolonial perspectives on the politics of knowledge integration in the second part. The following three parts focus on the practices of inclusive development and innovation through three major themes of transformative learning, evidence, and digitization. The final part of the book addresses the governance of knowledge and innovation in the light of political struggles about inclusivity. Exploring conceptual and practical themes through case studies from the Global North and South, this book will be of great interest to students, scholars and practitioners researching and working in development studies, epistemology, innovation studies, science and technology studies and sustainability studies more broadly. (shrink)

Despite wide recognition that speculation is critical for successful science, philosophers have attended little to it. When they have, speculation has been characterized in narrowly epistemic terms: a hypothesis is speculative due to its (lack of) evidential support. These ‘evidence-first’ accounts provide little guidance for what makes speculation productive or egregious, nor how to foster the former while avoiding the latter. I examine how scientists discuss speculation and identify various functions speculations play. On this basis, I develop a ‘function-first’ account (...) of speculation. This analysis grounds a richer discussion of when speculation is egregious and when it is productive, based in both fine-grained analysis of the speculation’s purpose, and what I call the ‘epistemic situation’ scientists face. (shrink)

There are some philosophical questions that can be answered without attention to the social context in which evidence is produced and distributed.ing away from social context is an excellent way to ignore messy details and lay bare the underlying structure of the limits of inference. Idealization is entirely appropriate when one is essentially asking: In the best of all possible worlds, what am I entitled to infer? Yet, philosophers’ concerns often go beyond this domain. As an example I examine the (...) debate on mechanistic evidence and then reevaluate a canonical case study in this debate. I show that for the assessment of actual evidence, produced in a world that is far from ideal, omission of the social aspects of medical epistemology leads philosophers to draw the wrong lessons from cases they take as paradigmatic cases for their views. I close by arguing that social epistemology provides an avenue to incorporate these complications and provides the necessary framework to understand medical evidence. (shrink)

Experimental design is one aspect of a scientific method. A well-designed, properly conducted experiment aims to control variables in order to isolate and manipulate causal effects and thereby maximize internal validity, support causal inferences, and guarantee reliable results. Traditionally employed in the natural sciences, experimental design has become an important part of research in the social and behavioral sciences. Experimental methods are also endorsed as the most reliable guides to policy effectiveness. Through a discussion of some of the central concepts (...) associated with experimental design, including controlled variation and randomization, this chapter will provide a summary of key ethical issues that tend to arise in experimental contexts. In addition, by exploring assumptions about the nature of causation and by analyzing features of causal relationships, systems, and inferences in social contexts, this chapter will summarize the ways in which experimental design can undermine the integrity of not only social and behavioral research but policies implemented on the basis of such research. (shrink)

ABSTRACT“Evidence-based” methods, which most prominently include randomized controlled trials, have gained increasing purchase as the “gold standard” for assessing the effect of public policies. But the enthusiasm for evidence-based research overlooks questions about the reliability and applicability of experimental findings to diverse real-world settings. Perhaps surprisingly, a qualitative study of British educators suggests that they are aware of these limitations and therefore take evidence-based findings with a much larger grain of salt than do policy makers. Their experience suggests that the (...) real world is more heterogeneous than the world imagined by evidence-based policy enthusiasts. (shrink)

The standardised randomised clinical trial (RCT) has been exceedingly popular in medical research, economics, and practical policy making. Recently, RCTs have faced criticism. First, it has been argued by John Worrall that we cannot be certain that our sample is not atypical with regard to possible confounding factors. I will argue that at least in the case of medical research, we know enough about the relevant causal mechanisms to be justified to ignore a number of factors we have good reason (...) not to expect to be disruptive. I will also argue against an argument provided by Nancy Cartwright and Eileen Munro that RCTs should not be taken to deductively infer probabilistic causal claims, but ampliatively. The paper will end on a discussion of evidence hierarchies and a defence of the stance of evidence-based medicine that RCTs are the best available method to assess a treatment’s efficacy. (shrink)

Harms of medical interventions are systematically underestimated in clinical research. Numerous factors—conceptual, methodological, and social—contribute to this underestimation. I articulate the depth of such underestimation by describing these factors at the various stages of clinical research. Before any evidence is gathered, the ways harms are operationalized in clinical research contributes to their underestimation. Medical interventions are first tested in phase 1 ‘first in human’ trials, but evidence from these trials is rarely published, despite the fact that such trials provide the (...) foundation for assessing the harm profile of medical interventions. If a medical intervention is deemed safe in a phase 1 trial, it is tested in larger phase 2 and 3 clinical trials. One way to think about the problem of underestimating harms is in terms of the statistical ‘power’ of a clinical trial—the ability of a trial to detect a difference of a certain effect size between the experimental group and the control group. Power is normally thought to be pertinent to detecting benefits of medical interventions. It is important, though, to distinguish between the ability of a trial to detect benefits and the ability of a trial to detect harms. I refer to the former as power-B and the latter as power-H. I identify several factors that maximize power-B by sacrificing powerH in phase 3 clinical trials. If a medical intervention is approved for general use, it is evaluated by phase 4 post-market surveillance. Phase 4 surveillance of harms further contributes to underestimating the harm profile of medical interventions. At every stage of clinical research the hunt for harms is shrouded in secrecy, which further contributes to the underestimation of the harm profiles of medical interventions. (shrink)

This paper outlines the methodological and empirical limitations of analysing the potential relationship between complex social phenomena such as democracy and inequality. It shows that the means to assess how they may be related is much more limited than recognised in the existing literature that is laden with contradictory hypotheses and findings. Better understanding our scientific limitations in studying this potential relationship is important for research and policy because many leading economists and other social scientists such as Acemoglu and Robinson (...) mistakenly claim to identify causal linkages between inequality and democracy but at times still inform policy. In contrast to the existing literature, the paper argues that ‘structural’ or ‘causal’ mechanisms that may potentially link the distribution of economic wealth and different political regimes will remain unknown given reasons such as their highly complex and idiosyncratic characteristics, fundamental econometric constraints and analysis at the macro-level. Neither new data sources, different analysed time periods nor new data analysis techniques can resolve this question and provide robust, general conclusions about this potential relationship across countries. Researchers are thus restricted to exploring rough correlations over specific time periods and geographic contexts with imperfect data that are very limited for cross-country comparisons. (shrink)

This thesis examines philosophical controversies surrounding the evaluation of medical treatments, with a focus on the evidential roles of randomised trials and mechanisms in Evidence-Based Medicine. Current 'best practice' usually involves excluding non-randomised trial evidence from systematic reviews in cases where randomised trials are available for inclusion in the reviews. The first paper challenges this practice and evaluates whether adding of evidence from non-randomised trials might improve the quality and precision of some systematic reviews. The second paper compares the alleged (...) methodological benefits of randomised trials over observational studies for investigating treatment benefits. It suggests that claims about the superiority of well-conducted randomised controlled trials over well-conducted observational studies are justified, especially when results from the two methods are contradictory. The third paper argues that postulating the unpredictability paradox in systematic reviews when no detectable empirical differences can be found requires further justification. The fourth paper examines the problem of absence causation in the context of explaining causal mechanisms and argues that a recent solution is incomplete and requires further justification. Solving the problem by describing absences as causes of 'mechanism failure' fails to take into account the effects of absences that lead to vacillating levels of mechanism functionality. The fifth paper criticises literature that has emphasised functioning versus 'broken' or 'non-functioning' mechanisms emphasising that many diseases result from increased or decreased mechanism function, rather than complete loss of function. Mechanistic explanations must account for differences in the effectiveness of performed functions, yet current philosophical mechanistic explanations do not achieve this. The last paper argues that the standard of evidence embodied in the ICE theory of technological function is too permissive for evaluating whether the proposed functions of medical technologies have been adequately assessed and correctly ascribed. It argues that high-quality evidence from clinical studies is necessary to justify functional ascriptions to health care technologies. (shrink)

Consensus conferences are social techniques which involve bringing together a group of scientific experts, and sometimes also non-experts, in order to increase the public role in science and related policy, to amalgamate diverse and often contradictory evidence for a hypothesis of interest, and to achieve scientific consensus or at least the appearance of consensus among scientists. For consensus conferences that set out to amalgamate evidence, I propose three desiderata: Inclusivity, Constraint, and Evidential Complexity. Two examples suggest that consensus conferences can (...) readily satisfy Inclusivity and Evidential Complexity, but consensus conferences do not as easily satisfy Constraint. I end by discussing the relation between social inclusivity and the three desiderata. (shrink)

The issue of how to incorporate the individual's first‐hand experience of illness into broader medical understanding is a major question in medical theory and practice. In a philosophical context, phenomenology, with its emphasis on the subject's perception of phenomena as the basis for knowledge and its questioning of naturalism, seems an obvious candidate for addressing these issues. This is a review of current phenomenological approaches to medicine, looking at what has motivated this philosophical approach, the main problems it faces and (...) suggesting how it might become a useful philosophical tool within medicine, with its own individual, but interrelated, contribution to make to current medical debates. After the general background, there is a brief summary of phenomenological ideas and their current usage in a medical context. Next is a critique of four key claims within current phenomenological medical works, concerning both the role phenomenology plays and the supposedly clear divide between phenomenology and other approaches. There are significant problems within these claims, largely because they overlook the complexity of the questions they consider. Finally, there is some more in‐depth examination of phenomenology itself and the true complexity of phenomenological debate concerning subjectivity. The aim is to show that it will be both more productive and truer to phenomenology itself, if we use phenomenology as a philosophical method for explicating and gaining deeper understanding of complex and fundamental problems, which are central to medicine, rather than as providing simple, but flawed solutions. (shrink)

One way to make philosophy of science more socially relevant is to attend to specific scientific practises that affect society to a great extent. One such practise is biomedical research. This paper looks at contemporary U.S. biomedical research in particular and argues that it suffers from important epistemic, moral and socioeconomic failings. It then discusses and criticises existing approaches to improve on the status quo, most prominently by Thomas Pogge (a political philosopher), Joseph Stiglitz (a Nobel-prize winning economist) and James (...) Robert Brown (a philosopher of science). Finally, it sketches an alternative proposal and argues for its superiority. The proposal has four components: changing the intellectual property regime; instituting independent clinical research; aligning innovators' and patients' interests; and enacting additional regulation. (shrink)

I describe two traditions of philosophical accounts of evidence: one characterizes the notion in terms of signs of success, the other characterizes the notion in terms of conditions of success. The best examples of the former rely on the probability calculus, and have the virtues of generality and theoretical simplicity. The best examples of the latter describe the features of evidence which scientists appeal to in practice, which include general features of methods, such as quality and relevance, and general features (...) of evidence, such as patterns in data, concordance with other evidence, and believability of the evidence. Two infamous episodes from biomedical research help to illustrate these features. Philosophical characterization of these latter features—conditions of success—has the virtue of potential relevance to, and descriptive accuracy of, practices of experimental scientists. (shrink)

Evidence-Based Medicine (EBM) developed from the work of clinical epidemiologists at McMaster University and Oxford University in the 1970s and 1980s and self-consciously presented itself as a "new paradigm" called "evidence-based medicine" in the early 1990s. The techniques of the randomized controlled trial, systematic review and meta-analysis have produced an extensive and powerful body of research. They have also generated a critical literature that raises general concerns about its methods. This paper is a systematic review of the critical literature. It (...) finds the description of EBM as a Kuhnian paradigm helpful and worth taking further. Three kinds of criticism are evaluated in detail: criticisms of procedural aspects of EBM (especially from Cartwright, Worrall and Howick), data showing the greater than expected fallibility of EBM (Ioaanidis and others), and concerns that EBM is incomplete as a philosophy of science (Ashcroft and others). The paper recommends a more instrumental or pragmatic approach to EBM, in which any ranking of evidence is done by reference to the actual, rather than the theoretically expected, reliability of results. Emphasis on EBM has eclipsed other necessary research methods in medicine. With the recent emphasis on translational medicine, we are seeing a restoration of the recognition that clinical research requires an engagement with basic theory (e.g. physiological, genetic, biochemical) and a range of empirical techniques such as bedside observation, laboratory and animal studies. EBM works best when used in this context. (shrink)

Empirical success is a central criterion for scientific decision-making. Yet its understanding in philosophical studies of science deserves renewed attention: Should philosophers think differently about the advancement of science when they deal with the uncertainty of outcome in ongoing research in comparison with historical episodes? This paper argues that normative appeals to empirical success in the evaluation of competing scientific explanations can result in unreliable conclusions, especially when we are looking at the changeability of direction in ongoing investigations. The challenges (...) we encounter arise from the inherent dynamics of disciplinary and experimental objectives in research practice. In this paper we discuss how these dynamics inform the evaluation of empirical success by analyzing three of its requirements: data accommodation, instrumental reliability, and predictive power. We conclude that the assessment of empirical success in developing inquiry is set against the background of a model’s interactive success and prospective value in an experimental context. Our argument is exemplified by the analysis of an apparent controversy surrounding the model of a quantum nose in research on olfaction. Notably the public narrative of this controversy rests on a distorted perspective on measures of empirical success. (shrink)

If well-designed, the results of a Randomised Clinical Trial can justify a causal claim between treatment and effect in the study population; however, additional information might be needed to carry over this result to another population. RCTs have been criticized exactly on grounds of failing to provide this sort of information Evidence, inference and enquiry. Oxford University Press, New York, 2011), as well as to black-box important details regarding the mechanisms underpinning the causal law instantiated by the RCT result. On (...) the other side, so-called In Silico Clinical Trials face the same criticisms addressed against standard modelling and simulation techniques, and cannot be equated to experiments Philosophy of molecular medicine: foundational issues in research and practice, Routledge, New York, 2017; Parker in Synthese 169:483–496, 2009; Parke in Philos Sci 81:516–536, 2014; Diez Roux in Am J Epidemiol 181:100–102, 2015 and related discussions in Frigg and Reiss in Synthese 169:593–613, 2009; Winsberg in Synthese 169:575–592, 2009; Beisbart and Norton in Int Stud Philos Sci 26:403–422, 2012). We undertake a formal analysis of both methods in order to identify their distinct contribution to causal inference in the clinical setting. Britton et al.’s study :E2098–E2105, 2013) on the impact of ion current variability on cardiac electrophysiology is used for illustrative purposes. We deduce that, by predicting variability through interpolation, ISCTs aid with problems regarding extrapolation of RCTs results, and therefore in assessing their external validity. Furthermore, ISCTs can be said to encode “thick” causal knowledge —as opposed to “thin” difference-making information inferred from RCTs. Hence, ISCTs and RCTs cannot replace one another but rather, they are complementary in that the former provide information about the determinants of variability of causal effects, while the latter can, under certain conditions, establish causality in the first place. (shrink)

Philosophical discussions on causal inference in medicine are stuck in dyadic camps, each defending one kind of evidence or method rather than another as best support for causal hypotheses. Whereas Evidence Based Medicine advocates the use of Randomised Controlled Trials and systematic reviews of RCTs as gold standard, philosophers of science emphasise the importance of mechanisms and their distinctive informational contribution to causal inference and assessment. Some have suggested the adoption of a pluralistic approach to causal inference, and an inductive (...) rather than hypothetico-deductive inferential paradigm. However, these proposals deliver no clear guidelines about how such plurality of evidence sources should jointly justify hypotheses of causal associations. We here develop such guidelines by first giving a philosophical analysis of the underpinnings of Hill’s viewpoints on causality. We then put forward an evidence-amalgamation framework adopting a Bayesian net approach to model causal inference in pharmacology for the assessment of harms. Our framework accommodates a number of intuitions already expressed in the literature concerning the EBM vs. pluralist debate on causal inference, evidence hierarchies, causal holism, relevance, and reliability. (shrink)

The disconnect problem arises wherever there is ongoing and severe discordance between the scientific assessment of a politically relevant issue, and the politics and legislation of said issue. Here, we focus on the disconnect problem as it arises in the case of climate change, diagnosing a failure to respect the necessary tradeoff between authority and autonomy within a public institution like science. After assessing the problematic deployment of scientific authority in this arena, we offer suggestions for how to mitigate climate (...) change’s particular disconnect problem, as well as more general proposals for reforming science advising. (shrink)

Philosophical discussions have critically analysed the methodological pitfalls and epistemological implications of evidence assessment in medicine, however they have mainly focused on evidence of treatment efficacy. Most of this work is devoted to statistical methods of causal inference with a special attention to the privileged role assigned to randomized controlled trials (RCTs) in evidence based medicine. Regardless of whether the RCT’s privilege holds for efficacy assessment, it is nevertheless important to make a distinction between causal inference of intended and unintended (...) effects, in that the unknowns at stake are heterogonous in the two contexts. However, although “lower level” evidence is increasingly acknowledged to be a valid source of information contributory to assessing the risk profile of medications on theoretical or empirical grounds, current practices have difficulty in assigning a precise epistemic status to this kind of evidence because they are more or less implicitly parasitic on the (statistical) methods developed to test drug efficacy. My thesis is that (1) “lower level” evidence is justified on distinct grounds and at different conditions depending on the different epistemologies which one wishes to endorse, in that each impose different constraints on the methods we adopt to collect and evaluate evidence; (2) such constraints ought to be understood to be different in the case of evidence for risk versus benefit assessment for a series of reasons which I will illustrate on the basis of the recent debate on the causal association between acetaminophen (a.k.a. paracetamol) and asthma. (shrink)

This article critically analyses the concept of evidence in evidence‐based policy, arguing that there is a key problem: there is no existing practicable theory of evidence, one which is philosophically‐grounded and yet applicable for evidence‐based policy. The article critically considers both philosophical accounts of evidence and practical treatments of evidence in evidence‐based policy. It argues that both fail in different ways to provide a theory of evidence that is adequate for evidence‐based policy. The article contributes to the debate about how (...) evidence can and should be used to reduce contingency in science and in policy based on science. (shrink)

The anti-causal prophecies of last century have been disproved. Causality is neither a ‘relic of a bygone’ nor ‘another fetish of modern science’; it still occupies a large part of the current debate in philosophy and the sciences. This investigation into causal modelling presents the rationale of causality, i.e. the notion that guides causal reasoning in causal modelling. It is argued that causal models are regimented by a rationale of variation, nor of regularity neither invariance, thus breaking down the dominant (...) Human paradigm. The notion of variation is shown to be embedded in the scheme of reasoning behind various causal models: e.g. Rubin’s model, contingency tables, and multilevel analysis. It is also shown to be latent – yet fundamental – in many philosophical accounts. Moreover, it has significant consequences for methodological issues: the warranty of the causal interpretation of causal models, the levels of causation, the characterisation of mechanisms, and the interpretation of probability. This book offers a novel philosophical and methodological approach to causal reasoning in causal modelling and provides the reader with the tools to be up to date about various issues causality rises in social science. (shrink)

A coherent and helpful public policy based on science is difficult to achieve for at least three reasons. First, there are purely practical problems—for example, that scientific experts often disagree on policy-relevant questions and their debates often continue well beyond policy appropriate timelines. Second, there are epistemic problems—for example, that science is hardly the neutral supplier of factual information that traditionally has been supposed. And third, there are social problems: given the commercialization of today’s science and its enduring limitations, much (...) of scientific research today fails to meet the moral and political standards one would expect it to meet in order to inform public policy. In this paper, we examine such problems in the context of breast cancer screening policy and suggest the role philosophy of science should play in dealing with the situation. (shrink)

Evidence hierarchies are widely used to assess evidence in systematic reviews of medical studies. I give several arguments against the use of evidence hierarchies. The problems with evidence hierarchies are numerous, and include methodological shortcomings, philosophical problems, and formal constraints. I argue that medical science should not employ evidence hierarchies, including even the latest and most-sophisticated of such hierarchies.

Philosophy of medicine has traditionally examined two issues: the scientific ontology for medicine and the epistemic significance of the types of evidence used in medical research. In answering each question, philosophers have typically brought to bear tools from traditional analytic philosophy. In contrast, this volume explores medical knowledge from the perspective offered by social epistemology.While many of the same issues are addressed, the approach to these issues generates both fresh questions and new insights into old debates. In addition, the broader (...) purview offered by social epistemology opens up opportunities to address new topics such as the role of consensus conferences, epistemic injustice, the value of medical knowledge, continuing medical education, and industry funding. This article situates and summarizes the contributions to this special issue. (shrink)

Medical research makes intensive use of statistics in order to support its claims. In this paper we make explicit an epistemological tension between the conduct of clinical trials and their interpretation: statistical evidence is sometimes discarded on the basis of an underlined Bayesian reasoning. We suggest that acknowledging the potentiality of Bayesian statistics might contribute to clarify and improve comprehension of medical research. Nevertheless, despite Bayesianism may provide a better account for scientific inference with respect to the standard frequentist approach, (...) Bayesian statistics is rarely adopted in clinical research. The main reason lies in the supposed subjective elements characterizing this perspective. Hence, we discuss this objection presenting the so-called Reference analysis, a formal method which has been developed in the context of objective Bayesian statistics in order to define priors which have a minimal or null impact on posterior probabilities. Furthermore, according to this method only available data are relevant sources of information, so that it resists the most common criticisms against Bayesianism. (shrink)

In this paper we consider the problem of how to measure the strength of statistical evidence from the perspective of evidence amalgamation operations. We begin with a fundamental measurement amalgamation principle : for any measurement, the inputs and outputs of an amalgamation procedure must be on the same scale, and this scale must have a meaningful interpretation vis a vis the object of measurement. Using the p value as a candidate evidence measure, we examine various commonly used approaches to amalgamation (...) of evidence across similar studies, including standard forms of meta-analysis. We show that none of these methods satisfies MAP. Thus an underlying measurement problem remains. We argue that a successful approach to evidence amalgamation necessitates a solution to the problem of evidence measurement, and we suggest some lines of reasoning that might guide further work towards this end. (shrink)

The most conspicuous form of agricultural experiment is the field trial, and within the history of such trials, the arrival of the randomised control trial is considered revolutionary. Originating with R.A. Fisher within British agricultural science in the 1920s and 30s, the RCT has since become one of the most prodigiously used experimental techniques throughout the natural and social sciences. Philosophers of science have already scrutinised the epistemological uniqueness of RCTs, undermining their status as the ‘gold standard’ in experimental design. (...) The present paper introduces a historical case study from the origins of the RCT, uncovering the initially cool reception given to this method by agricultural scientists at the University of Cambridge and the National Institute of Agricultural Botany. Rather than giving further attention to the RCT, the paper focuses instead on a competitor method – the half-drill strip – which both predated the RCT and remained in wide use for at least a decade beyond the latter’s arrival. In telling this history, John Pickstone’s Ways of Knowing is adopted, as the most flexible, and productive way to write the history of science, particularly when sciences and scientists have to work across a number of different kinds of place. It is shown that those who resisted the RCT did so in order to preserve epistemic and social goals that randomisation would have otherwise run a tractor through. (shrink)

Why do classic biostatistical studies, alleged to provide causal explanations of effects, often fail? This article argues that in statistics-relevant areas of biology—such as epidemiology, population biology, toxicology, and vector ecology—scientists often misunderstand epistemic constraints on use of the statistical-significance rule (SSR). As a result, biologists often make faulty causal inferences. The paper (1) provides several examples of faulty causal inferences that rely on tests of statistical significance; (2) uncovers the flawed theoretical assumptions, especially those related to randomization, that likely (...) contribute to flawed biostatistics; (3) re-assesses the three classic (SSR-warrant, avoiding-selection-bias, and avoiding-confounders) arguments for using SSR only with randomization; and (4) offers five new reasons for biologists to use SSR only with randomized experiments. (shrink)

The initial successes in recent years in harnessing machine learning technologies to improve medical practice and benefit patients have attracted attention in a wide range of healthcare fields. Particularly, it should be achieved by providing automated decision recommendations to the treating clinician. Some hopes placed in such ML-based systems for healthcare, however, seem to be unwarranted, at least partially because of their inherent lack of transparency, although their results seem convincing in accuracy and reliability. Skepticism arises when the physician as (...) the agent responsible for the implementation of diagnosis, therapy, and care is unable to access the generation of findings and recommendations. There is widespread agreement that, generally, a complete traceability is preferable to opaque recommendations; however, there are differences about addressing ML-based systems whose functioning seems to remain opaque to some degree—even if so-called explicable or interpretable systems gain increasing amounts of interest. This essay approaches the epistemic foundations of ML-generated information specifically and medical knowledge generally to advocate differentiations of decision-making situations in clinical contexts regarding their necessary depth of insight into the process of information generation. Empirically accurate or reliable outcomes are sufficient for some decision situations in healthcare, whereas other clinical decisions require extensive insight into ML-generated outcomes because of their inherently normative implications. (shrink)

This paper examines the case of Ebola, ça Suffit trial which was conducted in Guinea during Ebola Virus Disease (EVD) outbreak in 2015. I demonstrate that various non-epistemic considerations may legitimately influence the criteria for evaluating the efficacy and effectiveness of a candidate vaccine. Such non-epistemic considerations, which are social, ethical, and pragmatic, can be better placed and addressed in scientific research by appealing to non-epistemic values. I consider two significant features any newly developed vaccine should possess; (1) the duration (...) of immunity the vaccine provides; and (2) safety with respect to the side effects of the vaccine. Then, I argue that social and ethical values are relevant and desirable in setting the parameters for evaluating these two features of vaccines. The parameters that are employed for setting up the criteria for assessing the features might have far-reaching implications on the well-being of society in general, and the health conditions of several thousand people in particular. The reason is that these features can play a decisive role during the evaluation of the efficacy and effectiveness of the vaccine. I conclude by showing why it is necessary to reject the concept of epistemic priority, at least when scientists engage in policy-oriented research. (shrink)

Psychotherapy research is known for its pursuit of evidence-based treatment. Psychotherapeutic efficacy is assessed by calculation of aggregated differences between pre treatment- and post treatment symptom levels. As this ‘gold standard methodology’ is regarded as ‘procedurally objective’, the efficacy number that results from the procedure is taken as a valid indicator of treatment efficacy. However, I argue that the assumption of procedural objectivity is not justified, as the methodology is build upon a problematic numerical basis. I use an empirical case (...) study to show how measurement problems practically occur in the first step of data collection, i.e. in individual symptom measurement. These problems have been discussed and acknowledged for decades, but still measurement is regarded as the best epistemic means to gain evidence on psychotherapeutic efficacy. Therefore, I show how initial measurement problems are overlooked in the remainder of the methodological procedure, which harms the ‘evidence-base’ of psychotherapeutic EBT. Via this applied analysis, I exhibit concerns that are increasingly raised in the literature in an empirical way, to emphasize the need for a non-idealized consideration of the ‘gold standard methodology’ as a means towards its clinical end. (shrink)

Several have argued that the aims of scientific research are not always independent of social and ethical values. Yet this is often assumed only to have implications for decisions about what is studied, or which research projects are funded, and not for methodological decisions or standards of evidence. Using the case of the recently developed HPV vaccines, we argue that the social aims of research can also play important roles in justifying decisions about (1) how research problems are defined in (...) drug development, (2) evidentiary standards used in testing drug “success”, and (3) clinical trial methodology. As a result, attending to the social aims at stake in particular research contexts will produce more rational methodological decisions as well as more socially relevant science. (shrink)

Reasoning in medicine requires the critical use of a clinical methodology whose validity must be evaluated as well as its limits. In the last decade, an increasing amount of evidence has shown severe limitations and flaws in the conduct of prognostic studies. The main reason behind this fact is that prognostic judgments are at high risk of error. In this paper we investigate the pragmatic and illocutionary aspects of different forms of linguistic acts and judgments involved in clinical practice. More (...) specifically, we analyze the role of uncertainty with regard to ‘particular’ clinical judgments and its relation with ‘general’ evidence. Focusing on how prognostic judgments are formulated and justified, our main purpose is to highlight the explication, the structure and the limits of prognosis from a linguistic and epistemological perspective. (shrink)

Pluralism about scientific method is more-or-less accepted, but the consequences have yet to be drawn out. Scientists adopt different methods in response to different epistemic situations: depending on the system they are interested in, the resources at their disposal, and so forth. If it is right that different methods are appropriate in different situations, then mismatches between methods and situations are possible. This is most likely to occur due to method bias: when we prefer a particular kind of method, despite (...) that method clashing with evidential context or our aims. To explore these ideas, we sketch a kind of method pluralism which turns on two properties of evidence, before using agent-based models to examine the relationship between methods, epistemic situations, and bias. Based on our results, we suggest that although method bias can undermine the efficiency of a scientific community, it can also be productive through preserving a diversity of evidence. We consider circumstances where method bias could be particularly egregious, and those where it is a potential virtue, and argue that consideration of method bias reveals that community standards deserve a central place in the epistemology of science. (shrink)

Clinical equipoise has been proposed as an ethical principle relating uncertainty and moral leeway in clinical research. Although CE has traditionally been indicated as a necessary condition for a morally justified introduction of a new RCT, questions related to the interpretation of this principle remain woefully open. Recent proposals to rehabilitate CE have divided the bioethical community on its ethical merits. This paper presents a new argument that brings out the epistemological difficulties we encounter in justifying CE as a principle (...) to connect uncertainty and moral leeway in clinical ethics. The argument proposes, first, that the methodology of hypothetical retrospection is applicable to the RCT design and that it can accommodate uncertainty. As currently understood, however, HR should give up its reliance on the assumption of uncertainty transduction, because the latter assumes the principle of indifference, which does not accommodate uncertainty in the right way. The same principle is then seen to distort also the received interpretations of CE. (shrink)

This thesis analyses the use of 'Evidence-Based' methodologies of evidence assessment and intervention and policy design from medicine, and their use in sport and exercise science. It argues that problems exist with the application of Evidence-Based methodologies in sports science, meaning that the quality of evidence used to inform decision-making is lower than is often assumed. This thesis also offers realistic solutions to these problems, broadly arguing for the importance of taking evidence from mechanistic studies seriously, in addition to evidence (...) from RCTs. (shrink)

Evidence-based policy has support in many areas of government and in public affairs more generally. In this paper we outline what evidence-based policy is, then we discuss its strengths and weaknesses. In particular, we argue that it faces a serious challenge to provide a plausible, over-arching account of evidence. We contrast evidence-based policy with evidence-based medicine, especially the role of evidence in assessing the effectiveness of medicines. The evidence required for policy decisions does not easily lend itself to randomized controlled (...) trials, nor, for that matter, being listed in a single all-purpose hierarchy. (shrink)

This paper reflects on the relationship between theory and practice in bioethics, by using various concepts drawn from debates on innovation in healthcare research—in particular debates around how best to connect up blue skies ‘basic’ research with practical innovations that can improve human lives. It argues that it is a mistake to assume that the most difficult and important questions in bioethics are the most abstract ones, and also a mistake to assume that getting clear about abstract cases will automatically (...) be of much help in getting clear about more complex cases. It replaces this implicitly linear model with a more complex one that draws on the idea of translational research in healthcare. On the translational model, there is a continuum of cases from the most simple and abstract to the most concrete and complex. Insights need to travel in both directions along this continuum—from the more abstract to the more concrete and from the more concrete to the more abstract. The paper maps out some difficulties in moving from simpler to more complex cases, and in doing so makes recommendations about the future of bioethics. (shrink)

While the history and methodology of laboratory experiments in economics have been extensively studied by philosophers, those of field experiments have not attracted much attention until recently. What is the historical context in which field experiments have been advocated? And what are the methodological rationales for conducting experiments in the field as opposed to in the lab? This article addresses these questions by combining historical and methodological perspectives. In terms of history, we show that the movement toward field experiments in (...) economics has two distinct roots. One is the general orientation of medical and social sciences to evidence-based policy evaluation, which gave rise to randomized field experiments in economics (e.g., behavioral public policy, poverty alleviation policy). The other is an awareness of several methodological limitations of lab experiments in economics, which required practitioners to get out of the lab and into the field. In these senses, the movement is a consequence of influences from both outside and inside economics: the general evidence-based trend in policy science and an internal methodological development of experimental economics. In terms of methodology, we show that these two roots resulted in two somewhat different notions of “external validity” as methodological rationales of field experiment. Finally, we suggest that analysis of experiments as exhibits highlights a methodological strategy in which both strands complement each other. (shrink)

The paper addresses charges of risk and loss aversion as well as of irrationality directed against the precautionary principle, by providing an epistemic analysis of its specific role in the safety law system. In particular, I contend that: 1) risk aversion is not a form of irrational or biased behaviour; 2) both risk and loss aversion regard the form of the utility function, whereas PP rather regards the information on which to base the decision; 3) thus PP has formally nothing (...) to do with risk or loss aversion but rather with risk awareness; 4) PP removes a fictional construct in the legal system, according to which any hazard should be ignored and denied until it is scientifically proven; 5) the quandary originates in the tension between current methods of evidence evaluation, and the logic underlying PP which demands for a probabilistic epistemology. (shrink)

The problem of collecting, analyzing and evaluating evidence on adverse drug reactions (ADRs) is an example of the more general class of epistemological problems related to scientific inference and prediction, as well as a central problem of the health-care practice. Philosophical discussions have critically analysed the methodological pitfalls and epistemological implications of evidence assessment in medicine, however they have mainly focused on evidence of treatment efficacy. Most of this work is devoted to statistical methods of causal inference with a special (...) attention to the privileged role assigned to randomized controlled trials in Evidence Based Medicine. Regardless of whether the RCT’s privilege holds for efficacy assessment, it is nevertheless important to make a distinction between causal inference of intended and unintended effects, in that the unknowns at stake are heterogonous in the two contexts. This point has been emphasized by epidemiologists in the last decade. Their main focus is methodological, and regards the fact that bias and confounding do not affect studies on intended and unintended effects in the same way. However, deeper concerns ground the intuition for such a distinction; these are related to the constraints which we impose on evidence and their epistemological justification. My thesis is that such constraints ought to be understood to be different in the case of evidence for risk vs. benefit assessment. I present the recent debate on the causal association between acetaminophen and asthma in order to illustrate the point at issue. (shrink)

Many in the international development community have embraced the randomized controlled field experiment, akin to a biomedical clinical trial for social interventions, as the new “gold evidential standard” in program impact evaluation. In response, critics have called upon the method’s advocates to consider the moral dimensions of randomization, leading to a debate about the method’s ethics. My research intervenes in this debate by empirically investigating how researchers manage the perception of randomization in the field. Without the possibility of a placebo, (...) researchers rhetorically and materially frame the experiment differently for the control and treatment groups. Three technologies allow for this differential framing: geographic separation, temporal delay, and public randomization ceremonies. Geographic separation is a “technology of opacity” designed to obscure unequal resource distribution by disentangling the intervention and research components of the experiment for the control group. The latter two are technologies of transparency designed to expose the element of randomization but downplay conditions that may affect participant buy in. All three technologies work to preclude collective definitions of fair resource allocation, yet they are not fully successful in preventing modes of confrontation and resistance that lie outside of the experiment’s framing. (shrink)

Evidence-based approaches to policy-making are growing in popularity. A generally embraced view is that with the appropriate evidence at hand, decision and policy making will be optimal, legitimate and publicly accountable. In practice, however, evidence-based policy making is constrained by a variety of problems of evidence. Some of these problems will be explored in this article, in the context of the debates on evidence from which they originate. It is argued that the source of much disagreement might be a failure (...) to addressing crucial philosophical assumptions that inform, often silently, these debates. Three controversial questions will be raised which appear central to some of the challenges faced by evidence-based policy making: firstly, how do certain types of facts candidate themselves as evidence; secondly, how do we decide what evidence we have, and how much of it; and thirdly, can we combine evidence. In addressing these questions it will be shown how a philosophically informed debate might prove instrumental in clarifying and settling practical difficulties. (shrink)