In this paper we argue for a naturalistic solution to some of the methodological controversies in regulatory science, on the basis of two case studies: toxicology and health claim regulation. We analyze the debates related to the scientific evidence that is considered necessary for regulatory decision making in each of those two fields, with a particular attention to the interactions between scientific and regulatory aspects. This analysis allows us to identify two general stances in the debate: a) one that argues (...) for more permissive standards of evidence and for methodological pluralism, and b) an opposing one that not only defends strict evidence requirements but also stipulates the use of one particular scientific methodologies for data generation. We argue that the real-world outcomes produced by alternative regulatory options are a vital piece of information that allows for the empirical assessment of these two stances. In particular, this information on outcomes makes it possible to analyze which standards of evidence and scientific methods generate the most useful knowledge as input for regulatory decision making. Our conclusion is that instead of an a priori selection of methodologies and standards, such decisions ought to be based on empirical evidence related to real-world outcomes. (shrink)

Philosophers of science and medicine now aspire to provide useful, socially relevant accounts of mechanism. Existing accounts have forged the path by attending to mechanisms in historical context, scientific practice, the special sciences, and policy. Yet, their primary focus has been on more proximate issues related to therapeutic effectiveness. To take the next step toward social relevance, we must investigate the challenges facing researchers, clinicians, and policy makers involving values and social context. Accordingly, we learn valuable lessons about the connections (...) between mechanistic processes and more fundamental reasons for medical interventions, particularly moral, ethical, religious, and political concerns about health, agency, and power. This paper uses debates over the controversial morning-after pill to gain insight into the deeper reasons for the production and use of mechanistic knowledge throughout biomedical research, clinical practice, and governmental regulation. To practice socially relevant philosophy of science, I argue that we need to account for mechanistic knowledge beyond immediate effectiveness, such as how it can also provide moral guidance, aid ethical categorization in the clinic, and function as a political instrument. Such insights have implications for medical epistemology, including the value-laden dimensions of mechanistic reasoning and the “epistemic friction” of values. Furthermore, there are broader impacts for teaching research ethics and understanding the role of science advisors as political advocates. (shrink)

Experimental design is one aspect of a scientific method. A well-designed, properly conducted experiment aims to control variables in order to isolate and manipulate causal effects and thereby maximize internal validity, support causal inferences, and guarantee reliable results. Traditionally employed in the natural sciences, experimental design has become an important part of research in the social and behavioral sciences. Experimental methods are also endorsed as the most reliable guides to policy effectiveness. Through a discussion of some of the central concepts (...) associated with experimental design, including controlled variation and randomization, this chapter will provide a summary of key ethical issues that tend to arise in experimental contexts. In addition, by exploring assumptions about the nature of causation and by analyzing features of causal relationships, systems, and inferences in social contexts, this chapter will summarize the ways in which experimental design can undermine the integrity of not only social and behavioral research but policies implemented on the basis of such research. (shrink)

It has recently been argued that successful evidence-based policy should rely on two kinds of evidence: statistical and mechanistic. The former is held to be evidence that a policy brings about the desired outcome, and the latter concerns how it does so. Although agreeing with the spirit of this proposal, we argue that the underlying conception of mechanistic evidence as evidence that is different in kind from correlational, difference-making or statistical evidence, does not correctly capture the role that information about (...) mechanisms should play in evidence-based policy. We offer an alternative account of mechanistic evidence as information concerning the causal pathway connecting the policy intervention to its outcome. Not only can this be analyzed as evidence of difference-making, it is also to be found at any level and is obtainable by a broad range of methods, both experimental and observational. Using behavioral policy as an illustration, we draw the implications of this revised understanding of mechanistic evidence for debates concerning policy extrapolation, evidence hierarchies, and evidence integration. (shrink)

Natural experiments are an increasingly popular research design in political science. This popularity raises a number of questions. First, what are natural experiments and why are they appealing? Second, what makes a good natural experiment? And finally, are natural experiments able to provide resources for knowledge production that other methodologies cannot or do not provide? Using Mary Morgan’s and Thad Dunning’s recent work on natural experiments, I offer answers to the first two questions and use the analysis to argue that (...) natural experiments highlight features of knowledge production that support methodological pluralism and the multiple aims of research. (shrink)

Science-based methods for assessing the practical rationality of a proposed public policy typically represent assumed future outcomes of policies and values attributed to these outcomes in an idealized, that is, intentionally distorted way and abstracted from aspects that are deemed irrelevant. Different types of methods do so in different ways. As a consequence, they instantiate the properties that result from abstraction and idealization such as conceptual simplicity versus complexity, or comprehensiveness versus selectivity of the values under consideration to different degrees. (...) I hold that none of these methods is best in general. Instead, I opt for the valuation method that is useful for the policy issue in question both in terms of its relevance and in terms of its practicability. Relevance requires that the method can represent and account for what is at stake in the policy issue. Practicability refers to aspects such as easy versus difficult handling of the method. To argue for the claim, I evaluate three types of valuation methods: cost–benefit analysis that rests on unidimensional measurement and ranking, multi-criteria decision analysis that applies multi-dimensional measurement but unidimensional ranking, and non-aggregate indicator systems that operate with multi-dimensional measurement and sometimes also multi-dimensional ranking. Second-order justification indicating whether and how the valuation method chosen is capable of accounting for the substantive value considerations that constitute the real-world policy issue in question renders the conditions on which the results of a proposed policy evaluation rest transparent. (shrink)

Farmers all over the world perform experiments, and have done so since long before modern experimental science and its recognized forerunners. There is a rich anthropological literature on these experiments, but the philosophical issues that they give rise to have not received much attention. Based on the anthropological literature, this study investigates methodological and philosophical issues pertaining to farmers’ experiments, including the choice of interventions to be tested, the planning of experiments, and the use of control fields and other means (...) to deal with confounding factors. Farmers’ experiments have some advantages over the field trials of agricultural scientists, but also some comparative disadvantages. The two experimental traditions are complementary, and neither of them can replace the other. Several aspects of farmers’ experiments are shown to have a direct bearing on central topics in the philosophy of science. (shrink)

In this paper we consider the problem of how to measure the strength of statistical evidence from the perspective of evidence amalgamation operations. We begin with a fundamental measurement amalgamation principle : for any measurement, the inputs and outputs of an amalgamation procedure must be on the same scale, and this scale must have a meaningful interpretation vis a vis the object of measurement. Using the p value as a candidate evidence measure, we examine various commonly used approaches to amalgamation (...) of evidence across similar studies, including standard forms of meta-analysis. We show that none of these methods satisfies MAP. Thus an underlying measurement problem remains. We argue that a successful approach to evidence amalgamation necessitates a solution to the problem of evidence measurement, and we suggest some lines of reasoning that might guide further work towards this end. (shrink)

The aim of this paper is to reconstruct and critically assess the evidential relationship between neuroscience and educational practice. To do this, I reconstruct a standard way in which evidence from neuroscience is used to support recommendations about educational practice, that is, testing pedagogical interventions using neuroimaging methods, and discuss and critically assess the inference behind this approach. I argue that this inference rests on problematic assumptions, and, therefore, that neuroimaging intervention studies have no special evidential status for basing educational (...) practice. I conclude arguing that these limitations could be resolved by integrating evidence from neurocognitive and educational science. (shrink)

There are some philosophical questions that can be answered without attention to the social context in which evidence is produced and distributed.ing away from social context is an excellent way to ignore messy details and lay bare the underlying structure of the limits of inference. Idealization is entirely appropriate when one is essentially asking: In the best of all possible worlds, what am I entitled to infer? Yet, philosophers’ concerns often go beyond this domain. As an example I examine the (...) debate on mechanistic evidence and then reevaluate a canonical case study in this debate. I show that for the assessment of actual evidence, produced in a world that is far from ideal, omission of the social aspects of medical epistemology leads philosophers to draw the wrong lessons from cases they take as paradigmatic cases for their views. I close by arguing that social epistemology provides an avenue to incorporate these complications and provides the necessary framework to understand medical evidence. (shrink)

Philosophical discussions on causal inference in medicine are stuck in dyadic camps, each defending one kind of evidence or method rather than another as best support for causal hypotheses. Whereas Evidence Based Medicine advocates the use of Randomised Controlled Trials and systematic reviews of RCTs as gold standard, philosophers of science emphasise the importance of mechanisms and their distinctive informational contribution to causal inference and assessment. Some have suggested the adoption of a pluralistic approach to causal inference, and an inductive (...) rather than hypothetico-deductive inferential paradigm. However, these proposals deliver no clear guidelines about how such plurality of evidence sources should jointly justify hypotheses of causal associations. We here develop such guidelines by first giving a philosophical analysis of the underpinnings of Hill’s viewpoints on causality. We then put forward an evidence-amalgamation framework adopting a Bayesian net approach to model causal inference in pharmacology for the assessment of harms. Our framework accommodates a number of intuitions already expressed in the literature concerning the EBM vs. pluralist debate on causal inference, evidence hierarchies, causal holism, relevance, and reliability. (shrink)