Scientists and philosophers generally agree that the replication of experiments is a key ingredient of good and successful scientific practice. “One-offs“ are not significant; experiments must be replicable to be considered valid and important. But the term “replication“ has been used in a number of ways, and it is therefore quite difficult to appraise the meaning and significance of replications. I consider how history may help - and has helped - with this task. I propose that: 1) Studies of past (...) scientific episodes in historical context and of recent philosophical contributions to the discussion are heuristic tools for exploring and clarifying the meaning of that concept. 2) The analysis of the development of the methodological imperative of replication sheds light on the significance scientists have attached to it, thereby contributing further to the clarification of the concept. 3) The analysis of the history of philosophical thought about methods and scientific methodology helps understand why philosophers have not paid much attention to the analysis of the concept of replication. (shrink)

This article seeks to provide a historically well-informed analysis of an important post-Newtonian area of research in experimental physics between 1798 and 1898, namely the determination of the mean density of the earth and, by the end of the nineteenth century, the gravitational constant. Traditionally, research on these matters is seen as a case of “puzzle solving.” In this article, the author shows that such focus does not do justice to the evidential significance of eighteenth- and nineteenth-century experimental research on (...) the mean density of the earth and the gravitational constant. As Newton’s theory of universal gravitation was mainly based on astronomical observation, it remained to be shown that Newton’s law of universal gravitation did not break down at terrestrial distances. In this context, Cavendish’ experiment and related nineteenth-century experiments played a decisive role, for they provided converging and increasingly stronger evidence for the universality of Newton’s theory of gravitation. More precisely, the author shall argue that, as the accuracy and precision of the experimental apparatuses and the procedures to eliminate external disturbances involved increasingly improved, the empirical support for the universality of Newton’s theory of gravitation improved correspondingly. (shrink)

Robustness is a common platitude: hypotheses are better supported with evidence generated by multiple techniques that rely on different background assumptions. Robustness has been put to numerous epistemic tasks, including the demarcation of artifacts from real entities, countering the “experimenter’s regress,” and resolving evidential discordance. Despite the frequency of appeals to robustness, the notion itself has received scant critique. Arguments based on robustness can give incorrect conclusions. More worrying is that although robustness may be valuable in ideal evidential circumstances (i.e., (...) when evidence is concordant), often when a variety of evidence is available from multiple techniques, the evidence is discordant. †To contact the author, please write to: Jacob Stegenga, Department of Philosophy, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093; e‐mail: [email protected]. (shrink)

Many philosophers have claimed that evidence for a theory is better when multiple independent tests yield the same result, i.e., when experimental results are robust. Little has been said about the grounds on which such a claim rests, however. The present essay presents an analysis of the evidential value of robustness that rests on the fallibility of assumptions about the reliability of testing procedures and a distinction between the strength of evidence and the security of an evidence claim. Robustness can (...) enhance the security of an evidence claim either by providing what I call second-order evidence, or by providing back-up evidence for a hypothesis. (shrink)

Jonathan Y. Tsou examines and defends positions on central issues in philosophy of psychiatry. The positions defended assume a naturalistic and realist perspective and are framed against skeptical perspectives on biological psychiatry. Issues addressed include the reality of mental disorders; mechanistic and disease explanations of abnormal behavior; definitions of mental disorder; natural and artificial kinds in psychiatry; biological essentialism and the projectability of psychiatric categories; looping effects and the stability of mental disorders; psychiatric classification; and the validity of the DSM's (...) diagnostic categories. The main argument defended by Tsou is that genuine mental disorders are biological kinds with harmful effects. This argument opposes the dogma that mental disorders are necessarily diseases that result from biological dysfunction. Tsou contends that the broader ideal of biological kinds offers a more promising and empirically ascertainable naturalistic standard for assessing the reality of mental disorders and the validity of psychiatric categories. (shrink)

There is an ongoing and robust tradition of science and technology studies scholars conducting ethnographic laboratory studies. These laboratory studies—like all ethnographies—are each conducted at a particular time, are situated in a particular place, and are about a particular culture. Presumably, this contextual specificity means that such ethnographies have limited applicability beyond the narrow slice of time, place, and culture that they each subject to examination. But we do not always or even often treat them that way. It is beyond (...) common for us to speak about what one or another laboratory study reveals about the laboratory, or “science” much more broadly. Given the contextual specificity of our ethnographic laboratory studies, what justifies this presumed generalizability? Initially, this manuscript surveys typical responses to this question, but then it pursues an unusual one: the potential replicability of ethnographic results. This potential is hereby explored, via an ethnographic replication attempt—one designed and conducted in order to test the generalizability of a particular laboratory study, that of Latour and Woolgar’s classic Laboratory Life. The results of the ethnographic replication attempt are reported, and a remarkable degree of replicability is established. (shrink)

The intuitive Variety of Evidence Thesis states that, ceteris paribus, more varied evidence for a hypothesis confirms it more strongly than less varied evidence. Recent Bayesian analyses have raised serious doubts in its validity. Claveau suggests the existence of a novel type of counter-example to this thesis: a gradual increase in source independence can lead to a decrease in hypothesis confirmation. I show that Claveau’s measure of gradual source independence suffers from two unsuspected types of inconsistencies. I hence put forward (...) a more natural measure of gradual source independence which is not plagued by inconsistencies. Claveau’s counter-examples to the variety of evidence thesis disappear with the measure I suggest. I hence argue that my measure is preferable and that this thesis does at least not seem to be troubled by Claveau’s arguments. (shrink)

In psychiatry, pharmacological research has played a crucial role in the formulation, revision, and refinement of neurobiological theories of psychopathology. Besides being utilized as potential treatments for various mental disorders, pharmacological drugs play an important epistemic role as experimental instruments that help scientists uncover the neurobiological underpinnings of mental disorders (Tsou, 2012). Interventions with psychiatric patients using pharmacological drugs provide researchers with information about the neurobiological causes of mental disorders that cannot be obtained in other ways. This important source of (...) evidence for the biological causes of mental disorder is often overlooked in philosophical analyses of psychiatry, especially in skeptical analyses that debase the biological aspects of psychopathology (e.g., Szasz, 1960; Scheff, 1963; Laing, 1967). In discussing pharmacological interventions as a form of evidence for the physical basis of mental disorders, this paper aims to clarify the nature, reliability, and limitations of this evidence. In addition, it illustrates the central role that pharmacological findings in applied clinical contexts play in the acquisition of neurobiological knowledge in research contexts. (shrink)

This paper is concerned with the problem of experimental error. The prevalent view that experimental errors can be dismissed as a tiresome but trivial blemish on the method of experimentation is criticized. It is stressed that the occurrence of errors in experiments constitutes a permanent feature of the attempt to test theories in the physical world, and this feature deserves proper attention. It is suggested that a classification of types of experimental error may be useful as a heuristic device in (...) studying the nature of these errors. However, the standard classification of systematic and random errors is mathematically based does not focus on the causes of the errors, their origins, or the contexts in which they arise. A new typology of experimental errors is therefore proposed whose criterion is epistemological. This typology reflects the various stages that can be discerned in the execution of an experiment, each stage constituting a category of a certain type of experimental error. The proposed classification consists of four categories which are illustrated by historical cases. (shrink)

There is an ongoing debate on whether or to what degree computer simulations can be likened to experiments. Many philosophers are sceptical whether a strict separation between the two categories is possible and deny that the materiality of experiments makes a difference (Morrison 2009, Parker 2009, Winsberg 2010). Some also like to describe computer simulations as a “third way” between experimental and theoretical research (Rohrlich 1990, Axelrod 2003, Kueppers/Lenhard 2005). In this article I defend the view that computer simulations are (...) not experiments but that they are tools for evaluating the consequences of theories and theoretical assumptions. In order to do so the (alleged) similarities and differences between simulations and experiments are examined. It is found that three fundamental differences between simulations and experiments remain: 1) Only experiments can generate new empirical data. 2) Only Experiments can operate directly on the target system. 3) Experiments alone can be employed for testing fundamental hypotheses. As a consequence, experiments enjoy a distinct epistemic role in science that cannot completely be superseded by computer simulations. This finding in connection with a discussion of border cases such as hybrid methods that combine measurement with simulation shows that computer simulations can clearly be distinguished from empirical methods. It is important to understand that computer simulations are not experiments, because otherwise there is a danger of systematically underestimating the need for empirical validation of simulations. (shrink)

Empiricism has traditionally been concerned with two questions: What is the source of our concepts and ideas? and How should claims to empirical knowledge be judged? The empiricist answer to the first question is ‘From observation or experience.’ The concern in the second question is not to ground science in pure observation or in direct experience, but rather to ensure that claims to scientific knowledge are judged against the natural phenomena themselves. Questions about nature must be settled by nature — (...) not by faith, nor metaphysics, nor mathematics, and not by convention or convenience either. From Francis Bacon to Karl Popper empiricists have wanted to police the methods of scientific enquiry. (shrink)

This paper addresses philosophical issues concerning whether mental disorders are natural kinds and how the DSM should classify mental disorders. I argue that some mental disorders (e.g., schizophrenia, depression) are natural kinds in the sense that they are natural classes constituted by a set of stable biological mechanisms. I subsequently argue that a theoretical and causal approach to classification would provide a superior method for classifying natural kinds than the purely descriptive approach adopted by the DSM since DSM-III. My argument (...) suggests that the DSM should classify natural kinds in order to provide predictively useful (i.e., projectable) diagnostic categories and that a causal approach to classification would provide a more promising method for formulating valid diagnostic categories. (shrink)

Robustness is often presented as a guideline for distinguishing the true or real from mere appearances or artifacts. Most of recent discussions of robustness have focused on the kind of derivational robustness analysis introduced by Levins, while the related but distinct idea of robustness as multiple accessibility, defended by Wimsatt, has received less attention. In this paper, I argue that the latter kind of robustness, when properly understood, can provide justification for ontological commitments. The idea is that we are justified (...) in believing that things studied by science are real insofar as we have robust evidence for them. I develop and analyze this idea in detail, and based on concrete examples show that it plays an important role in science. Finally, I demonstrate how robustness can be used to clarify the debate on scientific realism and to formulate new arguments. (shrink)

H. Collins has challenged the empiricist understanding of experimentation by identifying what he thinks constitutes the experimenter’s regress: an instrument is deemed good because it produces good results, and vice versa. The calibration of an instrument cannot alone validate the results: the regressive circling is broken by an agreement essentially external to experimental procedures. In response, A. Franklin has argued that calibration is a key reasonable strategy physicists use to validate production of results independently of their interpretation. The physicists’ arguments (...) about the merits of calibration are not coextensive with the interpretation of results, and thus an objective validation of results is possible. I argue, however, that the in-situ calibrating and measurement procedures and parameters at the Large Hadron Collider are closely and systematically interrelated. This requires empiricists to question their insistence on the independence of calibration from the outcomes of the experiment and rethink their position. Yet this does not leave the case of in-situ calibration open to the experimenter’s regress argument; it is predicated on too crude a view of the relationship between calibration and measurement that fails to capture crucial subtleties of the case. (shrink)

According to the methodological principle called ‘robustness’, empirical evidence is more reliable when it is generated using multiple, independent (experimental) routes that converge on the same result. As it happens, robustness as a methodological strategy is quite popular amongst philosophers. However, despite its popularity, my goal here is to criticize the value of this principle on historical grounds. My historical reasons take into consideration some recent history of astroparticle physics concerning the search for WIMPs (weakly interacting massive particles), one of (...) the main candidates for cosmic dark matter. On the basis of these reasons, I assert that robustness, at least in this historical case we are considering, has less value than usually assumed by philosophers. (shrink)

The use of multiple means of determination to “triangulate” on the existence and character of a common phenomenon, object, or result has had a long tradition in science but has seldom been a matter of primary focus. As with many traditions, it is traceable to Aristotle, who valued having multiple explanations of a phenomenon, and it may also be involved in his distinction between special objects of sense and common sensibles. It is implicit though not emphasized in the distinction between (...) primary and secondary qualities from Galileo onward. It is arguably one of several conceptions involved in Whewell’s method of the “consilience of inductions” (Laudan 1971) and is to be found in several places in Peirce. (From M. Brewer and B. Collins, eds., (1981); Scientific Inquiry in the Social Sciences (a festschrift for Donald T. Campbell), San Francisco: Jossey-Bass, pp. 123–162.). (shrink)

Varied evidence confirms more strongly than less varied evidence, ceteris paribus. This epistemological Variety of Evidence Thesis enjoys widespread intuitive support. We put forward a novel explication of one notion of varied evidence and the Variety of Evidence Thesis within Bayesian models of scientific inference by appealing to measures of entropy. Our explication of the Variety of Evidence Thesis holds in many of our models which also pronounce on disconfirmatory and discordant evidence. We argue that our models pronounce rightly. Against (...) a backdrop of failures of the Variety of Evidence Thesis, the intuitive case for the Variety of Evidence Thesis emerges strengthened. Our models do however not support the general case for the thesis since our explication of it fails to hold in certain cases. The parameter space of this failure is explored and an explanation for the failure is offered. (shrink)

In the philosophical literature concerning scientific experimentation, the notion of robustness has been solely discussed in relation to experimental results. In this paper, I propose a novel sense of experimental robustness that applies to experimental procedures. I call the foregoing sense of robustness procedure robustness and characterize it as the capacity of an experimental procedure to maintain its intended function invariant during the experimental process despite possible variations in its inputs. I argue that PR is a precondition for what I (...) call result robustness, which refers to the traditional sense of experimental robustness, namely the existence of convergent experimental results obtained through different and independent means of detection. Furthermore, I argue, PR and RR constitute useful experimental strategies in the context of high-energy physics experiments, but these strategies are not without limitations. (shrink)

Policy-makers face an uncertain world. One way of getting a handle on decision-making in such an environment is to rely on evidence. Despite the recent increase in post-fact figures in politics, evidence-based policymaking takes centre stage in policy-setting institutions. Often, however, policy-makers face large volumes of evidence from different sources. Robustness analysis can, prima facie, handle this evidential diversity. Roughly, a hypothesis is supported by robust evidence if the different evidential sources are in agreement. In this thesis, I strengthen the (...) case for the use of robustness analysis in evidence-based policymaking by answering open research questions about this inference technique. First, I argue that existing taxonomies miss a fruitful category of robustness reasoning, that is predictive stability. Second, I claim that derivational robustness analysis – the investigation of whether the results of different models are in agreement – can yield interesting insights even if not the entire relevant model space is covered by available models or if the model results are only partially in agreement. Third, I claim that expert knowledge is necessary to address questions that arise when one applies measurement robustness analysis – the investigation into whether multiple means of measurement yield the same result. Finally, I argue that, in situations where evidence from different measurements is not in agreement, it can be advisable to no longer take all of the evidence into account. This can be done in a rationally defensible way by choosing the most adequate theory or model underlying parts of the evidence set. I discuss examples from climate, medical, and economic policy-making to establish my claims. (shrink)

This article examines a metaphilosophical issue, namely existing disagreements in philosophy of science about the significance of using multiple means of determination in scientific practice. We argue that this disagreement can, in part, be resolved by separating different questions that can be asked about the use of multiple means of determination, including the following: what can be concluded from the convergence of data or the convergence of claims about phenomena? Are the conclusions drawn from the convergence of data and of (...) statements about phenomena of special importance to the debate about realism and antirealism? Do inferences based on multiple means of determination have stronger epistemic force than inferences that are secured in other ways? Is the epistemic goal of deploying multiple means of determination well entrenched within the scientific community? Most of these questions can be discussed both in a formal and in an empirical perspective. If the differences in perspective are taken into account, some disagreements can be easily resolved. In part, however, the disagreements reflect historiographical challenges that are very difficult, if not impossible to meet. (shrink)

Experiments often disagree. How then can scientific knowledge be based on experimental evidence? In this paper I will examine four episodes from the history of recent physics: the suggestion of a Fifth Force, a modification of Newton’s law of gravitation; early attempts to detect gravitational radiation ; the claim that a 17-keV neutrino exists; and experiments on atomic-parity violation and on the scattering of polarized electrons and their relation to the Weinberg-Salam unified theory of electroweak interactions. In each of these (...) episodes discordant results were reported, and a consensus was later reached that one result—or set of results—was incorrect. I will examine the process of reaching that consensus. I will show that the decision was reached by reasoned discussion based on epistemological and methodological criteria. It then follows that we may use experimental evidence as the basis of scientific knowledge. (shrink)

Robustness concepts are often invoked to manage two obstacles confronting models of ecological systems: complexity and uncertainty. The intuitive idea is that any result derived from many idealized but credible models is thereby made more reliable or is better confirmed. An appropriate basis for this inference has proven elusive. Here, several representations of robustness analysis are vetted, paying particular attention to complex models of ecosystems and the global climate. The claim that robustness is itself confirmatory because robustness analysis employs a (...) Bayesian variety-of-evidence argument is criticized, but recent overwhelming pessimism about robustness may have a silver lining. (shrink)

The replication or reproducibility crisis in psychological science has renewed attention to philosophical aspects of its methodology. I provide herein a new, functional account of the role of replication in a scientific discipline: to undercut the underdetermination of scientific hypotheses from data, typically by hypotheses that connect data with phenomena. These include hypotheses that concern sampling error, experimental control, and operationalization. How a scientific hypothesis could be underdetermined in one of these ways depends on a scientific discipline’s epistemic goals, theoretical (...) development, material constraints, institutional context, and their interconnections. I illustrate how these apply to the case of psychological science. I then contrast this “bottom-up” account with “top-down” accounts, which assume that the role of replication in a particular science, such as psychology, must follow from a uniform role that it plays in science generally. Aside from avoiding unaddressed problems with top-down accounts, my bottom-up account also better explains the variability of importance of replication of various types across different scientific disciplines. (shrink)

This paper examines how scientists conceptualize their research methodologies. Do scientists raise concerns about vague criteria and genuine uncertainties in experimental practice? If so, what sorts of issues do they identify as problematic? Do scientists acknowledge the presence of value judgments in scientific research, and do they reflect on the relation between epistemic and non-epistemic criteria for decisionmaking? We present findings from an analysis of qualitative interviews with 63 scientific researchers who talk about their views on good research practice. We (...) argue that analysts of science should care about scientists’ conceptualizations of the criteria and of the practical judgments that scientific inquiry involves. While scientists’ accounts of their own research methodologies alone do not give us a full picture of how science really works, they can point us to areas of concern. They can inspire and direct philosophical reflections about how science works. Throughout the interviews, the participating researchers provided specific examples from their own research contexts as illustrations of their methodological points. These examples reveal that scientists often struggle to evaluate the quality of their data, to figure out whether the available evidence confirms their hypothesis, whether a replication was successful, or to what extent they can rely on peer-reviewed papers. General ideas about good research methods do not directly translate into specific evaluation criteria or strategies that can guide research and help validate empirical data. (shrink)

While it is widely acknowledged that psychology is in the throes of a replication ‘crisis’, relatively little attention has been paid to the role theory plays in our evaluation of replications as ‘failed’ or ‘successful’. This paper applies well-known arguments in philosophy of science about the interplay between theory and experiment to a contemporary case study of infants’ understanding of false belief (Onishi and Baillargeon [2005]), and attempts to replicate it. It argues that the lack of consensus about over-arching theories (...) informing both the concepts under study and the methodologies used to track them means that researchers disagree over which experiments constitute replications of the original. The second part of the paper places this specific debate within a broader discussion of the replication crisis as a crisis of ‘theory’, developing work by Muthukrishna and Henrich ([2018]) and Bird ([2018]). Bird argues that the lack of agreed over-arching theories in psychology means that a high rate of replication failure is to be expected; this paper agrees with his diagnosis but challenges his proposal that more replication will resolve the problem. (shrink)

Varied evidence for a hypothesis confirms it more strongly than less varied evidence, ceteris paribus. This epistemological Variety of Evidence Thesis enjoys long-standing widespread intuitive support. Recent literature has raised serious doubts that the correlational approach of explicating the thesis can vindicate it. By contrast, the eliminative approach due to Horwich vindicates the Variety of Evidence Thesis but only within a relatively narrow domain. I investigate the prospects of extending the eliminative approach to a larger domain by considering a larger (...) class of sensible explications of evidential variety. For a large subclass class of such explications I show how to construct cases in which the less diverse body of evidence for a hypothesis confirms more strongly. I hence argue that these prospects are dire since the eliminative approach widely fails to vindicate the thesis. (shrink)

This is my personal homepage. Find my philosophical papers under "Philosophy".

Traditional philosophy of science believes that scientists can achieve agreement on every experimental result provided it can be replicated in an appropriate way, that is, reproducible with the same experimen­tal arrangement and procedure. By analyzing the role of skills in experiment appraisal, I explain why in fact scientists do not always have consensus on experimental results despite their replication attempts. Based on a detailed analysis of a historical case, I argue that experiment replications inevitably involve a processor skill­ transference, which (...) is frequently not articulated in linguistic dis­courses. Hence, it is very difficult to make identical replications if experimental reports are the only resources. Furthermore, I argue that, because transferred skills have to be integrated with scientists' prior experience, skill-transference is sensitive to contextual factors, which can prevent scientists from reaching consensus on experimental results by influencing the effectiveness of communication in experi­ment appraisal. (shrink)

A common methodological adage holds that diverse evidence better confirms a hypothesis than does the same amount of similar evidence. Proponents of Bayesian approaches to scientific reasoning such as Horwich, Howson and Urbach, and Earman claim to offer both a precise rendering of this maxim in probabilistic terms and an explanation of why the maxim should be part of the methodological canon of good science. This paper contends that these claims are mistaken and that, at best, Bayesian accounts of diverse (...) evidence are crucially incomplete. This failure should lend renewed force to a long-neglected global worry about Bayesian approaches. (shrink)

A familiar adage in the philosophy of science is that general hypotheses are better supported by varied evidence than by uniform evidence. Several studies suggest that young children do not respect this principle, and thus suffer from a defect in their inductive methodology. We argue that the diversity principle does not have the normative status that psychologists attribute to it, and should be replaced by a simple rule of probability. We then report experiments designed to detect conformity to the latter (...) rule in children's inductive judgment. The results suggest that young children in both the United States and Taiwan are sensitive to the constraints imposed by the rule on judgments of probability and evidential strength. We conclude with a suggested reinterpretation of the thesis that children's inductive methodology qualifies them as “little scientists.”. (shrink)

I present a justification for the intution that more-varied data are more valuable than the same number of less-varied data by showing that the more-varied data help to improve the accuracy of our predictions.

According to a standard account of evidence, one piece of information is stronger evidence for an hypothesis than is another iff the probability of the hypothesis on the one is greater than it is on the other. This condition, I argue, is neither necessary nor sufficient because various factors can strengthen the evidence for an hypothesis without increasing (and even decreasing) its probability. Contrary to what probabilists claim, I show that this obtains even if a probability function can take these (...) evidential factors into account in ways they suggest and yield a unique probability value. Nor will the problem be solved by appealing to second-order probabilities. (shrink)

This article seeks rapprochement between the sociology of knowledge and philosophy of science by attempting to capture the best social constructionist insights within a strongly realistic philosophy of science. Key to doing so are separating the grounds for the individual scientist coming to know that P from those grounds for socially credentialing the claim that P within the relevant scientific subcommunity and showing how truth considerations can enter into the analysis of knowledge without interfering with social constructionist treatments of credentialing (...) claims. A detailed epistemology and associated account of the social credentialing of knowledge claims are presented. Close attention is paid to the epistemological and social aspects of the nature and role of observation reports, the interpretation of data, simplifications, idealizations, and approximations, and the nature of replication. The article contains a National Aeronautics and Space Administration remote-sensing case study and a detailed analysis of replication in the 1881–1933 Michelson-Morley-Miller interferometer ether studies. (shrink)

How can one determine if an experimental apparatus is giving an incorrect result, if it is speaking falsely? An interesting example of this occurred in the experimental investigation, in the early twentieth century, of the energy spectrum of electrons emitted in β decay. Meitner and her collaborators (1911), using photographic detection, found that all the electrons emitted by a single radioactive element were monoenergetic. Chadwick (1914), on the other hand, using either an ionization chamber or a Geiger counter, found a (...) continuous energy spectrum. Meitner et al. proposed various mechanisms whereby initially monoenergetic electrons might lose energy. These were shown to be unsatisfactory, although the possibility of an unknown mechanism for energy loss remained. In 1927 Ellis and Wooster, using a total-absorption calorimeter, which eliminated all of these possibilities, demonstrated that the energy spectrum was indeed continuous. It had taken fifteen years to show that the photographic detection had spoken falsely. (shrink)

Laboratory experimentation was once considered impossible or irrelevant in economics. Recently, however, economic science has gone through a real ‘laboratory revolution’, and experimental economics is now a most lively subfield of the discipline. The methodological advantages and disadvantages of controlled experimentation constitute the main subject of this thesis. After a survey of the literature on experiments in philosophy and economics, the problem of testing normative theories of rationality is tackled. This philosophical issue was at the centre of a famous controversy (...) in decision theory, during which a methodology of normative falsification was first articulated and used to assess experimental results. In the third chapter, the methodological advantages of controlled experimentation are illustrated and discussed with examples taken from the experiments on the so-called ‘preference reversal’ phenomenon. Laboratory testing allows to establish with a high degree of certainty that certain phenomena lie behind the experimental data, by means of independent testing, elimination of alternative hypotheses, and the use of different instruments of observation. The fourth chapter is devoted to a conceptual analysis of the problem of ‘parallelism’. This is the problem of inferring from the occurrence of a phenomenon in the laboratory, to its instantiation also in non-laboratory environments. Experimental economists have discussed parallelism at length, and their views are presented and criticised. Eventually, it is argued that parallelism is a factual matter and as such can only be established on empirical grounds. The fifth chapter provides an example of how one can argue for parallelism, focusing on the case of experimentation on the ‘winner’s curse’ phenomenon. The role of experiments as ‘mediators’ between theoretical models and their target domain of application is illustrated, and the structure of parallelism arguments analysed in detail. Finally, in the last chapter, economic experiments are compared to simulations, in order to highlight their specific characteristics. (shrink)

According to the variety of evidence thesis items of evidence from independent lines of investigation are more confirmatory, ceteris paribus, than, for example, replications of analogous studies. This thesis is known to fail (Bovens and Hartmann; Claveau). However, the results obtained by Bovens and Hartmann only concern instruments whose evidence is either fully random or perfectly reliable; instead, for Claveau, unreliability is modelled as deterministic bias. In both cases, the unreliable instrument delivers totally irrelevant information. We present a model that (...) formalizes both reliability and unreliability differently. Our instruments either are reliable, but affected by random error, or are biased but not deterministically so.Bovens and Hartmann’s results are counter-intuitive in that in their model a long series of consistent reports from the same instrument does not raise suspicion of ‘too-good-to-be-true’ evidence. This happens precisely because they contemplate neither the role of systematic bias, nor unavoidable random error of reliable instruments. In our model, the variety of evidence thesis fails as well, but the area of failure is considerably smaller than for Bovens and Hartmann and Claveau, and holds for (the majority of) realistic cases (that is, where biased instruments are very biased). The essential mechanism that triggers variety of evidence thesis failure is the rate of false to true positives for the two kinds of instruments. Our emphasis is on modelling beliefs about sources of knowledge and their role in hypothesis confirmation in interaction with dimensions of evidence, such as variety and consistency. (shrink)