While it is widely acknowledged that psychology is in the throes of a replication ‘crisis’, relatively little attention has been paid to the role theory plays in our evaluation of replications as ‘failed’ or ‘successful’. This paper applies well-known arguments in philosophy of science about the interplay between theory and experiment to a contemporary case study of infants’ understanding of false belief (Onishi and Baillargeon [2005]), and attempts to replicate it. It argues that the lack of consensus about over-arching theories (...) informing both the concepts under study and the methodologies used to track them means that researchers disagree over which experiments constitute replications of the original. The second part of the paper places this specific debate within a broader discussion of the replication crisis as a crisis of ‘theory’, developing work by Muthukrishna and Henrich ([2018]) and Bird ([2018]). Bird argues that the lack of agreed over-arching theories in psychology means that a high rate of replication failure is to be expected; this paper agrees with his diagnosis but challenges his proposal that more replication will resolve the problem. (shrink)

Laboratory experimentation was once considered impossible or irrelevant in economics. Recently, however, economic science has gone through a real ‘laboratory revolution’, and experimental economics is now a most lively subfield of the discipline. The methodological advantages and disadvantages of controlled experimentation constitute the main subject of this thesis. After a survey of the literature on experiments in philosophy and economics, the problem of testing normative theories of rationality is tackled. This philosophical issue was at the centre of a famous controversy (...) in decision theory, during which a methodology of normative falsification was first articulated and used to assess experimental results. In the third chapter, the methodological advantages of controlled experimentation are illustrated and discussed with examples taken from the experiments on the so-called ‘preference reversal’ phenomenon. Laboratory testing allows to establish with a high degree of certainty that certain phenomena lie behind the experimental data, by means of independent testing, elimination of alternative hypotheses, and the use of different instruments of observation. The fourth chapter is devoted to a conceptual analysis of the problem of ‘parallelism’. This is the problem of inferring from the occurrence of a phenomenon in the laboratory, to its instantiation also in non-laboratory environments. Experimental economists have discussed parallelism at length, and their views are presented and criticised. Eventually, it is argued that parallelism is a factual matter and as such can only be established on empirical grounds. The fifth chapter provides an example of how one can argue for parallelism, focusing on the case of experimentation on the ‘winner’s curse’ phenomenon. The role of experiments as ‘mediators’ between theoretical models and their target domain of application is illustrated, and the structure of parallelism arguments analysed in detail. Finally, in the last chapter, economic experiments are compared to simulations, in order to highlight their specific characteristics. (shrink)

The last forty years have produced a dramatic reversal in leading accounts of science. Once thought necessary to (explain) scientific progress, a rigid method of science is now widely considered impossible. Study of products yields to study of processes and practices, .unity gives way to diversity, generality to particularity, logic to luck, and final justification to heuristic scaffolding. I sketch the story, from Bacon and Descartes to the present, of the decline and fall of traditional scientific method, conceived as The (...) Central Planning Bureau for Science or as Rationality Czar. I defend a deflationary account of method and of rational judgment,. with emphasis on heuristic appraisal and cognitive economy. (shrink)

Experimental data are often acclaimed on the grounds that they can be consistently generated. They are, it is said, reproducible. In this paper I describe how this feature of experimental-data (their pragmatic reliability) leads to their epistemic worth (their epistemic reliability). An important part of my description is the supposition that experimental procedures are to certain extent fixed and stable. Various illustrations from the actual practice of science are introduced, the most important coming at the end of the paper with (...) a discussion of Ray Davis' 1967 solar-neutrino detection experiment (as it is portrayed in Pinch, 1980). (shrink)

Policy-makers face an uncertain world. One way of getting a handle on decision-making in such an environment is to rely on evidence. Despite the recent increase in post-fact figures in politics, evidence-based policymaking takes centre stage in policy-setting institutions. Often, however, policy-makers face large volumes of evidence from different sources. Robustness analysis can, prima facie, handle this evidential diversity. Roughly, a hypothesis is supported by robust evidence if the different evidential sources are in agreement. In this thesis, I strengthen the (...) case for the use of robustness analysis in evidence-based policymaking by answering open research questions about this inference technique. First, I argue that existing taxonomies miss a fruitful category of robustness reasoning, that is predictive stability. Second, I claim that derivational robustness analysis – the investigation of whether the results of different models are in agreement – can yield interesting insights even if not the entire relevant model space is covered by available models or if the model results are only partially in agreement. Third, I claim that expert knowledge is necessary to address questions that arise when one applies measurement robustness analysis – the investigation into whether multiple means of measurement yield the same result. Finally, I argue that, in situations where evidence from different measurements is not in agreement, it can be advisable to no longer take all of the evidence into account. This can be done in a rationally defensible way by choosing the most adequate theory or model underlying parts of the evidence set. I discuss examples from climate, medical, and economic policy-making to establish my claims. (shrink)

This article seeks rapprochement between the sociology of knowledge and philosophy of science by attempting to capture the best social constructionist insights within a strongly realistic philosophy of science. Key to doing so are separating the grounds for the individual scientist coming to know that P from those grounds for socially credentialing the claim that P within the relevant scientific subcommunity and showing how truth considerations can enter into the analysis of knowledge without interfering with social constructionist treatments of credentialing (...) claims. A detailed epistemology and associated account of the social credentialing of knowledge claims are presented. Close attention is paid to the epistemological and social aspects of the nature and role of observation reports, the interpretation of data, simplifications, idealizations, and approximations, and the nature of replication. The article contains a National Aeronautics and Space Administration remote-sensing case study and a detailed analysis of replication in the 1881–1933 Michelson-Morley-Miller interferometer ether studies. (shrink)