Sections include: experiments and generalised causal inference; statistical conclusion validity and internal validity; construct validity and external validity; quasi-experimental designs that either lack a control group or lack pretest observations on the outcome; quasi-experimental designs that use both control groups and pretests; quasi-experiments: interrupted time-series designs; regresssion discontinuity designs; randomised experiments: rationale, designs, and conditions conducive to doing them; practical problems 1: ethics, participation recruitment and random assignment; practical problems 2: treatment implementation and attrition; generalised causal inference: a grounded theory; (...) generalised causal inference: methods for single studies; generalised causal inference: methods for multiple studies; a critical assessment of our assumptions. (shrink)

Recent philosophical work has praised the reward structure of science, while recent empirical work has shown that many scientific results may not be reproducible. I argue that the reward structure of science incentivizes scientists to focus on speed and impact at the expense of the reproducibility of their work, thus contributing to the so-called reproducibility crisis. I use a rational choice model to identify a set of sufficient conditions for this problem to arise, and I argue that these conditions plausibly (...) apply to a wide range of research situations. Currently proposed solutions will not fully address this problem. Philosophical commentators should temper their optimism about the reward structure of science. (shrink)

Starting with some illustrative examples, I develop a systematic account of a specific type of experimentation--an experimentation which is not, as in the "standard view", driven by specific theories. It is typically practiced in periods in which no theory or--even more fundamentally--no conceptual framework is readily available. I call it exploratory experimentation and I explicate its systematic guidelines. From the historical examples I argue furthermore that exploratory experimentation may have an immense, but hitherto widely neglected, epistemic significance.

Offers a new approach to a number of central issues concerning the theoretical interpretation and normative evaluation of contemporary science and technology.

Experiments are commonly thought to have epistemic privilege over simulations. Two ideas underpin this belief: first, experiments generate greater inferential power than simulations, and second, simulations cannot surprise us the way experiments can. In this article I argue that neither of these claims is true of experiments versus simulations in general. We should give up the common practice of resting in-principle judgments about the epistemic value of cases of scientific inquiry on whether we classify those cases as experiments or simulations, (...) per se. To the extent that either methodology puts researchers in a privileged epistemic position, this is context sensitive. (shrink)

A heated debate surrounds the significance of reproducibility as an indicator for research quality and reliability, with many commentators linking a "crisis of reproducibility" to the rise of fraudulent, careless and unreliable practices of knowledge production. Through the analysis of discourse and practices across research fields, I point out that reproducibility is not only interpreted in different ways, but also serves a variety of epistemic functions depending on the research at hand. Given such variation, I argue that the uncritical pursuit (...) of reproducibility as an overarching epistemic value is misleading and potentially damaging to scientific advancement. Requirements for reproducibility, however they are interpreted, are one of many available means to secure reliable research outcomes. Furthermore, there are cases where the focus on enhancing reproducibility turns out not to foster high-quality research. Scientific communities and Open Science advocates should learn from inferential reasoning from irreproducible data, and promote incentives for all researchers to explicitly and publicly discuss their methodological commitments, the ways in which they learn from mistakes and problems in everyday practice, and the strategies they use to choose which research component of any project needs to be preserved in the long term, and how. (shrink)

Biologists in the last 50 years have increasingly emphasized the role of historical contingency in explaining the distribution and dynamics of biological systems. However, recent work in philosophy of biology has shown that historical contingency carries various interpretations and that we are still lacking a general understanding of historicity, i.e., a framework from which to interpret why and to what extent history matters in biological processes. Building from examples and analyses of the long-term experimental evolution (LTEE) project, this paper argues (...) that historicity possess three essential conditions: (1) multiple possible pasts, (2) multiple possible outcomes at a given instant, and (3) a relationship of causal dependence between these two sets. These criteria can be further specified in two general forms of historicity: dependence on initial conditions and path dependence. More attention is devoted to developing a rigorous account of the latter, which captures the type of historicity displayed by stochastic processes. This paper also highlights that it is often more productive to adopt an instant-relative approach and think in terms of degree of historicity instead of trying to maintain a rigid and absolute dichotomy between historical and ahistorical (completely convergent) processes. (shrink)

My aim in this article is to introduce readers to the topic of exploratory experimentation and briefly explain how the three articles that follow, by Richard Burian, Kevin Elliott, and Maureen O'Malley, advance our understanding of the nature and significance of exploratory research. I suggest that the distinction between exploratory and theory-driven experimentation is multidimensional and that some of the dimensions are continuums. I point out that exploratory experiments are typically theory-informed even if they are not theory-driven. I also distinguish (...) between research programs and experiments. Research programs that are largely exploratory, such as the ones discussed in these case studies, can involve both exploratory and theory-driven experimentation. (shrink)

Experimental modeling in biology involves the use of living organisms (not necessarily so-called "model organisms") in order to model or simulate biological processes. I argue here that experimental modeling is a bona fide form of scientific modeling that plays an epistemic role that is distinct from that of ordinary biological experiments. What distinguishes them from ordinary experiments is that they use what I call "in vivo representations" where one kind of causal process is used to stand in for a physically (...) different kind of process. I discuss the advantages of this approach in the context of evolutionary biology. (shrink)

Exploratory experimentation and high-throughput molecular biology appear to have considerable affinity for each other. Included in the latter category is metagenomics, which is the DNA-based study of diverse microbial communities from a vast range of non-laboratory environments. Metagenomics has already made numerous discoveries and these have led to reinterpretations of fundamental concepts of microbial organization, evolution, and ecology. The most outstanding success story of metagenomics to date involves the discovery of a rhodopsin gene, named proteorhodopsin, in marine bacteria that were (...) never suspected to have any photobiological capacities. A discussion of this finding and its detailed investigation illuminates the relationship between exploratory experimentation and metagenomics. Specifically, the proteorhodopsin story indicates that a dichotomous interpretation of theory-driven and exploratory experimentation is insufficient and that an interactive understanding of these two types of experimentation can be usefully supplemented by another category, "natural history experimentation". Further reflection on the context of metagenomics suggests the necessity of thinking more historically about exploratory and other forms of experimentation. (shrink)

The paper explores standardisation from the perspective of epistemology. Its aim is to enquire into the reality of standards as being very specific tools with defined uses, but at the same time sharing general suppositions about which ends they serve within the realm of science. The paper focuses on the questions how standards relate to ends that facilitate and/or allow for knowledge claims in the sciences. Therefore, scientific practices in different fields of research are assessed, ranging from measurement to experimental (...) trial design in medicine and psychology. (shrink)