When scientists seek further confirmation of their results, they often attempt to duplicate the results using diverse means. To the extent that they are successful in doing so, their results are said to be robust. This paper investigates the logic of such "robustness analysis" [RA]. The most important and challenging question an account of RA can answer is what sense of evidential diversity is involved in RAs. I argue that prevailing formal explications of such diversity are unsatisfactory. I propose a (...) unified, explanatory account of diversity in RAs. The resulting account is, I argue, truer to actual cases of RA in science; moreover, this account affords us a helpful, new foothold on the logic undergirding RAs. (shrink)

According to the methodological principle called ‘robustness’, empirical evidence is more reliable when it is generated using multiple, independent (experimental) routes that converge on the same result. As it happens, robustness as a methodological strategy is quite popular amongst philosophers. However, despite its popularity, my goal here is to criticize the value of this principle on historical grounds. My historical reasons take into consideration some recent history of astroparticle physics concerning the search for WIMPs (weakly interacting massive particles), one of (...) the main candidates for cosmic dark matter. On the basis of these reasons, I assert that robustness, at least in this historical case we are considering, has less value than usually assumed by philosophers. (shrink)

Recent philosophical attention to climate models has highlighted their weaknesses and uncertainties. Here I address the ways that models gain support through observational data. I review examples of model fit, variety of evidence, and independent support for aspects of the models, contrasting my analysis with that of other philosophers. I also investigate model robustness, which often emerges when comparing climate models simulating the same time period or set of conditions. Starting from Michael Weisberg’s analysis of robustness, I conclude that his (...) approach involves a version of reasoning from variety of evidence, enabling this robustness to be a confirmatory virtue.. (shrink)

Many philosophers have claimed that evidence for a theory is better when multiple independent tests yield the same result, i.e., when experimental results are robust. Little has been said about the grounds on which such a claim rests, however. The present essay presents an analysis of the evidential value of robustness that rests on the fallibility of assumptions about the reliability of testing procedures and a distinction between the strength of evidence and the security of an evidence claim. Robustness can (...) enhance the security of an evidence claim either by providing what I call second-order evidence, or by providing back-up evidence for a hypothesis. (shrink)

Robustness arguments hold that hypotheses are more likely to be true when they are confirmed by diverse kinds of evidence. Robustness arguments require the confirming evidence to be independent. We identify two kinds of independence appealed to in robustness arguments: ontic independence —when the multiple lines of evidence depend on different materials, assumptions, or theories—and probabilistic independence. Many assume that OI is sufficient for a robustness argument to be warranted. However, we argue that, as typically construed, OI is not a (...) sufficient independence condition for warranting robustness arguments. We show that OI evidence can collectively confirm a hypothesis to a lower degree than individual lines of evidence, contrary to the standard assumption undergirding usual robustness arguments. We employ Bayesian networks to represent the ideal empirical scenario for a robustness argument and a variety of ways in which empirical scenarios can fall short of this ideal. (shrink)

Modelers often rely on robustness analysis, the search for predictions common to several independent models. Robustness analysis has been characterized and championed by Richard Levins and William Wimsatt, who see it as central to modern theoretical practice. The practice has also been severely criticized by Steven Orzack and Elliott Sober, who claim that it is a nonempirical form of confirmation, effective only under unusual circumstances. This paper addresses Orzack and Sober's criticisms by giving a new account of robustness analysis and (...) showing how the practice can identify robust theorems. Once the structure of robust theorems is clearly articulated, it can be shown that such theorems have a degree of confirmation, despite the lack of direct empirical evidence for their truth. (shrink)

The use of multiple means of determination to “triangulate” on the existence and character of a common phenomenon, object, or result has had a long tradition in science but has seldom been a matter of primary focus. As with many traditions, it is traceable to Aristotle, who valued having multiple explanations of a phenomenon, and it may also be involved in his distinction between special objects of sense and common sensibles. It is implicit though not emphasized in the distinction between (...) primary and secondary qualities from Galileo onward. It is arguably one of several conceptions involved in Whewell’s method of the “consilience of inductions” (Laudan 1971) and is to be found in several places in Peirce. (From M. Brewer and B. Collins, eds., (1981); Scientific Inquiry in the Social Sciences (a festschrift for Donald T. Campbell), San Francisco: Jossey-Bass, pp. 123–162.). (shrink)

Rasmussen (1993) argues that, because electron microscopists did not use robustness and would not have been warranted in using it as a criterion for the reality or the artifactuality of mesosomes, the bacterial mesosome serves as a test case for robustness that it fails. I respond by arguing that a more complete reading of the research literature on the mesosome shows that ultimately the more robust body of data did not support the mesosome and that electron microscopists used and were (...) warranted in using robustness as a criterion for the artifactuality of mesosomes. (shrink)

Theory-ladenness is the view that observation cannot function in an unbiased way in the testing of theories because observational judgments are affected by the theoretical beliefs of the observer. Its more radical cousin, incommensurability, argues that because there is no theory-neutral language, paradigms, or worldviews, cannot be compared because in different paradigms the meaning of observational terms is different, even when the word used is the same. There are both philosophical and practical components to these problems. I argue, using a (...) procedurally-defined, theory-neutral experiment that paradigms are indeed commensurable. The practical problems of theory ladenness include experimental design, failure to interpret observations correctly, possible experimenter bias, and difficulties in data acquisition. I suggest that there are methods to deal with these problems, although sometimes they cannot be dealt with completely. I believe that the philosophical problems of theory-ladenness have been solved, although the practical problems remain. (shrink)

Specifically, Allan Franklin is concerned with two problems in the use of experimental results in science: selectivity of data or analysis procedures and the resolution of discordant results.

In this paper, I propose an account that accommodates the possibility of experimentation being exploratory in cases where the procedures necessary to plan and perform an experiment are dependent on the theoretical accounts of the phenomena under investigation. The present account suggests that experimental exploration requires the implementation of an exploratory procedure that serves to extend the range of possible outcomes of an experiment, thereby enabling it to pursue its objectives. Furthermore, I argue that the present account subsumes the notion (...) of exploratory experimentation, which is often attributed in the relevant literature to the works of Friedrich Steinle and Richard Burian, as a particular type of experimental exploration carried out in the special cases where no well-formed theoretical framework of the phenomena under investigation exists. I illustrate the present account in the context of the ATLAS experiment at CERN’s Large Hadron Collider, where the long-sought Higgs boson has been discovered in 2012. I argue that the data selection procedure carried out in the ATLAS experiment illustrates an exploratory procedure in the sense suggested by the present account. I point out that this particular data selection procedure is theory-laden in that its implementation is crucially dependent on the theoretical models of high energy particle physics which the ATLAS experiment is aimed to test. However, I argue that the foregoing procedure is not driven by the above-mentioned theoretical models, but rather by a particular data selection strategy. I conclude that the ATLAS experiment illustrates that, contrary to what previous studies have suggested, there are cases of experimentation in which exploration serves to test theoretical predictions and that theory-ladenness plays an essential role in experimentation being exploratory. (shrink)

Significance testing is the primary method for establishing causal relationships in psychology. Meehl [1978, 1990a, 1990b] and Faust [1984] argue that significance tests and their interpretation are subject to actuarial and psychological biases, making continued adherence to these practices irrational, and even partially responsible for the slow progress of the ‘soft’ areas of psychology. I contend that familiar standards of testing and literature review, along with recently developed meta-analytic techniques, are able to correct the proposed actuarial and psychological biases. In (...) particular, psychologists embrace a principle of robustness which states that real psychological effects are (1) reproducible by similar methods, (2) detectable by diverse means, and (3) able to survive theoretical integration. By contrast, spurious significant findings perish under the strain of persistent tests of their robustness. The resulting vindication of significance testing confers on the world a role in determining the rationality of a method, and also affords us an explanation for the fast progress of ‘hard’ areas of psychology. *I would like to thank Dick Boyd and Phil Gasper for helpful comments on the ideas presented here. (shrink)

The success of particle detection in high energy physics colliders critically depends on the criteria for selecting a small number of interactions from an overwhelming number that occur in the detector. It also depends on the selection of the exact data to be analyzed and the techniques of analysis. The introduction of automation into the detection process has traded the direct involvement of the physicist at each stage of selection and analysis for the efficient handling of vast amounts of data. (...) This tradeoff, in combination with the organizational changes in laboratories of increasing size and complexity, has resulted in automated and semi-automated systems of detection. Various aspects of the semi-automated regime were greatly diminished in more generic automated systems, but turned out to be essential to a number of surprising discoveries of anomalous processes that led to theoretical breakthroughs, notably the establishment of the Standard Model of particle physics. The automated systems are much more efficient in confirming specific hypothesis in narrow energy domains than in performing broad exploratory searches. Thus, in the main, detection processes relying excessively on automation are more likely to miss potential anomalies and impede potential theoretical advances. I suggest that putting substantially more effort into the study of electron–positron colliders and increasing its funding could minimize the likelihood of missing potential anomalies, because detection in such an environment can be handled by the semi-automated regime—unlike detection in hadron colliders. Despite virtually unavoidable excessive reliance on automated detection in hadron colliders, their development has been deemed a priority because they can operate at currently highest energy levels. I suggest, however, that a focus on collisions at the highest achievable energy levels diverts funds from searches for potential anomalies overlooked due to tradeoffs at the previous energy thresholds. I also note that even in the same collision environment, different research strategies will opt for different tradeoffs and thus achieve different experimental outcomes. Finally, I briefly discuss current searches for anomalous process in the context of the previous analysis. (shrink)

Culp (1994) provides a defense for a form of experimental reasoning entitled 'robustness'. Her strategy is to examine a recent episode in experimental microbiology--the case of the mistaken discovery of a bacterial organelle called a 'mesosome'--with an eye to showing how experimenters effectively used robust experimental reasoning (or could have used robust reasoning) to refute the existence of the mesosome. My plan is to criticize Culp's assessment of the mesosome episode and to cast doubt on the epistemic significance of robustness. (...) In turn, I present a different account of the experimental reasoning microbiologists used in arriving at the conclusion that mesosomes are artifacts. I call this form of reasoning 'reliable process reasoning', and close the paper with a brief discussion of how experimental microbiologists justify the claim that an experimental process is reliable. (shrink)