When scientists seek further confirmation of their results, they often attempt to duplicate the results using diverse means. To the extent that they are successful in doing so, their results are said to be robust. This paper investigates the logic of such "robustness analysis" [RA]. The most important and challenging question an account of RA can answer is what sense of evidential diversity is involved in RAs. I argue that prevailing formal explications of such diversity are unsatisfactory. I propose a (...) unified, explanatory account of diversity in RAs. The resulting account is, I argue, truer to actual cases of RA in science; moreover, this account affords us a helpful, new foothold on the logic undergirding RAs. (shrink)

Robustness is a common platitude: hypotheses are better supported with evidence generated by multiple techniques that rely on different background assumptions. Robustness has been put to numerous epistemic tasks, including the demarcation of artifacts from real entities, countering the “experimenter’s regress,” and resolving evidential discordance. Despite the frequency of appeals to robustness, the notion itself has received scant critique. Arguments based on robustness can give incorrect conclusions. More worrying is that although robustness may be valuable in ideal evidential circumstances (i.e., (...) when evidence is concordant), often when a variety of evidence is available from multiple techniques, the evidence is discordant. †To contact the author, please write to: Jacob Stegenga, Department of Philosophy, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093; e‐mail: [email protected]. (shrink)

Intellectual courage requires acting to promote epistemic goods despite significant risk of harm. Courage is distinguished from recklessness and cowardice because the expected epistemic benefit of a courageous action outweighs (in some sense) the threatened harm. Sometimes, however, inquirers pursue theories that are not best supported by their current evidence. For these inquirers, the expected epistemic benefit of their actions cannot be explained by appeal to their evidence alone. The probability of pursuing the true theory cannot contribute enough to the (...) expected epistemic benefit for the action to count as courageous rather than reckless. Thus, there must be some other epistemic consideration which favors their action, besides evidence for their theory. I argue that the proper account of intellectual courage requires recognition of inquisitive reasons: a distinct category of epistemic reasons which concern successful inquiry. This category includes reasons to think a theory itself is promising, e.g., that the theory suggests potentially fruitful new research. It also includes social epistemic reasons, e.g., that pursuing a theory will improve the distribution of cognitive labor. Inquisitive reasons help explain why researchers who pursue improbable theories count as intellectually courageous, rather than reckless. The expected epistemic benefit of an action is partially determined by the inquisitive reasons in its favor. On my account, intellectually courageous inquiry requires acting in a way that is sensitive to inquisitive reasons. (shrink)

Recent philosophical attention to climate models has highlighted their weaknesses and uncertainties. Here I address the ways that models gain support through observational data. I review examples of model fit, variety of evidence, and independent support for aspects of the models, contrasting my analysis with that of other philosophers. I also investigate model robustness, which often emerges when comparing climate models simulating the same time period or set of conditions. Starting from Michael Weisberg’s analysis of robustness, I conclude that his (...) approach involves a version of reasoning from variety of evidence, enabling this robustness to be a confirmatory virtue.. (shrink)

This article identifies conditions under which robust predictive modeling results have special epistemic significance---related to truth, confidence, and security---and considers whether those conditions hold in the context of present-day climate modeling. The findings are disappointing. When today’s climate models agree that an interesting hypothesis about future climate change is true, it cannot be inferred---via the arguments considered here anyway---that the hypothesis is likely to be true or that scientists’ confidence in the hypothesis should be significantly increased or that a claim (...) to have evidence for the hypothesis is now more secure. (shrink)

Robustness is often presented as a guideline for distinguishing the true or real from mere appearances or artifacts. Most of recent discussions of robustness have focused on the kind of derivational robustness analysis introduced by Levins, while the related but distinct idea of robustness as multiple accessibility, defended by Wimsatt, has received less attention. In this paper, I argue that the latter kind of robustness, when properly understood, can provide justification for ontological commitments. The idea is that we are justified (...) in believing that things studied by science are real insofar as we have robust evidence for them. I develop and analyze this idea in detail, and based on concrete examples show that it plays an important role in science. Finally, I demonstrate how robustness can be used to clarify the debate on scientific realism and to formulate new arguments. (shrink)

This paper addresses philosophical issues concerning whether mental disorders are natural kinds and how the DSM should classify mental disorders. I argue that some mental disorders (e.g., schizophrenia, depression) are natural kinds in the sense that they are natural classes constituted by a set of stable biological mechanisms. I subsequently argue that a theoretical and causal approach to classification would provide a superior method for classifying natural kinds than the purely descriptive approach adopted by the DSM since DSM-III. My argument (...) suggests that the DSM should classify natural kinds in order to provide predictively useful (i.e., projectable) diagnostic categories and that a causal approach to classification would provide a more promising method for formulating valid diagnostic categories. (shrink)

The article argues for the epistemic rationale of triangulation, namely, the use of multiple and independent sources of evidence. It claims that triangulation is to be understood as causal reasoning from data to phenomenon, and it rationalizes its epistemic value in terms of controlling for likely errors and biases of particular data-generating procedures. This perspective is employed to address objections against triangulation concerning the fallibility and scope of the inference, as well as problems of independence, incomparability, and discordance of evidence. (...) The debate on the existence of social preferences is used as an illustrative case. (shrink)

Computer simulations are nowadays often directly involved in the generation of experimental results. Given this dependency of experiments on computer simulations, that of simulations on models, and that of the models on free parameters, how do researchers establish trust in their experimental results? Using high-energy physics (HEP) as a case study, I will identify three different types of robustness that I call conceptual, methodological, and parametric robustness, and show how they can sanction this trust. However, as I will also show, (...) simulation models in HEP themselves fail to exhibit a type of robustness I call inverse parametric robustness. This combination of robustness and failures thereof is best understood by distinguishing different epistemic capacities of simulations and different senses of trust: Trusting simulations in their capacity to facilitate credible experimental results can mean accepting them as means for generating belief in these results, while this need not imply believing the models themselves in their capacity to represent an underlying reality. (shrink)

Reports of quantitative experimental results often distinguish between the statistical uncertainty and the systematic uncertainty that characterize measurement outcomes. This article discusses the practice of estimating systematic uncertainty in high-energy physics. The estimation of systematic uncertainty in HEP should be understood as a minimal form of quantitative robustness analysis. The secure evidence framework is used to explain the epistemic significance of robustness analysis. However, the empirical value of a measurement result depends crucially not only on the resulting systematic uncertainty estimate, (...) but on the learning aims for which that result will be used. Important conceptual and practical questions regarding systematic uncertainty assessment call for further investigation. _1_ Introduction _2_ Systematic Uncertainty: The Very Idea _3_ Systematic Uncertainty in High-Energy Physics _4_ Methodological Debates in High-Energy Physics _5_ The Secure Evidence Framework _6_ Systematic Uncertainty Assessment as Robustness Analysis _7_ Security and Sensitivity _8_ Conclusion Appendix. (shrink)

In psychiatry, pharmacological drugs play an important experimental role in attempts to identify the neurobiological causes of mental disorders. Besides being developed in applied contexts as potential treatments for patients with mental disorders, pharmacological drugs play a crucial role in research contexts as experimental instruments that facilitate the formulation and revision of neurobiological theories of psychopathology. This paper examines the various epistemic functions that pharmacological drugs serve in the discovery, refinement, testing, and elaboration of neurobiological theories of mental disorders. I (...) articulate this thesis with reference to the history of antipsychotic drugs and the evolution of the dopamine hypothesis of schizophrenia in the second half of the twentieth century. I argue that interventions with psychiatric patients through the medium of antipsychotic drugs provide researchers with information and evidence about the neurobiological causes of schizophrenia. This analysis highlights the importance of pharmacological drugs as research tools in the generation of psychiatric knowledge and the dynamic relationship between practical and theoretical contexts in psychiatry. (shrink)

Today's climate models are supported in a couple of ways that receive little attention from philosophers or climate scientists. In addition to standard 'model fit', wherein a model's simulation is compared to observational data, there is an additional type of confirmation available through the variety of instances of model fit. When a model performs well at fitting first one variable and then another, the probability of the model under some standard confirmation function, say, likelihood, goes up more than under each (...) individual case of fit alone. Thus, two instances of fit of distinct variables of a global climate model using distinct data sets considered collectively will provide stronger evidence for a model than either one of the instances considered individually. This has consequences for model robustness. Sets of models that produce robust results will, if their assumptions vary enough and they each are observationally sound, provide reasons to endorse common structures found in those models. Finally, independent empirical support for aspects and assumptions of the model provides an additional confirmational virtue for climate models, contrary to what is implied by some current philosophical writing on this topic. (shrink)

Amalgamating evidence of different kinds for the same hypothesis into an overall confirmation is analogous, I argue, to amalgamating individuals’ preferences into a group preference. The latter faces well-known impossibility theorems, most famously “Arrow’s Theorem”. Once the analogy between amalgamating evidence and amalgamating preferences is tight, it is obvious that amalgamating evidence might face a theorem similar to Arrow’s. I prove that this is so, and end by discussing the plausibility of the axioms required for the theorem.

Derivational robustness may increase the degree to which various pieces of evidence indirectly confirm a robust result. There are two ways in which this increase may come about. First, if one can show that a result is robust, and that the various individual models used to derive it also have other confirmed results, these other results may indirectly confirm the robust result. Confirmation derives from the fact that data not known to bear on a result are shown to be relevant (...) when it is shown to be robust. Second, robustness may increase the degree to which the robust result is indirectly confirmed if it increases the weight with which existing evidence indirectly confirms it. This may happen when it strengthens the connection between the core and the robust result by showing that auxiliaries are not responsible for the result. (shrink)

Reactivity in qualitative data collection occurs when a researcher generates data about a situation with reactivity, that is, a situation in which the ongoing research affects the research participants such that they, say, diverge from their routines when the researcher is present, or tell the researcher what they think she wants to hear. In qualitative research, there are two basic approaches to reactivity. The traditional position maintains that data should ideally be collected in situations without any reactivity. In other words, (...) good data are reactivity free. By contrast, the more recent view holds that data from situations with reactivity are fine as long as the researcher is aware of the occurring reactivity so that she can take it into account when interpreting her data. In this fashion, good data are reactivity transparent. In this paper, I first spell out and defend the more recent approach to reactivity. I argue that qualitative data are reactivity transparent when conjoined with true reactivity assumptions and that, thus supplemented, data are informative about social life independently of its being studied. Next, I examine various issues raised by the requirement to put forth true reactivity assumptions. Lastly, I use my discussion of reactivity transparency as a basis for providing a framework for thinking about good qualitative data. (shrink)

This article presents a new framework for the analysis of experimental control. The framework highlights different functions for experimental controls in the realization of an experiment: experimental controls that serve as tests and experimental controls that serve as probes. The approach to experimental control proposed here can illuminate the constitutive role of controls in knowledge production, and it sheds new light on the notion of exploratory experimentation. It also clarifies what can and what cannot be expected from reviewers of scientific (...) journal articles giving feedback on experimental controls. (shrink)

In the philosophical literature concerning scientific experimentation, the notion of robustness has been solely discussed in relation to experimental results. In this paper, I propose a novel sense of experimental robustness that applies to experimental procedures. I call the foregoing sense of robustness procedure robustness and characterize it as the capacity of an experimental procedure to maintain its intended function invariant during the experimental process despite possible variations in its inputs. I argue that PR is a precondition for what I (...) call result robustness, which refers to the traditional sense of experimental robustness, namely the existence of convergent experimental results obtained through different and independent means of detection. Furthermore, I argue, PR and RR constitute useful experimental strategies in the context of high-energy physics experiments, but these strategies are not without limitations. (shrink)

Amalgamating evidence from heterogeneous sources and across levels of inquiry is becoming increasingly important in many pure and applied sciences. This special issue provides a forum for researchers from diverse scientific and philosophical perspectives to discuss evidence amalgamation, its methodologies, its history, its pitfalls, and its potential. We situate the contributions therein within six themes from the broad literature on this subject: the variety-of-evidence thesis, the philosophy of meta-analysis, the role of robustness/sensitivity analysis for evidence amalgamation, its bearing on questions (...) of extrapolation and external validity of experiments, its connection with theory development, and its interface with causal inference, especially regarding causal theories of cancer. (shrink)

Multiple determination is the epistemic strategy of establishing the same result by means of multiple, independent procedures. It is an important strategy praised by both philosophers of science and practicing scientists. Despite the heavy appeal to multiple determination, little analysis has been provided regarding the specific grounds upon which its epistemic virtues rest. This article distinguishes between the various dimensions of multiple determination and shows how they can be used to evaluate the epistemic force of the strategy in particular cases. (...) Distinguishing between the various dimensions helps to resolve the disagreements regarding the relevance and epistemic import of multiple determination for scientific research. It also suggests a fruitful mode of interaction between the philosophy of science and (historical) accounts of scientific practice. (shrink)

The paper discusses modelling uncertainties in climate models and how they can be addressed based on physical principles as well as based on how the models perform in light of empirical data. We ar...

This article examines a metaphilosophical issue, namely existing disagreements in philosophy of science about the significance of using multiple means of determination in scientific practice. We argue that this disagreement can, in part, be resolved by separating different questions that can be asked about the use of multiple means of determination, including the following: what can be concluded from the convergence of data or the convergence of claims about phenomena? Are the conclusions drawn from the convergence of data and of (...) statements about phenomena of special importance to the debate about realism and antirealism? Do inferences based on multiple means of determination have stronger epistemic force than inferences that are secured in other ways? Is the epistemic goal of deploying multiple means of determination well entrenched within the scientific community? Most of these questions can be discussed both in a formal and in an empirical perspective. If the differences in perspective are taken into account, some disagreements can be easily resolved. In part, however, the disagreements reflect historiographical challenges that are very difficult, if not impossible to meet. (shrink)

Which non-human animals are phenomenally conscious? In this paper I argue that the distribution of phenomenal consciousness in the animal world is ultimately an unsolvable issue, because of an underlying problem inherent in the field: what I call the Kinda Hard Problem. The Kinda Hard Problem arises because the grounds on which we base our consciousness attributions to humans third-personally are either unavailable or ambiguous once we move to the animal case. Its nature is that of an epistemic problem: we (...) cannot collect enough evidential grounds to justify attributions of phenomenal consciousness (and attributions of its lack) to non-human animals. Thus, it is impossible to ground rational belief one way or another regarding animal consciousness. This paper presents the problem and explains how it differs from other problems of consciousness. (shrink)

Philosophers of science have offered various accounts of climate model evaluation which have largely centered on model-fit assessment. However, despite the wide-spread prevalence of process-based evaluation in climate science practice, this sort of model evaluation has been undertheorized by philosophers of science. In this paper, I aim to expand this narrow philosophical view of climate model evaluation by providing a philosophical account of process evaluation that is rooted in a close examination of scientific practice. I propose dynamical adequacy as a (...) metric by which scientists test and evaluate models that represent and produce key regional climate processes and features. I argue that process-based evaluation confirms the adequacy of a regional climate model for simulating and predicting future changes of a specific regional climate feature. I offer a case study of how, in practice, scientists establish the reliability of model projections by assessing the dynamical adequacy of such models. I also show how process-based evaluation mitigates some well-known shortcomings of model-fit assessment and supports the adequacy and reliability of climate model projections against philosophical objections, like confirmational holism. Such adequacy is especially important for decision makers who need reliable regional model projections to guide their climate change mitigation and adaptation policies. (shrink)

The concordance of results that are “robust” across multiple scientific modalities is widely considered to play a critical role in the epistemology of science. But what should we make of those cases where such multimodal evidence is discordant? Jacob Stegenga has recently argued that robustness is “worse than useless” in these cases, suggesting that “different kinds of evidence cannot be combined in a coherent way.” In this article I respond to this critique and illustrate the critical methodological role that robustness (...) plays as an aim of scientific inquiry. (shrink)

According to the methodological principle called ‘robustness’, empirical evidence is more reliable when it is generated using multiple, independent (experimental) routes that converge on the same result. As it happens, robustness as a methodological strategy is quite popular amongst philosophers. However, despite its popularity, my goal here is to criticize the value of this principle on historical grounds. My historical reasons take into consideration some recent history of astroparticle physics concerning the search for WIMPs (weakly interacting massive particles), one of (...) the main candidates for cosmic dark matter. On the basis of these reasons, I assert that robustness, at least in this historical case we are considering, has less value than usually assumed by philosophers. (shrink)

: For philosophers of science interested in elucidating the social character of science, an important question concerns the manner in which and degree to which the objectivity of scientific knowledge is socially constituted. We address this broad question by focusing specifically on philosophical theories of evidence. To get at the social character of evidence, we take an interdisciplinary approach informed by categories from argumentation studies. We then test these categories by exploring their applicability to a case study from high-energy physics. (...) Our central claim is that normative philosophy of science must move beyond abstract theories of justification, confirmation, or evidence conceived impersonally and incorporate a theoretical perspective that includes dialogical elements, either as adjuncts to impersonal theories of evidence or as intrinsic to the cogency of scientific argumentation. (shrink)

This article explores Michael Faraday’s “Historical Sketch of Electro‐Magnetism” as a fruitful source for understanding the epistemic significance of experimentation. In this work Faraday provides a catalog of the numerous experimental and theoretical developments in the early history of electromagnetism. He also describes methods that enable experimentalists to dissociate experimental results from the theoretical commitments generating their research. An analysis of the methods articulated in this sketch is instructive for confronting epistemological worries about the theory‐dependence of experimentation. †To contact the (...) author, please write to: 10289 Saint Katherine Lane, Saint Ann, MO 63074; e‐mail: [email protected]. (shrink)

All economic models involve abstractions and idealisations. Economic theory itself does not tell which idealizations are truly fatal or harmful for the result and which are not. This is why much of what is seen as theoretical contribution in economics is constituted by deriving familiar results from different modelling assumptions. If a modelling result is robust with respect to particular modelling assumptions, the empirical falsity of these particular assumptions does not provide grounds for criticizing the result. In this paper we (...) demonstrate how derivational robustness analysis does carry epistemic weight and answer criticism concerning its non-empirical nature and the problematic form of the required independence of the ways of derivation. The epistemic rationale and importance of robustness analysis also challenge some common conceptions of the role of theory in economics. (shrink)

Case Study research is characterized by the employment of multiple data gathering methods. In this paper, I examine the concurrent use of participant observation and qualitative interviews. The question I examine is: what is the rationale behind their combination in case study research? In the literature on case study research, the two most common reasons for using multiple methods appeal to comprehensiveness and convergent confirmation respectively. I argue that there is a third significant, yet overlooked, way to motivate the joint (...) use of participant observation and qualitative interviews: the methods may generate complementary evidence and this puts the researcher in a better position to confirm that her data manifest central epistemic values and so are suitable as basis for providing an adequate answer to her research question. I refer to this as the rationale of blended epistemic value validation. (shrink)

In experimental settings, scientists often “make” new things, in which case the aim is to intervene in order to produce experimental objects and processes—characterized as ‘effects’. In this discussion, I illuminate an important performative function in measurement and experimentation in general: intervention-based experimental production. I argue that even though the goal of IEP is the production of new effects, it can be informative for causal details in scientific representations. Specifically, IEP can be informative about causal relations in: regularities under study; (...) ‘intervention systems’, which are measurement/experimental systems; and new technological systems. (shrink)

This discussion centers on two desiderata: the role of measurement in information-gathering and physical interaction in scientific practice. By taking inspiration from van Fraassen’s view, we present a methodological account of perspectival measurement that addresses empirical practice where there is complex intervention, disagreeing results, and limited theory. The specific aim of our account is to provide a methodological prescription for developing measurement processes in the context of limited theory. The account should be useful to philosophers of science, who are interested (...) in the intersection between representation and intervention; scientists, who are interested in methodological suggestions for theory-development and reliability; and interdisciplinary researchers, who are interested in the intersections between the pandemic, built environments, and social processes. We apply the process-view of measurement to COVID-19, specifically, measuring replication in the SARS-CoV-2 virus. The aim is to show that our account tracks key elements—manipulation/intervention, independence, invariance, and theory-development—by organizing unfolding measurement processes. Additionally, we use our account to make prescriptive suggestions for measurement practice in the COVID-19 context by discussing the need to broaden measurement perspective on interaction, manipulation, and production. (shrink)

Some prominent accounts of scientific evidence treat evidence as an unrelativized concept. But whether belief in a hypothesis is justified seems relative to the epistemic situation of the believer. The issue becomes yet more complicated in the context of group epistemic agents, for then one confronts the problem of relativizing to an epistemic situation that may include conflicting beliefs. As a step toward resolution of these difficulties, an ideal of justification is here proposed that incorporates both an unrelativized evidence requirement (...) and the requirement of the security of the evidence on which a conclusion from data is based. The latter requirement incorporates the consideration of epistemic modal statements. (shrink)

In biomedical measurement, biomarkers are used to achieve reliable prediction of, and useful causal information about patient outcomes while minimizing complexity of measurement, resources, and invasiveness. A biomarker is an assayable metric that discloses the status of a biological process of interest, be it normative, pathophysiological, or in response to intervention. The greatest utility from biomarkers comes from their ability to help clinicians (and researchers) make and evaluate clinical decisions. In this paper we discuss a specific methodological use of clinical (...) biomarkers in pharmacological measurement: Some biomarkers, called ‘surrogate markers’, are used to substitute for a clinically meaningful endpoint corresponding to events and their penultimate risk factors. We confront the reliability of clinical biomarkers that are used to gather information about clinically meaningful endpoints. Our aim is to present a systematic methodology for assessing the reliability of multiple surrogate markers (and biomarkers in general). To do this we draw upon the robustness analysis literature in the philosophy of science and the empirical use of clinical biomarkers. After introducing robustness analysis we present two problems with biomarkers in relation to reliability. Next, we propose an intervention-based robustness methodology for organizing the reliability of biomarkers in general. We propose three relevant conditions for a robust methodology for biomarkers: (R1) Intervention-based demonstration of partial independence of modes: In biomarkers partial independence can be demonstrated through exogenous interventions that modify a process some number of “steps” removed from each of the markers. (R2) Comparison of diverging and converging results across biomarkers: By systematically comparing partially-independent biomarkers we can track under what conditions markers fail to converge in results, and under which conditions they successfully converge. (R3) Information within the context of theory: Through a systematic cross-comparison of the markers we can make causal conclusions as well as eliminate competing theories. We apply our robust methodology to currently developing Alzheimer’s research to show its usefulness for making causal conclusions. (shrink)

Philosophy of science has witnessed substantial recent debate over the existence of a structural analogue of chaos, which is alleged to spell trouble for certain uses of climate models. The debate over the analogy can and should be separated from its alleged epistemic implications: chaos-like behavior is neither necessary nor sufficient for small dynamical misrepresentations to generate erroneous results. The kind of sensitivity that matters in epistemology is one that induces unsafe beliefs, and the existence of a structural analogue to (...) chaos is better seen as an explanation for known safety failures than as providing evidence for unknown ones. (shrink)

To study climate change, scientists employ computer models, which approximate target systems with various levels of skill. Given the imperfection of climate models, how do scientists use simulations to generate knowledge about the causes of observed climate change? Addressing a similar question in the context of biological modelling, Levins (1966) proposed an account grounded in robustness analysis. Recent philosophical discussions dispute the confirmatory power of robustness, raising the question of how the results of computer modelling studies contribute to the body (...) of evidence supporting hypotheses about climate change. Expanding on Staley’s (2004) distinction between evidential strength and security, and Lloyd’s (2015) argument connecting variety-of-evidence inferences and robustness analysis, I address this question with respect to recent challenges to the epistemology robustness analysis. Applying this epistemology to case studies of climate change, I argue that, despite imperfections in climate models, and epistemic constraints on variety-of-evidence reasoning and robustness analysis, this framework accounts for the strength and security of evidence supporting climatological inferences, including the finding that global warming is occurring and its primary causes are anthropogenic. (shrink)

In psychiatry, pharmacological research has played a crucial role in the formulation, revision, and refinement of neurobiological theories of psychopathology. Besides being utilized as potential treatments for various mental disorders, pharmacological drugs play an important epistemic role as experimental instruments that help scientists uncover the neurobiological underpinnings of mental disorders (Tsou, 2012). Interventions with psychiatric patients using pharmacological drugs provide researchers with information about the neurobiological causes of mental disorders that cannot be obtained in other ways. This important source of (...) evidence for the biological causes of mental disorder is often overlooked in philosophical analyses of psychiatry, especially in skeptical analyses that debase the biological aspects of psychopathology (e.g., Szasz, 1960; Scheff, 1963; Laing, 1967). In discussing pharmacological interventions as a form of evidence for the physical basis of mental disorders, this paper aims to clarify the nature, reliability, and limitations of this evidence. In addition, it illustrates the central role that pharmacological findings in applied clinical contexts play in the acquisition of neurobiological knowledge in research contexts. (shrink)

*PSA 2016, symposium on “Data in Time: Epistemology of Historical Data” organized by Sabina Leonelli, 5 November 2016* *See published version: "Radiocarbon Dating in Archaeology: Triangulation and Traceability" in Data Journeys in the Sciences (2020) - link below* Archaeologists put a premium on pressing “legacy data” into service, given the notoriously selective and destructive nature of their practices of data capture. Legacy data consist of material and records that been assembled over decades, sometimes centuries, often by means and for purposes (...) long since discredited or superseded. The primary strategies by which archaeologists put the data to work for new purposes are, I argue, secondary retrieval, recontextualization, and experimental modelling. I focus here on a particularly telling and complex example of secondary retrieval: the extraction of new data from old by means of radiocarbon dating. This is by no means a straightforward process of retrieving physical samples from legacy data to which 14C techniques can be applied that can, on their own, decisively settle chronological questions. When Libby’s post-war radiocarbon revolution got under way, it was expected to establish an absolute chronology that would render obsolete the local and relative chronologies on which archaeologists had long relied. Transformative though it has been, bringing these tools of physical dating to bear on archaeological problems has been a long, tortuous process, now described as proceeding through two subsequent radiocarbon revolutions. The second was an extended process of calibration by which 14C chronologies were corrected and refined, often against the very lines of evidence they were meant to displace. The most recent, a “pragmatic Bayesian” approach to archaeological dating, is motivated by concern that, no matter how much it is refined, radiocarbon dating cannot on its own resolve the chronological problems that archaeologists address; the challenge, its advocates argue, is to ‘fully integrate archaeological information with 14C dating in order to address archaeologically relevant timescales and episodes’. This is a genre of “robustness” reasoning that illustrates its epistemic risks as well as its appeal. As recent philosophical debate makes clear, it depends on appeals to the convergence of independent lines of evidence that may have more rhetorical than epistemic force and that may be spurious. Drawing on this philosophical literature I identify a set of conditions that must be met if these risks are to be avoided, all of which are an explicit focus of debate in cases of contestation about and reconciliation of chronologies based on legacy data. (shrink)

I describe conventions not of correct reasoning but of giving and taking advice about reasoning. This article is asn anticipation of part of the first chapter of my forthcoming *Bounded Thinking*, OUP 2012.

This paper responds to Achinstein’s criticism of the thesis that the only empirical fact that can affect the truth of an objective evidence claim such as ‘e is evidence for h’ (or ‘e confirms h to degree r’) is the truth of e. It shows that cases involving evidential flaws, which form the basis for Achinstein’s objections to the thesis, can satisfactorily be accounted for by appeal to changes in background information and working assumptions. The paper also argues that the (...) a priori and empirical accounts of evidence are on a par when we consider scientific practice, but that a study of artificial intelligence might serve to differentiate them. (shrink)

I contrast two modes of error-elimination relevant to evaluating evidence in accounts that emphasize frequentist reliability. The contrast corresponds to that between the use of of a reliable inference procedure and the critical scrutiny of a procedure with regard to its reliability, in light of what is and is not known about the setting in which the procedure is used. I propose a notion of security as a category of evidential assessment for the latter. In statistical settings, robustness theory and (...) misspecification testing exemplify two distinct strategies for securing statistical inferences. (shrink)

Economists reason causally. Like many other scientists, they aim at formulating justified causal claims about their object of study. This thesis contributes to our understanding of how causal reasoning proceeds in economics. By using the research on the causes of unemployment as a case study, three questions are adressed. What are the meanings of causal claims? How can a causal claim be adequately supported by evidence? How are causal beliefs affected by incoming facts? In the process of answering these semantic, (...) epistemic and dynamical questions, the richness of causal reasoning in economics is brought to the foreground. In the part on semantics, it is argued that the meaning of causal generalizations in policy-oriented economics is best captured by using the concept of ‘inferential relation’ rather than the one of ‘referential relation’. This part contributes to develop an inferentialist semantics by distinguishing among types of inferential relations. In the part on epistemology, it is argued that justification in sciences like economics often relies, and ought to rely, on evidential variety―i.e. the combination of evidence from multiple sources. An interpretation of independence among evidential sources is offered: two evidential sources are fully independent if and only if the reasons why they would be unreliable do not overlap. It is then shown that the variety-of-evidence thesis does not hold under extreme circumstances. The part on dynamics focuses on deviant-case study. A storyline is provided for what deviant-case study aims at and how it proceeds in what is characterized as an ‘eclectic science’. (shrink)

The 2011 Nobel Prize in Physics was awarded to researchers from the Supernova Cosmology Project and the High-z Supernova Search Team for the discovery of the accelerating expansion of the universe. In this paper, I provide a historical analysis of the supernova cosmology evidence put forward by these teams for the accelerating universe, in terms of an iterative model of scientific progress developed by Hasok Chang in the context of his study of the development of measurement standards. I argue, using (...) the key concept of epistemic iteration, that the iterative model adequately accounts for evidence production in experimental science as well. In order to apply Chang’s model to the experimental evidence for the accelerating universe, I introduce the concept of endorsement as a particular mode of progress, and argue that supernova scientists produced an endorsed measurement system to claim evidence for their discovery. Furthermore, I show that the credibility of the evidence was not based on a particular measurement, rather, what proved to be decisive was the “robust consistency” of many individual results. (shrink)

In 1966 the population biologist Richard Levins gave a forceful and in?uential defence of a method called “robustness analysis” (RA). RA is a way of assessing the result of a model by showing that different but related models give the same result. As Levins put it, “our truth is the intersection of independent lies” (1966, 423). Steven Orzack and Elliott Sober (1993) responded with an equally forceful critique of this method, concluding that the idea of robustness “lacks proper de?nition and (...) that its bearing on the question of whether a proposition is true is highly problematic” (533). Replies to Orzack and Sober, from Levins (1993) and Weisberg (2006b), have rejected the idea that RA, on its own, can con?rm the results of models. I argue that these replies have not properly addressed Orzack and Sober’s real criticisms, which focus not on the role of empirical data in RA but on the problem of ensuring that the models used in RA are independent. By drawing on accounts of RA in experimental science, I argue that there is in fact a fallible but viable form of RA that can both con?rm the results of models and incorporate empirical data. However, for reasons other than those of Orzack and Sober, I conclude that RA may be of limited use in model-based science, as the example of ?shery biology can show. (shrink)

This paper analyzes Deborah Mayo's error-statistical (ES) account of scientific evidence in order to clarify the kinds of "material postulates" it requires and to explain how those assumptions function. A secondary aim is to explain and illustrate the importance of the security of an inference. After finding that, on the most straightforward reading of the ES account, it does not succeed in its stated aims, two remedies are considered: either relativize evidence claims or introduce stronger assumptions. The choice between these (...) approaches turns on the value attached to two aims of inquiry that are in tension: drawing strong, informative conclusions and reasoning securely. (shrink)

: Evidence claims depend on fallible assumptions. Three strategies for making true evidence claims in spite of this fallibility are strengthening the support for those assumptions, weakening conclusions, and using multiple independent tests to produce robust evidence. Reliability itself, understood in frequentist terms, does not explain the usefulness of all three strategies; robustness, in particular, sometimes functions in a way that is not well-characterized in terms of reliability. I argue that, in addition to reliability, the security of evidence claims is (...) of epistemic value, where an evidence claim is secure relative to an epistemic situation if it remains true in all scenarios that are epistemically possible relative to that epistemic situation. (shrink)