The use of multiple means of determination to “triangulate” on the existence and character of a common phenomenon, object, or result has had a long tradition in science but has seldom been a matter of primary focus. As with many traditions, it is traceable to Aristotle, who valued having multiple explanations of a phenomenon, and it may also be involved in his distinction between special objects of sense and common sensibles. It is implicit though not emphasized in the distinction between (...) primary and secondary qualities from Galileo onward. It is arguably one of several conceptions involved in Whewell’s method of the “consilience of inductions” (Laudan 1971) and is to be found in several places in Peirce. (From M. Brewer and B. Collins, eds., (1981); Scientific Inquiry in the Social Sciences (a festschrift for Donald T. Campbell), San Francisco: Jossey-Bass, pp. 123–162.). (shrink)

Since Economics survey by Windrum et al., research on empirical validation of agent-based Agent-based model in Economics has made substantial advances, thanks to a constant flow of high-quality contributions. This Chapter attempts to take stock of such recent literature to offer an updated critical review of the existing validation techniques. We sketch a simple theoretical framework that conceptualizes existing validation approaches, which we examine along three different dimensions: Comparison between artificial and real-world Data; Calibration and estimation of model parameters; and (...) parameter space Exploration. Finally, we discuss open issues in the field of ABM validation and Estimation. In particular, we argue that more research efforts should be devoted toward advancing HypothesisTesting in ABM, with specific emphasis on model stationarity and ergodicity. (shrink)

Scientific theories are used for a variety of purposes. For example, physical theories such as classical mechanics and electrodynamics have important applications in engineering and technology, and we trust that this results in useful machines, stable bridges, and the like. Similarly, theories such as quantum mechanics and relativity theory have many applications as well. Beyond that, these theories provide us with an understanding of the world and address fundamental questions about space, time, and matter. Here we trust that the answers (...) scientific theories give are reliable and that we have good reason to believe that the features of the world are similar to what the theories say about them. But why do we trust scientific theories, and what counts as evidence in favor of them? (shrink)

The replication crisis has caused researchers to distinguish between exact replications, which duplicate all aspects of a study that could potentially affect the results, and direct replications, which duplicate only those aspects of the study that are thought to be theoretically essential to reproduce the original effect. The replication crisis has also prompted researchers to think more carefully about the possibility of making Type I errors when rejecting null hypotheses. In this context, the present article considers the utility of two (...) types of Type I error probability: the Neyman–Pearson long run Type I error rate and the Fisherian sample-specific Type I error probability. It is argued that the Neyman–Pearson Type I error rate is inapplicable in social science because it refers to a long run of exact replications, and social science deals with irreversible units that make exact replications impossible. Instead, the Fisherian sample-specific Type I error probability is recommended as a more meaningful way to conceptualize false positive results in social science because it can be applied to each sample-specific decision about rejecting the same substantive null hypothesis in a series of direct replications. It is concluded that the replication crisis may be partly due to researchers’ unrealistic expectations about replicability based on their consideration of the Neyman–Pearson Type I error rate across a long run of exact replications. (shrink)

When scientists seek further confirmation of their results, they often attempt to duplicate the results using diverse means. To the extent that they are successful in doing so, their results are said to be robust. This paper investigates the logic of such "robustness analysis" [RA]. The most important and challenging question an account of RA can answer is what sense of evidential diversity is involved in RAs. I argue that prevailing formal explications of such diversity are unsatisfactory. I propose a (...) unified, explanatory account of diversity in RAs. The resulting account is, I argue, truer to actual cases of RA in science; moreover, this account affords us a helpful, new foothold on the logic undergirding RAs. (shrink)

In robustness analysis, hypotheses are supported to the extent that a result proves robust, and a result is robust to the extent that we detect it in diverse ways. But what precise sense of diversity is at work here? In this paper, I show that the formal explications of evidential diversity most often appealed to in work on robustness – which all draw in one way or another on probabilistic independence – fail to shed light on the notion of diversity (...) relevant to robustness analysis. I close by briefly outlining a promising alternative approach inspired by Horwich’s (1982) eliminative account of evidential diversity. (shrink)

I propose a distinct type of robustness, which I suggest can support a confirmatory role in scientific reasoning, contrary to the usual philosophical claims. In model robustness, repeated production of the empirically successful model prediction or retrodiction against a background of independentlysupported and varying model constructions, within a group of models containing a shared causal factor, may suggest how confident we can be in the causal factor and predictions/retrodictions, especially once supported by a variety of evidence framework. I present climate (...) models of greenhouse gas global warming of the 20th Century as an example, and emphasize climate scientists’ discussions of robust models and causal aspects. The account is intended as applicable to a broad array of sciences that use complex modeling techniques. (shrink)

Philosophers of science have developed several accounts of how consideration of scientific models can prompt learning about real-world targets. In recent years, various authors advocated the thesis that consideration of so-called minimal models can prompt learning about such targets. In this paper, I draw on the philosophical literature on scientific modelling and on widely cited illustrations from economics and biology to argue that this thesis fails to withstand scrutiny. More specifically, I criticize leading proponents of such thesis for failing to (...) explicate in virtue of what properties or features minimal models supposedly prompt learning and for substantially overstating the epistemic import of minimal models. I then examine and refute several arguments one may put forward to demonstrate that consideration of minimal models can prompt learning about real-world targets. In doing so, I illustrate the implications of my critique for the wider debate on the epistemology of scientific modelling. (shrink)

Economies are complicated systems encompassing micro behaviors, interaction patterns, and global regularities. Whether partial or general in scope, studies of economic systems must consider how to handle difficult real-world aspects such as asymmetric information, imperfect competition, strategic interaction, collective learning, and the possibility of multiple equilibria. Recent advances in analytical and computational tools are permitting new approaches to the quantitative study of these aspects. One such approach is Agent-based Computational Economics (ACE), the computational study of economic processes modeled as dynamic (...) systems of interacting agents. This chapter explores the potential advantages and disadvantages of ACE for the study of economic systems. General points are concretely illustrated using an ACE model of a two-sector decentralized market economy. Six issues are highlighted: Constructive understanding of production, pricing, and trade processes; the essential primacy of survival; strategic rivalry and market power; behavioral uncertainty and learning; the role of conventions and organizations; and the complex interactions among structural attributes, institutional arrangements, and behavioral dispositions. (shrink)

Odenbaugh and Alexandrova provide a challenging critique of the epistemic benefits of robustness analysis, singling out for particular criticism the account we articulated in Kuorikoski et al.. Odenbaugh and Alexandrova offer two arguments against the confirmatory value of robustness analysis: robust theorems cannot specify causal mechanisms and models are rarely independent in the way required by robustness analysis. We address Odenbaugh and Alexandrova’s criticisms in order to clarify some of our original arguments and to shed further light on the properties (...) of robustness analysis and its epistemic rationale. (shrink)

Coherentism maintains that coherent beliefs are more likely to be true than incoherent beliefs, and that coherent evidence provides more confirmation of a hypothesis when the evidence is made coherent by the explanation provided by that hypothesis. Although probabilistic models of credence ought to be well-suited to justifying such claims, negative results from Bayesian epistemology have suggested otherwise. In this essay we argue that the connection between coherence and confirmation should be understood as a relation mediated by the causal relationships (...) among the evidence and a hypothesis, and we offer a framework for doing so by fitting together probabilistic models of coherence, confirmation, and causation. We show that the causal structure among the evidence and hypothesis is sometimes enough to determine whether the coherence of the evidence boosts confirmation of the hypothesis, makes no difference to it, or even reduces it. We also show that, ceteris paribus, it is not the coherence of the evidence that boosts confirmation, but rather the ratio of the coherence of the evidence to the coherence of the evidence conditional on a hypothesis. (shrink)

Theoretical biology and economics are remarkably similar in their reliance on mathematical models, which attempt to represent real world systems using many idealized assumptions. They are also similar in placing a great emphasis on derivational robustness of modeling results. Recently philosophers of biology and economics have argued that robustness analysis can be a method for confirmation of claims about causal mechanisms, despite the significant reliance of these models on patently false assumptions. We argue that the power of robustness analysis has (...) been greatly exaggerated. It is best regarded as a method of discovery rather than confirmation. (shrink)

APA PsycNET abstract: This is the first volume of a two-volume work on Probability and Induction. Because the writer holds that probability logic is identical with inductive logic, this work is devoted to philosophical problems concerning the nature of probability and inductive reasoning. The author rejects a statistical frequency basis for probability in favor of a logical relation between two statements or propositions. Probability "is the degree of confirmation of a hypothesis (or conclusion) on the basis of some given evidence (...) (or premises)." Furthermore, all principles and theorems of inductive logic are analytic, and the entire system is to be constructed by means of symbolic logic and semantic methods. This means that the author confines himself to the formalistic procedures of word and symbol systems. The resulting sentence or language structures are presumed to separate off logic from all subjectivist or psychological elements. Despite the abstractionism, the claim is made that if an inductive probability system of logic can be constructed it will have its practical application in mathematical statistics, and in various sciences. 16-page bibliography. (PsycINFO Database Record (c) 2016 APA, all rights reserved). (shrink)

Probabilistic models have much to offer to philosophy. We continually receive information from a variety of sources: from our senses, from witnesses, from scientific instruments. When considering whether we should believe this information, we assess whether the sources are independent, how reliable they are, and how plausible and coherent the information is. Bovens and Hartmann provide a systematic Bayesian account of these features of reasoning. Simple Bayesian Networks allow us to model alternative assumptions about the nature of the information sources. (...) Measurement of the coherence of information is a controversial matter: arguably, the more coherent a set of information is, the more confident we may be that its content is true, other things being equal. The authors offer a new treatment of coherence which respects this claim and shows its relevance to scientific theory choice. Bovens and Hartmann apply this methodology to a wide range of much discussed issues regarding evidence, testimony, scientific theories, and voting. Bayesian Epistemology is an essential tool for anyone working on probabilistic methods in philosophy, and has broad implications for many other disciplines. (shrink)

Modelers often rely on robustness analysis, the search for predictions common to several independent models. Robustness analysis has been characterized and championed by Richard Levins and William Wimsatt, who see it as central to modern theoretical practice. The practice has also been severely criticized by Steven Orzack and Elliott Sober, who claim that it is a nonempirical form of confirmation, effective only under unusual circumstances. This paper addresses Orzack and Sober's criticisms by giving a new account of robustness analysis and (...) showing how the practice can identify robust theorems. Once the structure of robust theorems is clearly articulated, it can be shown that such theorems have a degree of confirmation, despite the lack of direct empirical evidence for their truth. (shrink)

This book explores the Bayesian approach to the logic and epistemology of scientific reasoning. Section 1 introduces the probability calculus as an appealing generalization of classical logic for uncertain reasoning. Section 2 explores some of the vast terrain of Bayesian epistemology. Three epistemological postulates suggested by Thomas Bayes in his seminal work guide the exploration. This section discusses modern developments and defenses of these postulates as well as some important criticisms and complications that lie in wait for the Bayesian epistemologist. (...) Section 3 applies the formal tools and principles of the first two sections to a handful of topics in the epistemology of scientific reasoning: confirmation, explanatory reasoning, evidential diversity and robustness analysis, hypothesis competition, and Ockham's Razor. (shrink)

Diversity of evidence is widely claimed to be crucial for evidence amalgamation to have distinctive epistemic merits. Bayesian epistemologists capture this idea in the variety-of-evidence thesis: ceteris paribus, the strength of confirmation of a hypothesis by an evidential set increases with the diversity of the evidential elements in that set. Yet, formal exploration of this thesis has shown that it fails to be generally true. This article demonstrates that the thesis fails in even more circumstances than recent results would lead (...) us to expect. Most importantly, it can fail whatever the chance that the evidential sources are unreliable. Our results hold for two types of degrees of variety: reliability independence and testable aspect independence. We conclude that the variety-of-evidence thesis can, at best, be interpreted as an exception-prone rule of thumb. (shrink)

Robustness arguments hold that hypotheses are more likely to be true when they are confirmed by diverse kinds of evidence. Robustness arguments require the confirming evidence to be independent. We identify two kinds of independence appealed to in robustness arguments: ontic independence —when the multiple lines of evidence depend on different materials, assumptions, or theories—and probabilistic independence. Many assume that OI is sufficient for a robustness argument to be warranted. However, we argue that, as typically construed, OI is not a (...) sufficient independence condition for warranting robustness arguments. We show that OI evidence can collectively confirm a hypothesis to a lower degree than individual lines of evidence, contrary to the standard assumption undergirding usual robustness arguments. We employ Bayesian networks to represent the ideal empirical scenario for a robustness argument and a variety of ways in which empirical scenarios can fall short of this ideal. (shrink)

Derivational robustness may increase the degree to which various pieces of evidence indirectly confirm a robust result. There are two ways in which this increase may come about. First, if one can show that a result is robust, and that the various individual models used to derive it also have other confirmed results, these other results may indirectly confirm the robust result. Confirmation derives from the fact that data not known to bear on a result are shown to be relevant (...) when it is shown to be robust. Second, robustness may increase the degree to which the robust result is indirectly confirmed if it increases the weight with which existing evidence indirectly confirms it. This may happen when it strengthens the connection between the core and the robust result by showing that auxiliaries are not responsible for the result. (shrink)

What can we learn from “minimal” economic models? I argue that learning from such models is not limited to conceptual explorations—which show how something could be the case—but may extend to explanations of real economic phenomena—which show how something is the case. A model may be minimal qua certain world-linking properties, and yet “not-so-minimal” qua learning, provided it is externally valid. This, in turn, depends on using the right principles for model building and not necessarily “isolating” principles. My argument is (...) buttressed by a case study from computational economics, namely, two agent-based models of asset pricing. (shrink)

The variety-of-evidence thesis has been criticized by Bovens and Hartmann. This article points to two limitations of their Bayesian model: the conceptualization of unreliable evidential sources as randomizing and the restriction to comparing full independence to full dependence. It is shown that the variety-of-evidence thesis is rehabilitated when unreliable sources are reconceptualized as systematically biased. However, it turns out that allowing for degrees of independence leads to a qualification of the variety-of-evidence thesis: as Bovens and Hartmann claimed, more independence does (...) not always imply stronger confirmation. (shrink)

The variety of evidence thesis in confirmation theory states that more varied supporting evidence confirms a hypothesis to a greater degree than less varied evidence. Under a very plausible interpretation of this thesis, positive test results from multiple independent instruments confirm a hypothesis to a greater degree than positive test results from a single instrument. We invoke Bayesian Networks to model confirmation on grounds of evidence that is obtained from less than fully reliable instruments and show that the variety of (...) evidence thesis is not sacrosanct when testing is conducted with less than fully reliable instruments: under certain conditions, a hypothesis receives more confirmation from evidence that is obtained from one rather than from more independent instruments. In the appendix, we prove certain convergence results for large numbers of positive test results from single versus multiple less than fully reliable instruments. (shrink)

The replicability crisis refers to the apparent failures to replicate both important and typical positive experimental claims in psychological science and biomedicine, failures which have gained increasing attention in the past decade. In order to provide evidence that there is a replicability crisis in the first place, scientists have developed various measures of replication that help quantify or “count” whether one study replicates another. In this nontechnical essay, I critically examine five types of replication measures used in the landmark article (...) “Estimating the reproducibility of psychological science” (Open Science Collaboration, Science, 349, ac4716, 2015) based on the following techniques: subjective assessment, null hypothesis significance testing, comparing effect sizes, comparing the original effect size with the replication confidence interval, and meta-analysis. The first four, I argue, remain unsatisfactory for a variety of conceptual or formal reasons, even taking into account various improvements. By contrast, at least one version of the meta-analytic measure does not suffer from these problems. It differs from the others in rejecting dichotomous conclusions, the assumption that one study replicates another or not simpliciter. I defend it from other recent criticisms, concluding however that it is not a panacea for all the multifarious problems that the crisis has highlighted. (shrink)

The intuitive Variety of Evidence Thesis states that, ceteris paribus, more varied evidence for a hypothesis confirms it more strongly than less varied evidence. Recent Bayesian analyses have raised serious doubts in its validity. Claveau suggests the existence of a novel type of counter-example to this thesis: a gradual increase in source independence can lead to a decrease in hypothesis confirmation. I show that Claveau’s measure of gradual source independence suffers from two unsuspected types of inconsistencies. I hence put forward (...) a more natural measure of gradual source independence which is not plagued by inconsistencies. Claveau’s counter-examples to the variety of evidence thesis disappear with the measure I suggest. I hence argue that my measure is preferable and that this thesis does at least not seem to be troubled by Claveau’s arguments. (shrink)

Varied evidence for a hypothesis confirms it more strongly than less varied evidence, ceteris paribus. This epistemological Variety of Evidence Thesis enjoys long-standing widespread intuitive support. Recent literature has raised serious doubts that the correlational approach of explicating the thesis can vindicate it. By contrast, the eliminative approach due to Horwich vindicates the Variety of Evidence Thesis but only within a relatively narrow domain. I investigate the prospects of extending the eliminative approach to a larger domain by considering a larger (...) class of sensible explications of evidential variety. For a large subclass class of such explications I show how to construct cases in which the less diverse body of evidence for a hypothesis confirms more strongly. I hence argue that these prospects are dire since the eliminative approach widely fails to vindicate the thesis. (shrink)

Varied evidence confirms more strongly than less varied evidence, ceteris paribus. This epistemological Variety of Evidence Thesis enjoys widespread intuitive support. We put forward a novel explication of one notion of varied evidence and the Variety of Evidence Thesis within Bayesian models of scientific inference by appealing to measures of entropy. Our explication of the Variety of Evidence Thesis holds in many of our models which also pronounce on disconfirmatory and discordant evidence. We argue that our models pronounce rightly. Against (...) a backdrop of failures of the Variety of Evidence Thesis, the intuitive case for the Variety of Evidence Thesis emerges strengthened. Our models do however not support the general case for the thesis since our explication of it fails to hold in certain cases. The parameter space of this failure is explored and an explanation for the failure is offered. (shrink)

In the philosophy of science and epistemology literature, robustness analysis has become an umbrella term that refers to a variety of strategies. One of the main purposes of this paper is to argue that different strategies rely on different criteria for justifications. More specifically, I will claim that: i) robustness analysis differs from de-idealization even though the two concepts have often been conflated in the literature; ii) the comparison of different model frameworks requires different justifications than the comparison of models (...) that differ only for the assumption under test; iii) the replacement of specific assumptions with different ones can encounter specific difficulties in scientific practice. These claims will be supported by a case study in population ecology and a case study in geographical economics. (shrink)

This paper proposes an extended version of the interventionist account for causal inference in the practical context of biological mechanism research. This paper studies the details of biological mechanism researchers’ practices of assessing the evidential legitimacy of experimental data, arguing why quantity and variety are two important criteria for this assessment. Because of the nature of biological mechanism research, the epistemic values of these two criteria result from the independence both between the causation of data generation and the causation in (...) question and between different interventions, not techniques. The former independence ensures that the interventions in the causation in question are not affected by the causation that is responsible for data generation. The latter independence ensures the reliability of the final mechanisms not only in the empirical but also the formal aspects. This paper first explores how the researchers use quantity to check the effectiveness of interventions, where they at the same time determine the validity of the difference-making revealed by the results of interventions. Then, this paper draws a distinction between experimental interventions and experimental techniques, so that the reliability of mechanisms, as supported by the variety of evidence, can be safely ensured in the probabilistic sense. The latter process is where the researchers establish evidence of the mechanisms connecting the events of interest. By using case studies, this paper proposes to use ‘intervention’ as the fruitful connecting point of literature between evidence and mechanisms. (shrink)

The paper introduces, compares and contrasts formal models of source reliability proposed in the epistemology literature, in particular the prominent models of Bovens and Hartmann and Olsson :127–143, 2011). All are Bayesian models seeking to provide normative guidance, yet they differ subtly in assumptions and resulting behavior. Models are evaluated both on conceptual grounds and through simulations, and the relationship between models is clarified. The simulations both show surprising similarities and highlight relevant differences between these models. Most importantly, however, our (...) evaluations reveal that important normative concerns arguably remain unresolved. The philosophical implications of this for testimony are discussed. (shrink)

This book offers a comprehensive overview of the structure, strategy and methods of assessment of orthodox theoretical economics. In Part I Professor Hausman explains how economists theorise, emphasising the essential underlying commitment of economists to a vision of economics as a separate science. In Part II he defends the view that the basic axioms of economics are 'inexact' since they deal only with the 'major' causes; unlike most writers on economic methodology, the author argues that it is the rules that (...) economists espouse rather than their practice that is at fault. Part III links the conception of economics as a separate science to the fact that economic theories offer reasons and justifications for human actions, not just their causes. With its lengthy appendix introducing relevant issues in philosophy of science, this book is a major addition to philosophy of economics and of social science. (shrink)

This paper argues that even when simple analogue models picture parallel worlds, they generally still serve as isolating tools. But there are serious obstacles that often stop them isolating in just the right way. These are obstacles that face any model that functions as a thought-experiment but they are especially pressing for economic models because of the paucity of economic principles. Because of the paucity of basic principles, economic models are rich in structural assumptions. Without these no interesting conclusions can (...) be drawn. This, however, makes trouble when it comes to exporting conclusions from the model to the world. One uncontroversial constraint on induction from special cases is to beware of extending conclusions to situations that we know are different in relevant respects. In the case of economic models it is clear by inspection that the unrealistic structural assumptions of the model are intensely relevant to the conclusion. Any inductive leap to a real situation seems a bad bet. (shrink)

This paper examines two recent approaches to the nature and functioning of economic models: models as isolating representations and models as credible constructions. The isolationist view conceives of economic models as surrogate systems that isolate some of the causal mechanisms or tendencies of their respective target systems, while the constructionist approach treats them rather like pure constructions or fictional entities that nevertheless license different kinds of inferences. I will argue that whereas the isolationist view is still tied to the representationalist (...) understanding of models that takes the model-target dyad as the basic unit of analysis, the constructionist perspective can better accommodate the way we actually acquire knowledge through them. Using the example of Tobin’s ultra-Keynesian model I will show how many of the epistemic characteristics of modelling tend to go unrecognised if too much focus is placed on the model-target dyad. (shrink)

We claim that the process of theoretical model refinement in economics is best characterised as robustness analysis: the systematic examination of the robustness of modelling results with respect to particular modelling assumptions. We argue that this practise has epistemic value by extending William Wimsatt's account of robustness analysis as triangulation via independent means of determination. For economists robustness analysis is a crucial methodological strategy because their models are often based on idealisations and abstractions, and it is usually difficult to tell (...) which idealisations are truly harmful. (shrink)

Can computer simulation results be evidence for hypotheses about real-world systems and phenomena? If so, what sort of evidence? Can we gain genuinely new knowledge of the world via simulation? I argue that evidence from computer simulation is aptly characterized as higher-order evidence: it is evidence that other evidence regarding a hypothesis about the world has been collected. Insofar as particular epistemic agents do not have this other evidence, it is possible that they will gain genuinely new knowledge of the (...) world via simulation. I illustrate with examples inspired by uses of simulation in meteorology and astrophysics. (shrink)

It is often claimed that artificial society simulations contribute to the explanation of social phenomena. At the hand of a particular example, this paper argues that artificial societies often cannot provide full explanations, because their models are not or cannot be validated. Despite that, many feel that such simulations somehow contribute to our understanding. This paper tries to clarify this intuition by investigating whether artificial societies provide potential explanations. It is shown that these potential explanations, if they contribute to our (...) understanding, considerably differ from potential causal explanations. Instead of possible causal histories, simulations offer possible functional analyses of the explanandum . The paper discusses how these two kinds explanatory strategies differ, and how potential functional explanations can be appraised. (shrink)

We appeal to the theory of Bayesian Networks to model different strategies for obtaining confirmation for a hypothesis from experimental test results provided by less than fully reliable instruments. In particular, we consider (i) repeated measurements of a single test consequence of the hypothesis, (ii) measurements of multiple test consequences of the hypothesis, (iii) theoretical support for the reliability of the instrument, and (iv) calibration procedures. We evaluate these strategies on their relative merits under idealized conditions and show some surprising (...) repercussions on the variety-of-evidence thesis and the Duhem-Quine thesis. (shrink)

Diversity of evidence is widely claimed to be crucial for evidence amalgamation to have distinctive epistemic merits. Bayesian epistemologists capture this idea in the variety-of-evidence thesis: ceteris paribus, the strength of confirmation of a hypothesis by an evidential set increases with the diversity of the evidential elements in that set. Yet, formal exploration of this thesis has shown that it fails to be generally true. This article demonstrates that the thesis fails in even more circumstances than recent results would lead (...) us to expect. Most importantly, it can fail whatever the chance that the evidential sources are unreliable. Our results hold for two types of degrees of variety: reliability independence and testable aspect independence. We conclude that the variety-of-evidence thesis can, at best, be interpreted as an exception-prone rule of thumb. (shrink)

Robustness may increase the degree to which the robust result is indirectly confirmed if it is shown to depend on confirmed rather than disconfirmed assumptions. Although increasing the weight with which existing evidence indirectly confirms it in such a case, robustness may also be irrelevant for confirmation, or may even disconfirm. Whether or not it confirms depends on the available data and on what other results have already been established.

We argue that the robustness analysis of idealized models can have confirmational power. This responds to concerns recently raised in the literature, according to which the robustness analysis of models whose idealizations are not discharged is unable to confirm the causal mechanisms underlying these models, and the robustness analysis of models whose idealizations are discharged is unnecessary. In response, we make clear that, where idealizations sweep out, in a specific way, the space of possibilities— which is sometimes, though not always, (...) the case—they can be holistically discharged. In turn, this can be used to show that the robustness analysis of idealized models can have confirmational force after all. (shrink)

This paper defends the need for evidential diversity and the mix of methods that that can in train require. The focus is on causal claims, especially ‘singular’ claims about the effects of causes in a specific setting—either what will happen or what has happened. I do so by offering a template that categorises kinds of evidence that can support these claims. The catalogue is generated by considering what needs to happen for a causal process to carry through from putative cause (...) at the start to the targeted effect at the end. The usual call for mixed methods focusses on a single overall claim and argues that we increase certainty by the use of different methods with compensating strengths and weaknesses. My proposals instead focus on the evidence that supports the great many subsidiary claims that must hold if the overall one is to be true. As is typical for singular causal claims, the mix of methods that will generally be required to collect the kinds of evidence I urge will usually have little claim to the kind of rigour that is now widely demanded in evidencing causal claims, especially those for policy/treatment effectiveness. So I begin with an exploration of what seems to be intended by ‘rigour’ in such discussions, since it is seldom made clear just what makes the favoured methods especially rigorous. I then argue that the emphasis on rigour can be counterproductive. Rigour is often the enemy of evidential diversity, and evidential diversity—lots of it—can make for big improvements in the reliability of singular causal predictions and post hoc evaluations. I illustrate with the paragon of rigour for causal claims, randomised controlled trials (RCTs), rehearsing at some length what they can and cannot do to make it easier to assess the importance of rigour in warranting singular causal claims. (shrink)

This article contributes to the revision of the procedure of robustness analysis of mathematical models in epistemic democracy using the systematic review method. It identifies the drawbacks of robustness analysis in epistemic democracy in terms of sample universality and inference from samples with the same results. To exemplify the effectiveness of systematic review, this article conducted a pilot review of diversity trumps ability theorem models, which are mathematical models of deliberation often cited by epistemic democrats. A review of nine models (...) extracted from 352 papers exemplifies the effectiveness of robustness analysis supplemented by systematic review in epistemic democracy. (shrink)

Addresses ‘too-odd-not-to-be-true’ reasoning in the assessment of testimony. This is the curious phenomenon that an initially less plausible report from multiple independent witnesses may elicit more confidence than an initially more plausible report.

It is argued that one can learn from minimal economic models. Minimal models are models that are not similar to the real world, do not resemble some of its features, and do not adhere to accepted regularities. One learns from a model if constructing and analysing the model affects one’s confidence in hypotheses about the world. Economic models, I argue, are often assessed for their credibility. If a model is judged credible, it is considered to be a relevant possibility. Considering (...) such relevant possibilities may affect one’s confidence in necessity or impossibility hypotheses. Thus, one can learn from minimal economic models. (shrink)