The variety-of-evidence thesis has been criticized by Bovens and Hartmann. This article points to two limitations of their Bayesian model: the conceptualization of unreliable evidential sources as randomizing and the restriction to comparing full independence to full dependence. It is shown that the variety-of-evidence thesis is rehabilitated when unreliable sources are reconceptualized as systematically biased. However, it turns out that allowing for degrees of independence leads to a qualification of the variety-of-evidence thesis: as Bovens and Hartmann claimed, more independence does (...) not always imply stronger confirmation. (shrink)

In this paper, we identify a new and mathematically well-defined sense in which the coherence of a set of hypotheses can be truth-conducive. Our focus is not, as usual, on the probability but on the confirmation of a coherent set and its members. We show that, if evidence confirms a hypothesis, confirmation is “transmitted” to any hypotheses that are sufficiently coherent with the former hypothesis, according to some appropriate probabilistic coherence measure such as Olsson’s or Fitelson’s measure. Our findings have (...) implications for scientific methodology, as they provide a formal rationale for the method of indirect confirmation and the method of confirming theories by confirming their parts. (shrink)

On analogy with testimony, I define a notion of a scientific theory’s lacking or having candor, in a testing situation, according to whether the theory under test is probabilistically relevant to the processes in the test procedures, and thereby to the reliability of test outcomes. I argue that this property identifies what is distinctive about those theories that Karl Popper denounced as exhibiting “reinforced dogmatism” through their self-protective behavior (e.g., psychoanalysis, Hegelianism, Marxism). I explore whether lack of candor interferes with (...) the testing of theories, and conclude that (1) our default attitude toward theories that lack candor in a given test should be suspicion, but (2) the circumstance that a theory lacks candor in a testing situation does not preclude obtaining independent evidence for the auxiliary assumptions to which the theory is probabilistically relevant, and thereby eliminating the problem that lack of candor creates. Thus, Popper was right to think that lack of candor is a bad thing, but wrong to conclude that candor is a criterion of the scientificity of a theory. Seeing this requires recognition of some differences between intuitive relevance and probabilistic relevance, and proper appreciation of the notion of screening off and of the fact that probabilistic relevance is not transitive. (shrink)

We introduce a framework that can be used to model both mathematics and human reasoning about mathematics. This framework involves stochastic mathematical systems (SMSs), which are stochastic processes that generate pairs of questions and associated answers (with no explicit referents). We use the SMS framework to define normative conditions for mathematical reasoning, by defining a “calibration” relation between a pair of SMSs. The first SMS is the human reasoner, and the second is an “oracle” SMS that can be interpreted as (...) deciding whether the question–answer pairs of the reasoner SMS are valid. To ground thinking, we understand the answers to questions given by this oracle to be the answers that would be given by an SMS representing the entire mathematical community in the infinite long run of the process of asking and answering questions. We then introduce a slight extension of SMSs to allow us to model both the physical universe and human reasoning about the physical universe. We then define a slightly different calibration relation appropriate for the case of scientific reasoning. In this case the first SMS represents a human scientist predicting the outcome of future experiments, while the second SMS represents the physical universe in which the scientist is embedded, with the question–answer pairs of that SMS being specifications of the experiments that will occur and the outcome of those experiments, respectively. Next we derive conditions justifying two important patterns of inference in both mathematical and scientific reasoning: (i) the practice of increasing one’s degree of belief in a claim as one observes increasingly many lines of evidence for that claim, and (ii) abduction, the practice of inferring a claim’s probability of being correct from its explanatory power with respect to some other claim that is already taken to hold for independent reasons. (shrink)

The stopping rule for a sequential experiment is the rule or procedure for determining when that experiment should end. Accordingly, the stopping rule principle (SRP) states that the evidential relationship between the final data from a sequential experiment and a hypothesis under consideration does not depend on the stopping rule: the same data should yield the same evidence, regardless of which stopping rule was used. I clarify and provide a novel defense of two interpretations of the main argument against the (...) SRP, the foregone conclusion argument. According to the first, the SRP allows for highly confirmationally unreliable experiments, which concept I make precise, to confirm highly. According to the second, it entails the evidential equivalence of experiments differing significantly in their confirmational reliability. I rebut several attempts to deflate or deflect the foregone conclusion argument, drawing connections with replication in science and the likelihood principle. (shrink)

Philosophical Issues, Volume 31, Issue 1, Page 41-65, October 2021.

We model scientific theories as Bayesian networks. Nodes carry credences and function as abstract representations of propositions within the structure. Directed links carry conditional probabilities and represent connections between those propositions. Updating is Bayesian across the network as a whole. The impact of evidence at one point within a scientific theory can have a very different impact on the network than does evidence of the same strength at a different point. A Bayesian model allows us to envisage and analyze the (...) differential impact of evidence and credence change at different points within a single network and across different theoretical structures. (shrink)

There is an ongoing methodological debate in philosophy of science concerning the use of case studies as evidence for and/or against theories about science. In this paper, I aim to make a contribution to this debate by taking an empirical approach. I present the results of a systematic survey of the PhilSci-Archive, which suggest that a sizeable proportion of papers in philosophy of science contain appeals to case studies, as indicated by the occurrence of the indicator words “case study” and/or (...) “case studies.” These results are confirmed by data mined from the JSTOR database on research articles published in leading journals in the field: Philosophy of Science, the British Journal for the Philosophy of Science (BJPS), and the Journal for General Philosophy of Science (JGPS), as well as the Proceedings of the Biennial Meeting of the Philosophy of Science Association (PSA). The data also show upward trends in appeals to case studies in articles published in Philosophy of Science, the BJPS, and the JGPS. The empirical work I have done for this paper provides philosophers of science who are wary of the use of case studies as evidence for and/or against theories about science with a way to do philosophy of science that is informed by data rather than case studies. (shrink)

Amalgamating evidence from heterogeneous sources and across levels of inquiry is becoming increasingly important in many pure and applied sciences. This special issue provides a forum for researchers from diverse scientific and philosophical perspectives to discuss evidence amalgamation, its methodologies, its history, its pitfalls, and its potential. We situate the contributions therein within six themes from the broad literature on this subject: the variety-of-evidence thesis, the philosophy of meta-analysis, the role of robustness/sensitivity analysis for evidence amalgamation, its bearing on questions (...) of extrapolation and external validity of experiments, its connection with theory development, and its interface with causal inference, especially regarding causal theories of cancer. (shrink)

In this paper I criticize the current standards for the acceptability of expert testimony in current US legislation. The standards have been the subject of much academic literature after the Frye and Daubert cases. I expose what I call the Paradox of Proof, and argue that the historical and current standards have sidestepped the problem of determining who is an expert and who is not in a court of law. I then investigate the problem of recognizing expertise from the layperson’s (...) standpoint, and suggest what courses of action the future research ought to take on the problem of identifying expertise. (shrink)

A pervasive feature of the sciences, particularly the applied sciences, is an experimental focus on a few (often only one) possible causal connections. At the same time, scientists often advance and apply relatively broad models that incorporate many different causal mechanisms. We are naturally led to ask whether there are normative rules for integrating multiple local experimental conclusions into models covering many additional variables. In this paper, we provide a positive answer to this question by developing several inference rules that (...) use local causal models to place constraints on the integrated model, given quite general assumptions. We also demonstrate the practical value of these rules by applying them to a case study from ecology. Experimental scope in applied sciences Fusing the results of experiments A concrete example of the inference rules Application to a case study. (shrink)

Corroborating Testimony, Probability and Surprise’, Erik J. Olsson ascribes to L. Jonathan Cohen the claims that if two witnesses provide us with the same information, then the less probable the information is, the more confident we may be that the information is true (C), and the stronger the information is corroborated (C*). We question whether Cohen intends anything like claims (C) and (C*). Furthermore, he discusses the concurrence of witness reports within a context of independent witnesses, whereas the witnesses in (...) Olsson's model are not independent in the standard sense. We argue that there is much more than, in Olsson's words, ‘a grain of truth’ to claim (C), both on his own characterization as well as on Cohen's characterization of the witnesses. We present an analysis for independent witnesses in the contexts of decision-making under risk and decision-making under uncertainty and generalize the model for n witnesses. As to claim (C*), Olsson's argument is contingent on the choice of a particular measure of corroboration and is not robust in the face of alternative measures. Finally, we delimit the set of cases to which Olsson's model is applicable. 1 Claim (C) examined for Olsson's characterization of the relationship between the witnesses 2 Claim (C) examined for two or more independent witnesses 3 Robustness and multiple measures of corroboration 4 Discussion. (shrink)

Some authors claim that minimal models have limited epistemic value (Fumagalli, 2016; Grüne-Yanoff, 2009a). Others defend the epistemic benefits of modelling by invoking the role of robustness analysis for hypothesis confirmation (see, e.g., Levins, 1966; Kuorikoski et al., 2010) but such arguments find much resistance (see, e.g., Odenbaugh & Alexandrova, 2011). In this paper, we offer a Bayesian rationalization and defence of the view that robustness analysis can play a confirmatory role, and thereby shed light on the potential of minimal (...) models for hypothesis confirmation. We illustrate our argument by reference to a case study from macroeconomics. At the same time, we also show that there are cases in which robustness analysis is detrimental to confirmation. We characterize these cases and link them to recent investigations on evidential variety (Landes, 2020b, 2021; Osimani and Landes, forthcoming). We conclude that robustness analysis over minimal models can confirm, but its confirmatory value depends on concrete circumstances. (shrink)

Diversity of evidence is widely claimed to be crucial for evidence amalgamation to have distinctive epistemic merits. Bayesian epistemologists capture this idea in the variety-of-evidence thesis: ceteris paribus, the strength of confirmation of a hypothesis by an evidential set increases with the diversity of the evidential elements in that set. Yet, formal exploration of this thesis has shown that it fails to be generally true. This article demonstrates that the thesis fails in even more circumstances than recent results would lead (...) us to expect. Most importantly, it can fail whatever the chance that the evidential sources are unreliable. Our results hold for two types of degrees of variety: reliability independence and testable aspect independence. We conclude that the variety-of-evidence thesis can, at best, be interpreted as an exception-prone rule of thumb. (shrink)

Diversity of evidence is widely claimed to be crucial for evidence amalgamation to have distinctive epistemic merits. Bayesian epistemologists capture this idea in the variety-of-evidence thesis: ceteris paribus, the strength of confirmation of a hypothesis by an evidential set increases with the diversity of the evidential elements in that set. Yet, formal exploration of this thesis has shown that it fails to be generally true. This article demonstrates that the thesis fails in even more circumstances than recent results would lead (...) us to expect. Most importantly, it can fail whatever the chance that the evidential sources are unreliable. Our results hold for two types of degrees of variety: reliability independence and testable aspect independence. We conclude that the variety-of-evidence thesis can, at best, be interpreted as an exception-prone rule of thumb. (shrink)

The variety of evidence thesis in confirmation theory states that more varied supporting evidence confirms a hypothesis to a greater degree than less varied evidence. Under a very plausible interpretation of this thesis, positive test results from multiple independent instruments confirm a hypothesis to a greater degree than positive test results from a single instrument. We invoke Bayesian Networks to model confirmation on grounds of evidence that is obtained from less than fully reliable instruments and show that the variety of (...) evidence thesis is not sacrosanct when testing is conducted with less than fully reliable instruments: under certain conditions, a hypothesis receives more confirmation from evidence that is obtained from one rather than from more independent instruments. In the appendix, we prove certain convergence results for large numbers of positive test results from single versus multiple less than fully reliable instruments. (shrink)

Varied evidence for a hypothesis confirms it more strongly than less varied evidence, ceteris paribus. This epistemological Variety of Evidence Thesis enjoys long-standing widespread intuitive support. Recent literature has raised serious doubts that the correlational approach of explicating the thesis can vindicate it. By contrast, the eliminative approach due to Horwich vindicates the Variety of Evidence Thesis but only within a relatively narrow domain. I investigate the prospects of extending the eliminative approach to a larger domain by considering a larger (...) class of sensible explications of evidential variety. For a large subclass class of such explications I show how to construct cases in which the less diverse body of evidence for a hypothesis confirms more strongly. I hence argue that these prospects are dire since the eliminative approach widely fails to vindicate the thesis. (shrink)

Much of what we believe we know, we know through the testimony of others. While there has been long-standing evidence that people are sensitive to the characteristics of the sources of testimony, for example in the context of persuasion, researchers have only recently begun to explore the wider implications of source reliability considerations for the nature of our beliefs. Likewise, much remains to be established concerning what factors influence source reliability. In this paper, we examine, both theoretically and empirically, the (...) implications of using message content as a cue to source reliability. We present a set of experiments examining the relationship between source information and message content in people's responses to simple communications. The results show that people spontaneously revise their beliefs in the reliability of the source on the basis of the expectedness of a source's claim and, conversely, adjust message impact by perceived reliability; hence source reliability and message content have a bi-directional relationship. The implications are discussed for a variety of psychological, philosophical and political issues such as belief polarization and dual-route models of persuasion. (shrink)

The intuitive Variety of Evidence Thesis states that, ceteris paribus, more varied evidence for a hypothesis confirms it more strongly than less varied evidence. Recent Bayesian analyses have raised serious doubts in its validity. Claveau suggests the existence of a novel type of counter-example to this thesis: a gradual increase in source independence can lead to a decrease in hypothesis confirmation. I show that Claveau’s measure of gradual source independence suffers from two unsuspected types of inconsistencies. I hence put forward (...) a more natural measure of gradual source independence which is not plagued by inconsistencies. Claveau’s counter-examples to the variety of evidence thesis disappear with the measure I suggest. I hence argue that my measure is preferable and that this thesis does at least not seem to be troubled by Claveau’s arguments. (shrink)

Varied evidence confirms more strongly than less varied evidence, ceteris paribus. This epistemological Variety of Evidence Thesis enjoys widespread intuitive support. We put forward a novel explication of one notion of varied evidence and the Variety of Evidence Thesis within Bayesian models of scientific inference by appealing to measures of entropy. Our explication of the Variety of Evidence Thesis holds in many of our models which also pronounce on disconfirmatory and discordant evidence. We argue that our models pronounce rightly. Against (...) a backdrop of failures of the Variety of Evidence Thesis, the intuitive case for the Variety of Evidence Thesis emerges strengthened. Our models do however not support the general case for the thesis since our explication of it fails to hold in certain cases. The parameter space of this failure is explored and an explanation for the failure is offered. (shrink)

The article argues for the epistemic rationale of triangulation, namely, the use of multiple and independent sources of evidence. It claims that triangulation is to be understood as causal reasoning from data to phenomenon, and it rationalizes its epistemic value in terms of controlling for likely errors and biases of particular data-generating procedures. This perspective is employed to address objections against triangulation concerning the fallibility and scope of the inference, as well as problems of independence, incomparability, and discordance of evidence. (...) The debate on the existence of social preferences is used as an illustrative case. (shrink)

According to the variety of evidence thesis items of evidence from independent lines of investigation are more confirmatory, ceteris paribus, than, for example, replications of analogous studies. This thesis is known to fail (Bovens and Hartmann; Claveau). However, the results obtained by Bovens and Hartmann only concern instruments whose evidence is either fully random or perfectly reliable; instead, for Claveau, unreliability is modelled as deterministic bias. In both cases, the unreliable instrument delivers totally irrelevant information. We present a model that (...) formalizes both reliability and unreliability differently. Our instruments either are reliable, but affected by random error, or are biased but not deterministically so.Bovens and Hartmann’s results are counter-intuitive in that in their model a long series of consistent reports from the same instrument does not raise suspicion of ‘too-good-to-be-true’ evidence. This happens precisely because they contemplate neither the role of systematic bias, nor unavoidable random error of reliable instruments. In our model, the variety of evidence thesis fails as well, but the area of failure is considerably smaller than for Bovens and Hartmann and Claveau, and holds for (the majority of) realistic cases (that is, where biased instruments are very biased). The essential mechanism that triggers variety of evidence thesis failure is the rate of false to true positives for the two kinds of instruments. Our emphasis is on modelling beliefs about sources of knowledge and their role in hypothesis confirmation in interaction with dimensions of evidence, such as variety and consistency. (shrink)

Recent work in social epistemology has shown that, in certain situations, less communication leads to better outcomes for epistemic groups. In this paper, we show that, ceteris paribus, a Bayesian agent may believe less strongly that a single agent is biased than that an entire group of independent agents is biased. We explain this initially surprising result and show that it is in fact a consequence one may conceive on the basis of commonsense reasoning.

Do accounts of scientific theory formation and revision have implications for theories of everyday cognition? We maintain that failing to distinguish between importantly different types of theories of scientific inference has led to fundamental misunderstandings of the relationship between science and everyday cognition. In this article, we focus on one influential manifestation of this phenomenon which is found in Fodor's well-known critique of theories of cognitive architecture. We argue that in developing his critique, Fodor confounds a variety of distinct claims (...) about the holistic nature of scientific inference. Having done so, we outline more promising relations that hold between theories of scientific inference and ordinary cognition. (shrink)