This chapter contains sections titled: Highlights of Past Literature Current Work Future Work.

Randomized Controlled Trials are currently the gold standard within evidence-based medicine. Usually, they are conducted as sequential trials allowing for monitoring for early signs of effectiveness or harm. However, evidence from early stopped trials is often charged with being biased towards implausibly large effects. To our mind, this skeptical attitude is unfounded and caused by the failure to perform appropriate conditioning in the statistical analysis of the evidence. We contend that a shift from unconditional hypothesis tests in the style of (...) Neyman and Pearson to conditional hypothesis tests gives a superior appreciation of the obtained evidence and significantly improves the practice of sequential medical trials, while staying firmly rooted in frequentist methodology. (shrink)

This is a very selective survey of developments in epistemology, concentrating on work from the past twenty years that is of interest to philosophers of science. The selection is organized around interesting connections between distinct themes. I first connect issues about skepticism to issues about the reliability of belief-acquiring processes. Next I connect discussions of the defeasibility of reasons for belief to accounts of the theory-independence of evidence. Then I connect doubts about Bayesian epistemology to issues about the content of (...) perception. The last detailed connection is between considerations of the finiteness of cognition and epistemic virtues. To connect the connections I end by briefly discussing the pressure that consideration of social roles in the transmission of belief puts on the purposes of epistemology. (shrink)

Evidential holism begins with something like the claim that “it is only jointly as a theory that scientific statements imply their observable consequences.” This is the holistic claim that Elliott Sober tells us is an “unexceptional observation”. But variations on this “unexceptional” claim feature as a premise in a series of controversial arguments for radical conclusions, such as that there is no analytic or synthetic distinction that the meaning of a sentence cannot be understood without understanding the whole language of (...) which it is a part and that all knowledge is empirical knowledge. This paper is a survey of what evidential holism is, how plausible it is, and what consequences it has. Section 1 will distinguish a range of different holistic claims, Sections 2 and 3 explore how well motivated they are and how they relate to one another, and Section 4 returns to the arguments listed above and uses the distinctions from the previous sections to identify holism's role in each case. (shrink)

In this article, I argue that arguments from the history of science against scientific realism, like the arguments advanced by P. Kyle Stanford and Peter Vickers, are fallacious. The so-called Old Induction, like Vickers's, and New Induction, like Stanford's, are both guilty of confirmation bias—specifically, of cherry-picking evidence that allegedly challenges scientific realism while ignoring evidence to the contrary. I also show that the historical episodes that Stanford adduces in support of his New Induction are indeterminate between a pessimistic and (...) an optimistic interpretation. For these reasons, these arguments are fallacious, and thus do not pose a serious challenge to scientific realism. (shrink)

Charles Peirce’s model of inquiry is supposed to demarcate appropriate methods of inquiry from specious ones. Cheryl Misak points out that Peirce’s explicit account fails, but can nevertheless be rescued by elements of his own system. While Misak’s criticism is a propos, her own attempt to fortify Peirce’s account does not succeed, as it falls prey to the same criticism she raises against Peirce’s explicit account. The account provided in this paper—the ‘open path’ alternative—draws from Peirce’s corollary to his “first (...) rule of reason”, that one should not block the road to inquiry. The ‘open path’ account is able to withstand Misak’s objections, and when combined with other aspects of Peirce’s work, shows us why the optimal way to conduct inquiry is to follow the path of greatest resistance. Inquiry, however, is rarely conducted in optimal conditions. Actual, constrained conditions of inquiry require a measure of economy in terms of what can be reasonably pursued and how. The question, then, is how to conduct our inquiries so that they are as nearly optimal as possible given actual constraints. As a working scientist, Peirce was acutely aware of the need to economize in research. Nicholas Rescher has recognized this issue’s significance for Peirce, and noted its neglect in discussions of Peirce’s work. One issue that has been largely neglected in the literature on Peirce is what specifically he recommends, based on considerations of economy, for how to go about gathering evidence. Kronz & McLaughlin points out that Peirce’s primary recommendation is that in gathering evidence relevant to a particular hypothesis we should test for empirical consequences that would not have been expected otherwise. The view that is developed in this paper is that economy in research, according to Peirce, follows the general line of his theory of inquiry. Peirce’s theory of inquiry implies that pursuit of truth requires maximizing resistance. I argue that his considerations about how to navigate within the constraints introduced in the context of actual research are also best understood as recommendations for maximizing resistance. (shrink)

: Understanding Helmholtz's philosophy of science requires attention to his experimental practice. I sketch out such a project by showing how experiment shapes his theory of perception in three ways. One, the theory emerged out of empirical and experimental research. Two, the concept of experiment fills a critical conceptual gap in his theory of perception. Experiment functions not merely as a scientific technique, but also as a general epistemological strategy. Three, Helmholtz's experimental practice provides essential clues to the interpretation of (...) his theory of perception. A case study from experimental investigation of hearing shows how he designed such studies in accordance with the epistemological commitments of the theory of perception. Yet, while the theory was important to his experiments, the soundness of the experimental strategy was epistemically independent of those commitments. Secondly, the case study illustrates how Helmholtz consistently held that causal inferences underwrite reference to a real, but indirectly experienced external world. (shrink)

We argue for a naturalistic account for appraising scientific methods that carries non-trivial normative force. We develop our approach by comparison with Laudan’s (American Philosophical Quarterly 24:19–31, 1987, Philosophy of Science 57:20–33, 1990) “normative naturalism” based on correlating means (various scientific methods) with ends (e.g., reliability). We argue that such a meta-methodology based on means–ends correlations is unreliable and cannot achieve its normative goals. We suggest another approach for meta-methodology based on a conglomeration of tools and strategies (from statistical modeling, (...) experimental design, and related fields) that affords forward looking procedures for learning from error and for controlling error. The resulting “error statistical” appraisal is empirical—methods are appraised by examining their capacities to control error. At the same time, this account is normative, in that the strategies that pass muster are claims about how actually to proceed in given contexts to reach reliable inferences from limited data. (shrink)

The crisis of replication has led many to blame statistical significance tests for making it too easy to find impressive looking effects that do not replicate. However, the very fact it becomes difficult to replicate effects when features of the tests are tied down actually serves to vindicate statistical significance tests. While statistical significance tests, used correctly, serve to bound the probabilities of erroneous interpretations of data, this error control is nullified by data-dredging, multiple testing, and other biasing selection effects. (...) Arguments claiming to vitiate statistical significance tests attack straw person variants of tests that commit well-known fallacies and misinterpretations. There is a tension between popular calls for preregistration – arguably, one of the most promising ways to boost replication – and accounts that downplay error probabilities: Bayes Factors, Bayesian posteriors, likelihood ratios. By underscoring the importance of error control for well testedness, the replication crisis points to reformulating tests so as to avoid fallacies and report the extent of discrepancies that are and are not indicated with severity. (shrink)

While the common procedure of statistical significance testing and its accompanying concept of p-values have long been surrounded by controversy, renewed concern has been triggered by the replication crisis in science. Many blame statistical significance tests themselves, and some regard them as sufficiently damaging to scientific practice as to warrant being abandoned. We take a contrary position, arguing that the central criticisms arise from misunderstanding and misusing the statistical tools, and that in fact the purported remedies themselves risk damaging science. (...) We argue that banning the use of p-value thresholds in interpreting data does not diminish but rather exacerbates data-dredging and biasing selection effects. If an account cannot specify outcomes that will not be allowed to count as evidence for a claim—if all thresholds are abandoned—then there is no test of that claim. The contributions of this paper are: To explain the rival statistical philosophies underlying the ongoing controversy; To elucidate and reinterpret statistical significance tests, and explain how this reinterpretation ameliorates common misuses and misinterpretations; To argue why recent recommendations to replace, abandon, or retire statistical significance undermine a central function of statistics in science: to test whether observed patterns in the data are genuine or due to background variability. (shrink)

Despite the widespread use of key concepts of the Neyman–Pearson (N–P) statistical paradigm—type I and II errors, significance levels, power, confidence levels—they have been the subject of philosophical controversy and debate for over 60 years. Both current and long-standing problems of N–P tests stem from unclarity and confusion, even among N–P adherents, as to how a test's (pre-data) error probabilities are to be used for (post-data) inductive inference as opposed to inductive behavior. We argue that the relevance of error probabilities (...) is to ensure that only statistical hypotheses that have passed severe or probative tests are inferred from the data. The severity criterion supplies a meta-statistical principle for evaluating proposed statistical inferences, avoiding classic fallacies from tests that are overly sensitive, as well as those not sensitive enough to particular errors and discrepancies. Introduction and overview 1.1 Behavioristic and inferential rationales for Neyman–Pearson (N–P) tests 1.2 Severity rationale: induction as severe testing 1.3 Severity as a meta-statistical concept: three required restrictions on the N–P paradigm Error statistical tests from the severity perspective 2.1 N–P test T(): type I, II error probabilities and power 2.2 Specifying test T() using p-values Neyman's post-data use of power 3.1 Neyman: does failure to reject H warrant confirming H? Severe testing as a basic concept for an adequate post-data inference 4.1 The severity interpretation of acceptance (SIA) for test T() 4.2 The fallacy of acceptance (i.e., an insignificant difference): Ms Rosy 4.3 Severity and power Fallacy of rejection: statistical vs. substantive significance 5.1 Taking a rejection of H0 as evidence for a substantive claim or theory 5.2 A statistically significant difference from H0 may fail to indicate a substantively important magnitude 5.3 Principle for the severity interpretation of a rejection (SIR) 5.4 Comparing significant results with different sample sizes in T(): large n problem 5.5 General testing rules for T(), using the severe testing concept The severe testing concept and confidence intervals 6.1 Dualities between one and two-sided intervals and tests 6.2 Avoiding shortcomings of confidence intervals Beyond the N–P paradigm: pure significance, and misspecification tests Concluding comments: have we shown severity to be a basic concept in a N–P philosophy of induction? (shrink)

The growing acceptance and success of experimental economics has increased the interest of researchers in tackling philosophical and methodological challenges to which their work increasingly gives rise. I sketch some general issues that call for the combined expertise of experimental economists and philosophers of science, of experiment, and of inductive‐statistical inference and modeling. †To contact the author, please write to: 235 Major Williams, Virginia Tech, Blacksburg, VA 24061‐0126; e‐mail: [email protected].

A toast is due to one who slays Misguided followers of Bayes, And in their heart strikes fear and terror With probabilities of error! (E.L. Lehmann).

We argue that a responsible analysis of today's evidence-based risk assessments and risk debates in biology demands a critical or metascientific scrutiny of the uncertainties, assumptions, and threats of error along the manifold steps in risk analysis. Without an accompanying methodological critique, neither sensitivity to social and ethical values, nor conceptual clarification alone, suffices. In this view, restricting the invitation for philosophical involvement to those wearing a "bioethicist" label precludes the vitally important role philosophers of science may be able to (...) play as bioevidentialists. The goal of this paper is to give a brief and partial sketch of how a metascientific scrutiny of risk evidence might work. (shrink)

Ockham’s razor is the principle that, all other things being equal, scientists ought to prefer simpler theories. In recent years, philosophers have argued that simpler theories make better predictions, possess theoretical virtues like explanatory power, and have other pragmatic virtues like computational tractability. However, such arguments fail to explain how and why a preference for simplicity can help one find true theories in scientific inquiry, unless one already assumes that the truth is simple. One new solution to that problem is (...) the Ockham efficiency theorem, which states that scientists who heed Ockham’s razor retract their opinions less often and sooner than do their non-Ockham competitors. The theorem neglects, however, to consider competitors following random strategies and in many applications random strategies are known to achieve better worst-case loss than deterministic strategies. In this paper, we describe two ways to extend the result to a very general class of random, empirical strategies. The first extension concerns expected retractions, retraction times, and errors and the second extension concerns retractions in chance, times of retractions in chance, and chances of errors. (shrink)

The issues of double-counting, use-constructing, and selection effects have long been the subject of debate in the philosophical as well as statistical literature. I have argued that it is the severity, stringency, or probativeness of the test—or lack of it—that should determine if a double-use of data is admissible. Hitchcock and Sober ([2004]) question whether this ‘severity criterion' can perform its intended job. I argue that their criticisms stem from a flawed interpretation of the severity criterion. Taking their criticism as (...) a springboard, I elucidate some of the central examples that have long been controversial, and clarify how the severity criterion is properly applied to them. Severity and Use-Constructing: Four Points (and Some Clarificatory Notes) 1.1 Point 1: Getting beyond ‘all or nothing’ standpoints 1.2 Point 2: The rationale for prohibiting double-counting is the requirement that tests be severe 1.3 Point 3: Evaluate severity of a test T by its associated construction rule R 1.4 Point 4: The ease of passing vs. ease of erroneous passing: Statistical vs. ‘Definitional’ probability The False Dilemma: Hitchcock and Sober 2.1 Marsha measures her desk reliably 2.2 A false dilemma Canonical Errors of Inference 3.1 How construction rules may alter the error-probing performance of tests 3.2 Rules for accounting for anomalies 3.3 Hunting for statistically significant differences Concluding Remarks. (shrink)

In seeking general accounts of evidence, confirmation, or inference, philosophers have looked to logical relationships between evidence and hypotheses. Such logics of evidential relationship, whether hypothetico-deductive, Bayesian, or instantiationist fail to capture or be relevant to scientific practice. They require information that scientists do not generally have (e.g., an exhaustive set of hypotheses), while lacking slots within which to include considerations to which scientists regularly appeal (e.g., error probabilities). Building on my co-symposiasts contributions, I suggest some directions in which a (...) new and more adequate philosophy of evidence can move. (shrink)

Kuhn maintains that what marks the transition to a science is the ability to carry out ‘normal’ science—a practice he characterizes as abandoning the kind of testing that Popper lauds as the hallmark of science. Examining Kuhn's own contrast with Popper, I propose to recast Kuhnian normal science. Thus recast, it is seen to consist of severe and reliable tests of low-level experimental hypotheses (normal tests) and is, indeed, the place to look to demarcate science. While thereby vindicating Kuhn on (...) demarcation, my recasting of normal science is seen to tell against Kuhn's view of revolutionary science. (shrink)

This paper investigates some metaphysical and epistemological assumptions behind Bogen and Woodward’s data-to-phenomena inferences. I raise a series of points and suggest an alternative possible Kantian stance about data-to-phenomena inferences. I clarify the nature of the suggested Kantian stance by contrasting it with McAllister’s view about phenomena as patterns in data sets.

In this article, I introduce the notion of horizon for scientific practice (HSP), representing limits or boundaries within which scientists ply their trade, to facilitate analysis of scientific discovery and progress. The notion includes not only constraints that delimit scientific practice, e.g. of bringing experimentation to a temporary conclusion, but also possibilities that open up scientific practice to additional scientific discovery and to further scientific progress. Importantly, it represents scientific practice as a dynamic and developmental integration of activities to investigate (...) and analyze the natural world. I use the discovery of the clotting factor, thrombin, and the experiments conducted by the Johns Hopkins physiologist, William Howell, on the enzymatic nature of thrombin to illustrate the notion of HSP. In a concluding section, I compare the notion of HSP to other notions for scientific practice proposed in the history and philosophy of science literature. (shrink)

The purpose of this paper is to show how the degree of experimental validity of scientific procedures is crucially involved in determining two typical pragmatic modes in science, namely, the preservation of useful procedures and the disposal of useless ideas. The term 'pragmatic' will here be used following Schurz's characterisation of being internally pragmatic, as referring to that which proves useful for scientific or epistemic goals. The first part of the paper consists in a characterisation of the notion of experimental (...) validity. The second part is focused on several historical examples illustrating how the above pragmatic modes relate to the question of experimental validity. The Michelson-Morley experiment is presented as a case of a highly valid and useful experiment preserved through the development of different theories (like, on the one hand, the ether theory, upheld by Fitzgerald and Lorentz, and, on the other hand, Einstein's special theory of relativity). The concept of caloric will be discussed as an example of an idea that was discarded once it became useless, after heat was understood as a form of energy within the new frame provided by the kinetic theory of heat towards the middle of the 19th century. (shrink)

A notion of experimental series is developed, in which experiments or experimental sets are connected through experimental suggestions arising from previous experimental outcomes. To that end, the justification of Howard Temin’s DNA provirus hypothesis is examined. The hypothesis originated with evidence from two exploratory experimental sets on an oncogenic virus and was substantiated by including evidence from three additional experimental sets. Collectively these sets comprise an experimental series and the accumulative evidence from the series was adequate to justify the hypothesis (...) by persuading the virology community of its veracity. The notion of crucial experiment is also discussed in terms of experimental series. (shrink)

A notion of experimental series is developed, in which experiments or experimental sets are connected through experimental suggestions arising from previous experimental outcomes. To that end, the justification of Howard Temin's DNA provirus hypothesis is examined. The hypothesis originated with evidence from two exploratory experimental sets on an oncogenic virus and was substantiated by including evidence from three additional experimental sets. Collectively these sets comprise an experimental series and the accumulative evidence from the series was adequate to justify the hypothesis (...) by persuading the virology community of its veracity. The notion of crucial experiment is also discussed in terms of experimental series. (shrink)

An important question facing contemporary philosophy of science is whether the natural sciences in terms of their historical records exhibit distinguishing developmental patterns or structures. At least two philosophical stances are possible in answering this question. The first pertains to the plurality of the individual sciences. From this stance, the various sciences are analyzed individually and compared with one another in order to derive potential commonalities, if any, among them. The second stance involves a general philosophy of science in which (...) a thorough theory of the natural sciences is developed. The latter stance strives to account for more than possible commonalities among the sciences but also to provide a broad-spectrum philosophical framework to account for, or to explicate, the nature of science itself and its progress. In this paper, the second stance is taken in which an evolutionary philosophy of science is proposed. To that end, Thomas Kuhn’s evolutionary philosophy of science is initially discussed and critiqued. An evolutionary philosophy of science is then proposed based on a revision of Kuhn’s evolutionary philosophy of science in terms of George Gaylord Simpson’s various tempos and modes for biological evolution. Next, two historical case studies from the biological sciences are reconstructed to illustrate the robustness of the proposed evolutionary philosophy of science for explicating the progress of the natural sciences. A concluding section discusses the proposed evolutionary philosophy of science with respect to providing a broad-spectrum framework or general philosophy of science for understanding the nature and progress of the natural sciences. (shrink)

In this paper I explore an underdiscussed factor contributing to the replication crisis: Scientists, and following them policy makers, often neglect sources of errors in the production and interpretation of data and thus overestimate what can be learnt from them. This neglect leads scientists to conduct experiments that are insufficiently informative and science consumers, including other scientists, to put too much weight on experimental results. The former leads to fragile empirical literatures, the latter to surprise and disappointment when the fragility (...) of the empirical basis of some disciplines is revealed. (shrink)

The aim of this article is to contribute to the discussion about the so-called “empirical claim” and “empirical basis” of theory testing. First, the proposals of reconceptualization of the standard notions of partial potential model, intended application and empirical claim of a theory made by Balzer (1982, 1988, 1997a, 1997b, 2006, Balzer, Lauth & Zoubek 1993) and Gähde (1996, 2002, 2008) will be first discussed. Then, the distinction between “global” and “local empirical basis” will be introduced, linking it with that (...) of “global” and “local or particular empirical claim”. After that, following Balzer (1997a), along the lines of Suppes (1962), I will present a way in which what count as ‘data’ for a theory can be modeltheoretically represented and conceptualized. Finally, the proposed analysis will be exemplified with the case of classical genetics (reconstructed in Balzer & Lorenzano 2000, Lorenzano, 1995, 2000, 2002). (shrink)

A familiar adage in the philosophy of science is that general hypotheses are better supported by varied evidence than by uniform evidence. Several studies suggest that young children do not respect this principle, and thus suffer from a defect in their inductive methodology. We argue that the diversity principle does not have the normative status that psychologists attribute to it, and should be replaced by a simple rule of probability. We then report experiments designed to detect conformity to the latter (...) rule in children's inductive judgment. The results suggest that young children in both the United States and Taiwan are sensitive to the constraints imposed by the rule on judgments of probability and evidential strength. We conclude with a suggested reinterpretation of the thesis that children's inductive methodology qualifies them as “little scientists.”. (shrink)

In this paper, I show how one might resist two influential arguments for the Likelihood Principle by appealing to the ontological significance of creative intentions. The first argument for the Likelihood Principle that I consider is the argument from intentions. After clarifying the argument, I show how the key premiss in the argument may be resisted by maintaining that creative intentions sometimes independently matter to what experiments exist. The second argument that I consider is Gandenberger’s :475–503, 2015) rehabilitation of Birnbaum’s (...) :269–306, 1962) proof of the Likelihood Principle from the more intuitively obvious principles of conditionality and sufficiency. As with the argument from intentions, I show how Gandenberger’s argument for his Experimental Conditionality Principle may be resisted by maintaining that creative intentions sometimes independently matter to what experiments exist. (shrink)

Science depends on judgments of the bearing of evidence on theory. Scientists must judge whether an observation or the result of an experiment supports, disconfirms, or is simply irrelevant to a given hypothesis. Similarly, scientists may judge that, given all the available evidence, a hypothesis ought to be accepted as correct or nearly so, rejected as false, or neither. Occasionally, these evidential judgments can be made on deductive grounds. If an experimental result strictly contradicts a hypothesis, then the truth of (...) the data deductively entails the falsity of the hypothesis. In the great majority of cases, however, the connection between evidence and hypothesis is non-demonstrative, or inductive. In particular, this is so whenever a general hypothesis is inferred to be correct on the basis of the available data, since the truth of the data will not deductively entail the truth of the hypothesis. It always remains possible that the hypothesis is false even though the data are correct. (shrink)

I propose a framework that explicates and distinguishes the epistemic roles of data and models within empirical inquiry through consideration of their use in scientific practice. After arguing that Suppes’ characterization of data models falls short in this respect, I discuss a case of data processing within exploratory research in plant phenotyping and use it to highlight the difference between practices aimed to make data usable as evidence and practices aimed to use data to represent a specific phenomenon. I then (...) argue that whether a set of objects functions as data or models does not depend on intrinsic differences in their physical properties, level of abstraction or the degree of human intervention involved in generating them, but rather on their distinctive roles towards identifying and characterizing the targets of investigation. The paper thus proposes a characterization of data models that builds on Suppes’ attention to data practices, without however needing to posit a fixed hierarchy of data and models or a highly exclusionary definition of data models as statistical constructs. (shrink)

I propose a framework that explicates and distinguishes the epistemic roles of data and models within empirical inquiry through consideration of their use in scientific practice. After arguing that Suppes’ characterization of data models falls short in this respect, I discuss a case of data processing within exploratory research in plant phenotyping and use it to highlight the difference between practices aimed to make data usable as evidence and practices aimed to use data to represent a specific phenomenon. I then (...) argue that whether a set of objects functions as data or models does not depend on intrinsic differences in their physical properties, level of abstraction or the degree of human intervention involved in generating them, but rather on their distinctive roles towards identifying and characterizing the targets of investigation. The paper thus proposes a characterization of data models that builds on Suppes’ attention to data practices, without however needing to posit a fixed hierarchy of data and models or a highly exclusionary definition of data models as statistical constructs. (shrink)

I develop the idea of what I call an interpretive praxis as a generalized procedure for analyzing how experimenters can formulate observable predictions, discern real effects from experimental artifacts, and compare predictions with data. An interpretive praxis requires theories – theories not only about instruments and the interpretation of phenomena, but also theories that connect the use of instruments and interpretation of phenomena to high‐level theory. I will call all such theories that enable experimentation to work intermediate theories. I offer (...) an account of theory‐networks that embrace these intermediate theories and explain their functions. (shrink)

Proponents of Evidence-based medicine (EBM) do not provide a clear role for basic science in therapeutic decision making. Of what they do say about basic science, most of it is negative. Basic science resides on the lower tiers of EBM's hierarchy of evidence. Therapeutic decisions, according to proponents of EBM, should be informed by evidence from randomised studies (and systematic reviews of randomised studies) rather than basic science. A framework of models explicates the links between the mechanisms of basic science, (...) experimental inquiry, and observed data. Relying on the framework of models I show that basic science often plays a role not only in specifying experiments, but also analysing and interpreting the data that is provided. Further, and contradicting what is implied in EBM's hierarchy of evidence, appeals to basic science are often required to apply clinical research to therapeutic questions. © 2010 Springer Science+Business Media B.V. (shrink)

One persistent challenge in scientific practice is that the structure of the world can be unstable: changes in the broader context can alter which model of a phenomenon is preferred, all without any overt signal. Scientific discovery becomes much harder when we have a moving target, and the resulting incorrect understandings of relationships in the world can have significant real-world and practical consequences. In this paper, we argue that it is common (in certain sciences) to have changes of context that (...) lead to changes in the relationships under study, but that standard normative accounts of scientific inquiry have assumed away this problem. At the same time, we show that inference and discovery methods can “protect” themselves in various ways against this possibility by using methods with the novel methodological virtue of “diligence.” Unfortunately, this desirable virtue provably is incompatible with other desirable methodological virtues that are central to reliable inquiry. No scientific method can provide every virtue that we might want. (shrink)

We show that if among the tested hypotheses the number of true hypotheses is not equal to the number of false hypotheses, then Neyman-Pearson theory of testing hypotheses does not warrant minimal epistemic reliability. We also argue that N-P does not protect from the possible negative effects of the pragmatic value-laden unequal setting of error probabilities on N-P’s epistemic reliability. Most importantly, we argue that in the case of a negative impact no methodological adjustment is available to neutralize it, so (...) in such cases the discussed pragmatic-value-ladenness of N-P inevitably compromises the goal of attaining truth. (shrink)

Comparing alternative scientific theories obviously is relevant to theory assessment, but are comparativists (like Laudan) correct when they also make it necessary? This paper argues that they are not. Defining rationality solely in terms of theories' comparative problem-solving strengths, comparativist philosophers of science like Laudan subscribe to what I call the irrelevance claim (IC) and the necessity claim (NC). According to IC, a scientific theory's being well or poorly confirmed is "irrelevant" to its acceptance; NC is the claim that "all (...) evaluations of research traditions and theories must be made within a comparative context," how any theory "compares with its competitors" (Laudan 1977, 21, 120). Using current competing theories (T1 and T2) of population viability assessment (PVA) for the Florida panther, the paper investigates IC/NC. In part because dominant T2 panther biologists accept IC/NC (which T1 theorists reject), the paper argues that they appear both to have accepted flawed T2 and to have contributed to flawed panther science and policy. Correcting Laudan's Comparativist Philosophy of Science (LCPS), underlying the T1-versus-T2 debate, thus may hold promise for helping resolve both the scientific and policy controversy over panther PVA. (shrink)

Statistical reasoning is an integral part of modern scientific practice. In The Seven Pillars of Statistical Wisdom Stephen Stigler presents seven core ideas, or pillars, of statistical thinking and the historical developments of each of these pillars, many of which were concurrent with developments in biology. Here we focus on Stigler’s fifth pillar, regression, and his discussion of how regression to the mean came to be thought of as a solution to a challenge for the theory of natural selection. Stigler (...) argues that the purely mathematical phenomenon of regression to the mean provides a resolution to a problem for Darwin’s evolutionary theory. Thus, he argues that the resolution to the problem for Darwin’s theory is purely mathematical, rather than causal. We show why this argument is problematic. (shrink)

The anti-metaphysical argument against scientific realism is the following: Knowledge of unobservable entities implies metaphysical knowledge; There is no metaphysical knowledge. Therefore, there is no knowledge of unobservable entities. This argument has strangely received little attention in the profuse literature on scientific realism. This paper claims that the AMA is logically more fundamental than both the pessimistic meta-induction and the underdetermination argument. The second and main claim of this paper is that the instrumentalists’ use of AMA is incoherent. The gist (...) of my argument is that experimental knowledge requires minimally metaphysical knowledge, and that minimally metaphysical knowledge—when associated with empirical knowledge—suffices to yield a minimal knowledge of the unobservable. I then examine and reject two possible responses: minimally metaphysical instrumentalism and algebraic instrumentalism. (shrink)

Herbert Simon viewed innovation as a particular type of problem-solving behavior that entails refocus of attention and search for alternatives outside the existing domain of standard operations. This exploration outside of standard routines involves heuristic-based discovery and action, such as satisficing search for information and options. In our observations on the innovation process, we focus on knowledge generation. We propose viewing the process of generating knowledge—when knowledge is sufficient to instigate action, but not necessarily enough to eliminate the uncertainty of (...) the situation—as a heuristic process. Because many personal and organizational decisions are acted upon in the presence of some degree of uncertainty, we argue that heuristics structure the way in which information is processed innovatively. We provide a catalogue of instances in business decision making. (shrink)

According to the hierarchy of models account of scientific experimentation developed by Patrick Suppes and elaborated by Deborah Mayo, theoretical considerations about the phenomena of interest are involved in an experiment through theoretical models that in turn relate to experimental data through data models, via the linkage of experimental models. In this paper, I dispute the HoM account in the context of present-day high-energy physics experiments. I argue that even though the HoM account aims to characterize experimentation as a model-based (...) activity, it does not involve a modeling concept for the process of data acquisition, and it thus fails to provide a model-based characterization of the theory-experiment relationship underlying this process. In order to characterize the foregoing relationship, I propose the concept of a model of data acquisition and illustrate it in the case of the ATLAS experiment at CERN’s Large Hadron Collider, where the Higgs boson was discovered in 2012. I show that the process of data acquisition in the ATLAS experiment is performed according to a model of data acquisition that specifies and organizes the experimental procedures necessary to select the data according to a predetermined set of selection criteria. I also point out that this data acquisition model is theory-laden, in the sense that the underlying data selection criteria are determined by considering the testable predictions of the theoretical models that the ATLAS experiment is aimed to test. I take this sense of theory-ladenness to indicate that the relationship between the procedures of the ATLAS experiment and the theoretical models of the phenomena of interest is first established, prior to the formation of data models, through the data acquisition model of the experiment, thus not requiring the intermediary of other types of models as suggested by the HoM account. I therefore conclude that in the context of present-day HEP experiments, the HoM account does not consistently extend to the process of data acquisition so as to include models of data acquisition. (shrink)

We describe the three place relation of contrastive knowledge, which holds between a person, a target proposition, and a contrasting proposition. The person knows that p rather than that q. We argue for three claims about this relation. (a) Many common sense and philosophical ascriptions of knowledge can be understood in terms of it. (b) Its application is subject to fewer complications than non-contrastive knowledge is. (c) It applies over a wide range of human and nonhuman cases.

When do data provide good evidence for a hypothesis, evidence that warrants an inference to the hypothesis? Standard answers either reject the legitimacy of induction or else allow warranted inference from data to hypothesis when there are suitable relationships between and among the data and hypotheses. The severity account rejects all of these, maintaining instead that the good evidence relation concerns not only relations between data and hypotheses but also the methods for obtaining the data and the sensitivity of these (...) methods to detecting falsehood. So, for example, the severity account allows two rival hypotheses to fit the data equally well, even to the point of having the same prior and posterior probabilities relative to those data, while the data nonetheless favour one hypothesis over the other by virtue of those data providing a severe test of one hypothesis and not the other. The contributions to Error and Inference variously trace the genesis of the severity account from statistics-oriented sciences, elaborate upon the account’s details, apply the account to various philosophical issues, and offer challenges to the account’s adequacy and replies to major criticisms. The editors have developed the severity account in detail elsewhere, and this anthology is an effort …. (shrink)

This article starts an engagement on the aesthetics of experiments and offers an account for analysing how aesthetics features in the design, evaluation and reception of experiments. I identify two dimensions of aesthetic evaluation of experiments: design and significance. When it comes to design, a number of qualities, such as simplicity, economy and aptness, are analysed and illustrated with the famous Meselson-Stahl experiment. Beautiful experiments are also regarded to make significant discoveries, but I argue against a narrow construal of experimental (...) aims. By drawing on the plurality of goals experimenters have and diversity of aesthetic responses, I argue that experiments are aesthetically appreciated both when they discover and when they produce disruptive results. (shrink)

We provide a novel articulation of the epistemic peril of p-hacking using three resources from philosophy: predictivism, Bayesian confirmation theory, and model selection theory. We defend a nuanced position on p-hacking: p-hacking is sometimes, but not always, epistemically pernicious. Our argument requires a novel understanding of Bayesianism, since a standard criticism of Bayesian confirmation theory is that it cannot represent the influence of biased methods. We then turn to pre-analysis plans, a methodological device used to mitigate p-hacking. Some say that (...) pre-analysis plans are epistemically meritorious while others deny this, and in practice pre-analysis plans are often violated. We resolve this debate with a modest defence of pre-analysis plans. Further, we argue that pre-analysis plans can be epistemically relevant even if the plan is not strictly followed—and suggest that allowing for flexible pre-analysis plans may be the best available policy option. (shrink)

an observation to formulate a theory, it is no surprise that the resulting theory accurately captures that observation. However, when the theory makes a novel prediction—when it predicts an observation that was not used in its formulation—this seems to provide more substantial confirmation of the theory. This paper presents a new approach to the vexed problem of understanding the epistemic difference between prediction and accommodation. In fact, there are several problems that need to be disentangled; in all of them, the (...) key is the concept of overfitting. We float the hypothesis that accommodation is a defective methodology only when the methods used to accommodate the data fail to guard against the risk of overfitting. We connect our analysis with the proposals that other philosophers have made. We also discuss its bearing on the conflict between instrumentalism and scientific realism. Introduction Predictivisms—a taxonomy Observations Formulating the problem What might Annie be doing wrong? Solutions Observations explained Mayo on severe tests The miracle argument and scientific realism Concluding comments. (shrink)