Randomized Controlled Trials are currently the gold standard within evidence-based medicine. Usually, they are conducted as sequential trials allowing for monitoring for early signs of effectiveness or harm. However, evidence from early stopped trials is often charged with being biased towards implausibly large effects. To our mind, this skeptical attitude is unfounded and caused by the failure to perform appropriate conditioning in the statistical analysis of the evidence. We contend that a shift from unconditional hypothesis tests in the style of (...) Neyman and Pearson to conditional hypothesis tests gives a superior appreciation of the obtained evidence and significantly improves the practice of sequential medical trials, while staying firmly rooted in frequentist methodology. (shrink)

In this paper, I show how one might resist two influential arguments for the Likelihood Principle by appealing to the ontological significance of creative intentions. The first argument for the Likelihood Principle that I consider is the argument from intentions. After clarifying the argument, I show how the key premiss in the argument may be resisted by maintaining that creative intentions sometimes independently matter to what experiments exist. The second argument that I consider is Gandenberger’s :475–503, 2015) rehabilitation of Birnbaum’s (...) :269–306, 1962) proof of the Likelihood Principle from the more intuitively obvious principles of conditionality and sufficiency. As with the argument from intentions, I show how Gandenberger’s argument for his Experimental Conditionality Principle may be resisted by maintaining that creative intentions sometimes independently matter to what experiments exist. (shrink)

A “stopping rule” in a sequential experiment is a rule or procedure for deciding when that experiment should end. Accordingly, the “stopping rule principle” states that, in a sequential experiment, the evidential relationship between the final data and an hypothesis under consideration does not depend on the experiment’s stopping rule: the same data should yield the same evidence, regardless of which stopping rule was used. In this essay, I reconstruct and rebut five independent arguments for the SRP. Reminding oneself that (...) the stopping rule is a part of an experiment’s design and is no more mysterious than many other design aspects helps elucidate why some of these arguments for the SRP are unsound. (shrink)

The stopping rule for a sequential experiment is the rule or procedure for determining when that experiment should end. Accordingly, the stopping rule principle (SRP) states that the evidential relationship between the final data from a sequential experiment and a hypothesis under consideration does not depend on the stopping rule: the same data should yield the same evidence, regardless of which stopping rule was used. I clarify and provide a novel defense of two interpretations of the main argument against the (...) SRP, the foregone conclusion argument. According to the first, the SRP allows for highly confirmationally unreliable experiments, which concept I make precise, to confirm highly. According to the second, it entails the evidential equivalence of experiments differing significantly in their confirmational reliability. I rebut several attempts to deflate or deflect the foregone conclusion argument, drawing connections with replication in science and the likelihood principle. (shrink)

This paper considers a key point of contention between classical and Bayesian statistics that is brought to the fore when examining so-called ‘persistent experimenters’—the issue of stopping rules, or more accurately, outcome spaces, and their influence on statistical analysis. First, a working definition of classical and Bayesian statistical tests is given, which makes clear that (1) once an experimental outcome is recorded, other possible outcomes matter only for classical inference, and (2) full outcome spaces are nevertheless relevant to both the (...) classical and Bayesian approaches, when it comes to planning/choosing a test. The latter point is shown to have important repercussions. Here we argue that it undermines what Bayesians may admit to be a compelling argument against their approach—the Bayesian indifference to persistent experimenters and their optional stopping rules. We acknowledge the prima facie appeal of the pro-classical ‘optional stopping intuition’, even for those who ordinarily have Bayesian sympathies. The final section of the paper, however, provides three error theories that may assist a Bayesian in explaining away the apparent anomaly in their reasoning. (shrink)

According to influential accounts of scientific method, e.g., critical rationalism, scientific knowledge grows by repeatedly testing our best hypotheses. In comparison to rivaling accounts of scientific reasoning such as Bayesianism, these accounts are closer to crucial aspects of scientific practice. But despite the preeminence of hypothesis tests in statistical inference, their philosophical foundations are shaky. In particular, the interpretation of "insignificant results"---outcomes where the tested hypothesis has survived the test---poses a major epistemic challenge that is not sufficiently addressed by the (...) standard methodology for conducting such tests. In this paper, I argue that a quantitative explication of degree of corroboration can fill this important methodological and epistemological gap. First, I argue that this concept is distinct from the Bayesian notion of evidential support and that it plays an independent role in scientific reasoning. Second, I demonstrate that degree of corroboration cannot be suitably explicated in a probabilistic relevance framework, as proposed by Popper. Third, I derive two measures of corroboration that possess a large number of attractive properties, establish an insightful relation between corroboration and evidential support and are not committed to a Bayesian or a frequentist framework. In sum, the paper rethinks the foundations of inductive inference by providing a novel logic of hypothesis testing. (shrink)