According to influential accounts of scientific method, such as critical rationalism, scientific knowledge grows by repeatedly testing our best hypotheses. But despite the popularity of hypothesis tests in statistical inference and science in general, their philosophical foundations remain shaky. In particular, the interpretation of non-significant results—those that do not reject the tested hypothesis—poses a major philosophical challenge. To what extent do they corroborate the tested hypothesis, or provide a reason to accept it? Popper sought for measures of corroboration that could (...) adequately answer this question. According to Popper, corroboration is different from probability-raising, and grounded in the predictive success and testability of a hypothesis. As such, corroboration becomes an indicator of the scientific value of a hypothesis and guides our practical preferences over hypotheses that have been subjected to severe tests. This article proves two impossibility results for corroboration measures based on statistical relevance. The generality of these results shows that Popper’s qualitative characterization of corroboration must be misguided. I explore what a more promising and scientifically useful concept of corroboration could look like. (shrink)

Despite frequent calls for the overhaul of null hypothesis significance testing (NHST), this controversial procedure remains ubiquitous in behavioral, social and biomedical teaching and research. Little change seems possible once the procedure becomes well ingrained in the minds and current practice of researchers; thus, the optimal opportunity for such change is at the time the procedure is taught, be this at undergraduate or at postgraduate levels. This paper presents a tutorial for the teaching of data testing procedures, often referred to (...) as hypothesis testing theories. The first procedure introduced is Fisher's approach to data testing—tests of significance; the second is Neyman-Pearson's approach—tests of acceptance; the final procedure is the incongruent combination of the previous two theories into the current approach—NSHT. For those researchers sticking with the latter, two compromise solutions on how to improve NHST conclude the tutorial. (shrink)

In popular articles that play down the genetical differences among human populations, it is often stated that about 85% of the total genetical variation is due to individual differences within populations and only 15% to differences between populations or ethnic groups. It has therefore been proposed that the division of Homo sapiens into these groups is not justified by the genetic data. This conclusion, due to R.C. Lewontin in 1972, is unwarranted because the argument ignores the fact that most of (...) the information that distinguishes populations is hidden in the correlation structure of the data and not simply in the variation of the individual factors. The underlying logic, which was discussed in the early years of the last century, is here discussed using a simple genetical example. (shrink)

Biometrics has done damage with levels of R or p or Student’s t. The damage widened with Ronald A. Fisher’s victory in the 1920s and 1930s in devising mechanical methods of “testing,” against methods of common sense and scientific impact, “oomph.” The scale along which one would measure oomph is particularly clear in biomedical sciences: life or death. Cardiovascular epidemiology, to take one example, combines with gusto the “fallacy of the transposed conditional” and what we call the “sizeless stare” of (...) statistical significance. Some medical editors have battled against the 5% philosophy, as did, for example, Kenneth Rothman, the founder of Epidemiology. And decades ago a sensible few in education, ecology, and sociology initiated a “significance test controversy.” But, grantors, journal referees, and tenure committees in the statistical sciences had faith that probability spaces can substitute for scientific judgment. A finding of p <.05 is deemed to be “better” for variable X than p <.11 for variable Y. It is not. It depends on the oomph of X and Y—the effect size, size judged in the light of how much it matters for scientific or clinical purposes. In 1995 a Cancer Trialists’ Collaborative Group, for example, came to a rare consensus on effect size: 10 different studies had agreed that a certain drug for treating prostate cancer can increase patient survival by 12%. An 11th study published in the New England Journal in 1998 dismissed the drug. The dismissal was based on a t-test, not on what William Gosset had called, against Ronald A. Fisher’s machinery, “real” error. (shrink)

This paper evaluates attempts to defend established concepts of expertise and clinical judgement against the incursions of evidence‐based practice. Two related arguments are considered. The first suggests that standard accounts of evidence‐based practice imply an overly narrow view of ‘evidence’, and that a more inclusive concept, incorporating ‘patterns of knowing’ not recognised by the familiar evidence hierarchies, should be adopted. The second suggests that statistical generalisations cannot be applied non‐problematically to individual patients in specific contexts, and points out that this (...) is why we need clinical judgement. In evaluating the first argument, I propose a criterion for what counts as evidence. It is a minimalist criterion but the ‘patterns of knowing’, referred to in the literature, still fail to meet it. In evaluating the second argument, I will outline the powerful empirical reasons we have for thinking that decisions based on research evidence are usually better than decisions based on clinical judgement; and show that current efforts to rehabilitate clinical judgement seriously underestimate the strength of these reasons. By way of conclusion, I will sketch the ways in which the concept of expertise will have to be modified if we accept evidence‐based practice as a template for health‐care. (shrink)

It is widely held that falsification of statistical hypotheses is impossible. This view is supported by an analysis of the most important theories of statistical testing: these theories are not compatible with falsificationism. On the other hand, falsificationism yields a basically viable solution to the problems of explanation, prediction and theory testing in a deterministic context. The present paper shows how to introduce the falsificationist solution into the realm of statistics. This is done mainly by applying the concept of empirical (...) content to statistical hypotheses. It is shown that empirical content is a substitute for ‘power’ as defined by Neyman and Pearson. Since the empirical content of a hypothesis is independent of alternative hypotheses, the proposed theory of statistical testing allows for tests of isolated hypotheses. (shrink)

We introduce a distinction, unnoticed in the literature, between four varieties of objective Bayesianism. What we call ' strong objective Bayesianism' is characterized by two claims, that all scientific inference is 'logical' and that, given the same background information two agents will ascribe a unique probability to their priors. We think that neither of these claims can be sustained; in this sense, they are 'dogmatic'. The first fails to recognize that some scientific inference, in particular that concerning evidential relations, is (...) not (in the appropriate sense) logical, the second fails to provide a non-question-begging account of 'same background information'. We urge that a suitably objective Bayesian account of scientific inference does not require either of the claims. Finally, we argue that Bayesianism needs to be fine-grained in the same way that Bayesians fine-grain their beliefs. (shrink)