This paper discusses the outstanding problem of replicability of empirical data in the context of recent work on meta-analysis, especially within the field of evidence-based medicine. Specifically, it deals with the methodological issue of how to determine the degrees of heterogeneity between different collected studies. After critically reviewing the standard measures used to quantify meta-analytical heterogeneity, we argue that they should be revised in such a way to take into account the statistical power of the individual studies. We thus propose (...) some new measures of heterogeneity. Subsequently, we apply them to re-assess concrete case-studies from clinical research, thereby showing explicitly how the relevant values of heterogeneity diverge from those obtained with the original measures. (shrink)

Scientific disagreements are an important catalyst for scientific progress. But what happens when scientists disagree amidst times of crisis, when we need quick yet reliable policy guidance? In this paper we provide a normative account for how scientists facing disagreement in the context of ‘fast science’ should respond, and how policy makers should evaluate such disagreement. Starting from an argumentative, pragma-dialectic account of scientific controversies, we argue for the importance of ‘higher-order evidence’ (HOE) and we specify desiderata for scientifically relevant (...) HOE. We use our account to analyze the controversy about the aerosol transmission of COVID-19. (shrink)