An astonishing volume and diversity of evidence is available for many hypotheses in the biomedical and social sciences. Some of this evidence—usually from randomized controlled trials (RCTs)—is amalgamated by meta-analysis. Despite the ongoing debate regarding whether or not RCTs are the ‘gold-standard’ of evidence, it is usually meta-analysis which is considered the best source of evidence: meta-analysis is thought by many to be the platinum standard of evidence. However, I argue that meta-analysis falls far short of that standard. Different meta-analyses (...) of the same evidence can reach contradictory conclusions. Meta-analysis fails to provide objective grounds for intersubjective assessments of hypotheses because numerous decisions must be made when performing a meta-analysis which allow wide latitude for subjective idiosyncrasies to influence its outcome. I end by suggesting that an older tradition of evidence in medicine—the plurality of reasoning strategies appealed to by the epidemiologist Sir Bradford Hill—is a superior strategy for assessing a large volume and diversity of evidence. (shrink)

Amalgamating evidence of different kinds for the same hypothesis into an overall confirmation is analogous, I argue, to amalgamating individuals’ preferences into a group preference. The latter faces well-known impossibility theorems, most famously “Arrow’s Theorem”. Once the analogy between amalgamating evidence and amalgamating preferences is tight, it is obvious that amalgamating evidence might face a theorem similar to Arrow’s. I prove that this is so, and end by discussing the plausibility of the axioms required for the theorem.

Kuhn’s famous thesis that there is ‘no unique algorithm’ for choosing between rival scientific theories is analysed using the machinery of social choice theory. It is shown that the problem of theory choice as posed by Kuhn is formally identical to a standard social choice problem. This suggests that analogues of well-known results from the social choice literature, such as Arrow’s impossibility theorem, may apply to theory choice. If an analogue of Arrow’s theorem does hold for theory choice this would (...) refute Kuhn’s thesis, but it would also pose a threat to the rationality of science, a threat that is if anything more worrying than that posed by Kuhn. Various possible ‘escape routes’ from Arrow’s impossibility result are examined, in particular Amartya Sen’s idea of ‘enriching the informational basis’. It is shown that Sen’s idea can be applied to the problem of theory choice in science. This in turn sheds light on two well-known approaches to inductive inference in philosophy of science: Bayesianism and statistical model selection. (shrink)

Robustness is a common platitude: hypotheses are better supported with evidence generated by multiple techniques that rely on different background assumptions. Robustness has been put to numerous epistemic tasks, including the demarcation of artifacts from real entities, countering the “experimenter’s regress,” and resolving evidential discordance. Despite the frequency of appeals to robustness, the notion itself has received scant critique. Arguments based on robustness can give incorrect conclusions. More worrying is that although robustness may be valuable in ideal evidential circumstances (i.e., (...) when evidence is concordant), often when a variety of evidence is available from multiple techniques, the evidence is discordant. †To contact the author, please write to: Jacob Stegenga, Department of Philosophy, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093; e‐mail: [email protected]. (shrink)

Lloyd (2009) contends that climate models are confirmed by various instances of fit between their output and observational data. The present paper argues that what these instances of fit might confirm are not climate models themselves, but rather hypotheses about the adequacy of climate models for particular purposes. This required shift in thinking—from confirming climate models to confirming their adequacy-for-purpose—may sound trivial, but it is shown to complicate the evaluation of climate models considerably, both in principle and in practice.

There is a very plausible principle of the transitivity of justifying reasons. It says that if "p" is better justified than "q" (all things considered) and "q" better than "r", then "p" is better justified than "r" (all things considered). There is a corresponding principle of rational theory choice. Call one theory "a better theory than" another theory if all criteria of theory choice considered (explanatory power, simplicity, empirical adequacy, etc.), the first theory meets the criteria better than the second (...) theory. The corresponding transitivity principle says that if theory A is a better theory than theory B and if theory B is a better theory than theory C, then A is a better theory than theory C. I argue against this principle. It turns out that whenever there are 2 or more relevant and independent criteria of theory evaluation, and whenever at least of one the criteria is "non-linear" in a certain sense, there may be violations of transitivity which do not violate any standards of rationality (of theory choice). This shows, again, that theory choice cannot be seen as merely the application of given rules of rational theory choice. (shrink)

Is there a public good? A prevalent view in political science is that democracy is unavoidably chaotic, arbitrary, meaningless, and impossible. Such scepticism began with Condorcet in the eighteenth century, and continued most notably with Arrow and Riker in the twentieth century. In this powerful book, Gerry Mackie confronts and subdues these long-standing doubts about democratic governance. Problems of cycling, agenda control, strategic voting, and dimensional manipulation are not sufficiently harmful, frequent, or irremediable, he argues, to be of normative concern. (...) Mackie also examines every serious empirical illustration of cycling and instability, including Riker's famous argument that the US Civil War was due to arbitrary dimensional manipulation. Almost every empirical claim is erroneous, and none is normatively troubling, Mackie says. This spirited defence of democratic institutions should prove both provocative and influential. (shrink)

To study Earth’s climate, scientists now use a variety of computer simulation models. These models disagree in some of their assumptions about the climate system, yet they are used together as complementary resources for investigating future climatic change. This paper examines and defends this use of incompatible models. I argue that climate model pluralism results both from uncertainty concerning how to best represent the climate system and from difficulties faced in evaluating the relative merits of complex models. I describe how (...) incompatible climate models are used together in ‘multi-model ensembles’ and explain why this practice is reasonable, given scientists’ inability to identify a ‘best’ model for predicting future climate. Finally, I characterize climate model pluralism as involving both an ontic competitive pluralism and a pragmatic integrative pluralism. (shrink)