When can macroscopic data about a system be used to set parameters in a microfoundational simulation? We examine the epistemic viability of tweaking parameter values to generate a better fit between the outcome of a simulation and the available observational data. We restrict our focus to microfoundational simulations—those simulations that attempt to replicate the macrobehavior of a target system by modeling interactions between microentities. We argue that tweaking can be effective but that there are two central risks. First, tweaking risks (...) overfitting the simulation to the data and thus compromising predictive accuracy; and second, it risks compromising the microfoundationality of the simulation. We evaluate standard responses to tweaking and propose strategies to guard against these risks. (shrink)

Philosophers of science increasingly recognize the importance of idealization: the intentional introduction of distortion into scientific theories. Yet this recognition has not yielded consensus about the nature of idealization. e literature of the past thirty years contains disparate characterizations and justifications, but little evidence of convergence towards a common position.

In this paper, we explore the extent to which issues of simulation model validation take on novel characteristics when the models in question become particularly complex. Our central claim is that complex simulation models in general, and global models of climate in particular, face a form of confirmation holism. This holism, moreover, makes analytic understanding of complex models of climate either extremely difficult or even impossible. We argue that this supports a position we call convergence skepticism: the belief that the (...) existence of a plurality of different models making a plurality of different forecasts of future climate is likely to be a persistent feature of global climate science. (shrink)

Non-actual model systems discussed in scientific theories are compared to fictions in literature. This comparison may help with the understanding of similarity relations between models and real-world target systems. The ontological problems surrounding fictions in science may be particularly difficult, however. A comparison is also made to ontological problems that arise in the philosophy of mathematics.

The problem for the scientist created by using idealizations is to determine whether failures to achieve experimental fit are attributable to experimental error, falsity of theory, or of idealization. Even in the rare case when experimental fit within experimental error is achieved, the scientist must determine whether this is so because of a true theory and fortuitously canceling idealizations, or due to a fortuitous combination of false theory and false idealizations. For the engineer, the problem seems rather different. Experiment for (...) the engineer reveals the closeness of predictive fit that can be achieved by theory and idealization for a particular case. If the closeness of fit is good enough for some practical purpose, the job is done. If not, or there are reasons to consider variation, then the engineer needs to know how well the experimentally determined closeness of fit will extrapolate to new cases. This paper focuses on engineering measures of closeness of fit and the projectibility of those measures to new cases. (shrink)

an observation to formulate a theory, it is no surprise that the resulting theory accurately captures that observation. However, when the theory makes a novel prediction—when it predicts an observation that was not used in its formulation—this seems to provide more substantial confirmation of the theory. This paper presents a new approach to the vexed problem of understanding the epistemic difference between prediction and accommodation. In fact, there are several problems that need to be disentangled; in all of them, the (...) key is the concept of overfitting. We float the hypothesis that accommodation is a defective methodology only when the methods used to accommodate the data fail to guard against the risk of overfitting. We connect our analysis with the proposals that other philosophers have made. We also discuss its bearing on the conflict between instrumentalism and scientific realism. Introduction Predictivisms—a taxonomy Observations Formulating the problem What might Annie be doing wrong? Solutions Observations explained Mayo on severe tests The miracle argument and scientific realism Concluding comments. (shrink)

Until recently, realists and anti-realists alike have assumed that any approximations which appear in explanations and confirmations in the mathematically oriented physical and biological sciences are “mere distractions” (Laymon 1989, p. 353). When approximation techniques must be used, they are typically justified by appeals to their numerical accuracy. However, recent interest in computational complexity in the sciences has revealed that numerical accuracy is not always the only criterion which should be invoked to justify the use of approximations. Cartwright (1983), Franklin (...) (1988) and others have suggested that causal accuracy should be added as a criterion.Numerical and causal accuracy are important. However, they form a sufficient set of justificational criteria only when certain—often unstated—conditions obtain. These conditions are actually assumptions which are often not satisfied, especially in the computationally complex situations which characterize discovery and exploratory contexts. In these contexts, we need to concentrate on “delineating a justificatory set which is relevant to [the] given epistemic situation” (Duran 1988, p. 273). (shrink)

Lloyd (2009) contends that climate models are confirmed by various instances of fit between their output and observational data. The present paper argues that what these instances of fit might confirm are not climate models themselves, but rather hypotheses about the adequacy of climate models for particular purposes. This required shift in thinking—from confirming climate models to confirming their adequacy-for-purpose—may sound trivial, but it is shown to complicate the evaluation of climate models considerably, both in principle and in practice.

Simulation-based weather and climate prediction now involves the use of methods that reflect a deep concern with uncertainty. These methods, known as ensemble prediction methods, produce multiple simulations for predictive periods of interest, using different initial conditions, parameter values and/or model structures. This paper provides a non-technical overview of current ensemble methods and considers how the results of studies employing these methods should be interpreted, paying special attention to probabilistic interpretations. A key conclusion is that, while complicated inductive arguments might (...) be given for the trustworthiness of probabilistic weather forecasts obtained from ensemble studies, analogous arguments are out of reach in the case of long-term climate prediction. In light of this, the paper considers how predictive uncertainty should be conveyed to decision makers. (shrink)

Today’s most sophisticated simulation studies of future climate employ not just one climate model but a number of models. I explain why this “ensemble” approach has been adopted—namely, as a means of taking account of uncertainty—and why a comprehensive investigation of uncertainty remains elusive. I then defend a middle ground between two camps in an ongoing debate over the transformation of ensemble results into probabilistic predictions of climate change, highlighting requirements that I refer to as ownership, justification, and robustness.