Model-Selection Theory: The Need for a More Nuanced Picture of Use-Novelty and Double-Counting

Download Edit this record How to cite View on PhilPapers
Abstract
This article argues that common intuitions regarding (a) the specialness of ‘use-novel’ data for confirmation and (b) that this specialness implies the ‘no-double-counting rule’, which says that data used in ‘constructing’ (calibrating) a model cannot also play a role in confirming the model’s predictions, are too crude. The intuitions in question are pertinent in all the sciences, but we appeal to a climate science case study to illustrate what is at stake. Our strategy is to analyse the intuitive claims in light of prominent accounts of confirmation of model predictions. We show that on the Bayesian account of confirmation, and also on the standard classical hypothesis-testing account, claims (a) and (b) are not generally true; but for some select cases, it is possible to distinguish data used for calibration from use-novel data, where only the latter confirm. The more specialized classical model-selection methods, on the other hand, uphold a nuanced version of claim (a), but this comes apart from (b), which must be rejected in favour of a more refined account of the relationship between calibration and confirmation. Thus, depending on the framework of confirmation, either the scope or the simplicity of the intuitive position must be revised.
Categories
(categorize this paper)
Reprint years
2015, 2016, 2018
PhilPapers/Archive ID
STEMTT-2
Upload history
Archival date: 2017-12-13
View other versions
Added to PP index
2015-12-01

Total views
189 ( #23,137 of 51,740 )

Recent downloads (6 months)
15 ( #33,092 of 51,740 )

How can I increase my downloads?

Downloads since first upload
This graph includes both downloads from PhilArchive and clicks on external links on PhilPapers.