The growing range of methods for statistical model selection is inspiring new debates about how to handle the potential for conflicting results when different methods are applied to the same data. While many factors enter into choosing a model selection method, we focus on the implications of disagreements among scientists about whether, and in what sense, the true probability distribution is included in the candidate set of models. While this question can be addressed empirically, the data often provide inconclusive results (...) in practice. In such cases, we argue that differences in prior metaphysical views about the local adequacy of the models can produce underdetermination of results, even for the same data and candidate models. As a result, data alone are sometimes insufficient to settle rational beliefs about nature. (shrink)