One important strategy for dealing with error in our methods is triangulation, or the use multiple methods to investigate the same object. Current accounts of triangulation assume that its primary function is to provide a confirmatory boost to hypotheses beyond what confirmation of each method alone could produce. Yet, researchers often use multiple methods to examine new constructs about which they are uncertain. For example, social psychologists use multiple indirect measures to provide convergent evidence about implicit attitudes, but how to (...) characterize implicit attitudes is an open question. To make sense of triangulation under uncertainty about constructs, I suggest two changes: first, triangulation can serve multiple epistemic functions, including some that are non-confirmatory, and second, researchers should assess the epistemic risk in claims about evidence and the acceptance/rejection of hypotheses. (shrink)

This paper examines how scientists conceptualize their research methodologies. Do scientists raise concerns about vague criteria and genuine uncertainties in experimental practice? If so, what sorts of issues do they identify as problematic? Do scientists acknowledge the presence of value judgments in scientific research, and do they reflect on the relation between epistemic and non-epistemic criteria for decisionmaking? We present findings from an analysis of qualitative interviews with 63 scientific researchers who talk about their views on good research practice. We (...) argue that analysts of science should care about scientists’ conceptualizations of the criteria and of the practical judgments that scientific inquiry involves. While scientists’ accounts of their own research methodologies alone do not give us a full picture of how science really works, they can point us to areas of concern. They can inspire and direct philosophical reflections about how science works. Throughout the interviews, the participating researchers provided specific examples from their own research contexts as illustrations of their methodological points. These examples reveal that scientists often struggle to evaluate the quality of their data, to figure out whether the available evidence confirms their hypothesis, whether a replication was successful, or to what extent they can rely on peer-reviewed papers. General ideas about good research methods do not directly translate into specific evaluation criteria or strategies that can guide research and help validate empirical data. (shrink)

This paper examines how scientists conceptualize their research methodologies. Do scientists raise concerns about vague criteria and genuine uncertainties in experimental practice? If so, what sorts of issues do they identify as problematic? Do scientists acknowledge the presence of value judgments in scientific research, and do they reflect on the relation between epistemic and non-epistemic criteria for decisionmaking? We present findings from an analysis of qualitative interviews with 63 scientific researchers who talk about their views on good research practice. We (...) argue that analysts of science should care about scientists’ conceptualizations of the criteria and of the practical judgments that scientific inquiry involves. While scientists’ accounts of their own research methodologies alone do not give us a full picture of how science really works, they can point us to areas of concern. They can inspire and direct philosophical reflections about how science works. Throughout the interviews, the participating researchers provided specific examples from their own research contexts as illustrations of their methodological points. These examples reveal that scientists often struggle to evaluate the quality of their data, to figure out whether the available evidence confirms their hypothesis, whether a replication was successful, or to what extent they can rely on peer-reviewed papers. General ideas about good research methods do not directly translate into specific evaluation criteria or strategies that can guide research and help validate empirical data. (shrink)