One important strategy for dealing with error in our methods is triangulation, or the use multiple methods to investigate the same object. Current accounts of triangulation assume that its primary function is to provide a confirmatory boost to hypotheses beyond what confirmation of each method alone could produce. Yet, researchers often use multiple methods to examine new constructs about which they are uncertain. For example, social psychologists use multiple indirect measures to provide convergent evidence about implicit attitudes, but how to (...) characterize implicit attitudes is an open question. To make sense of triangulation under uncertainty about constructs, I suggest two changes: first, triangulation can serve multiple epistemic functions, including some that are non-confirmatory, and second, researchers should assess the epistemic risk in claims about evidence and the acceptance/rejection of hypotheses. (shrink)

Psychologists say a measure has construct validity when it, in fact, measures the construct it is intended to measure. Construct validity is both an important notion in psychological research methods and the source of much confusion and debate among psychologists. I argue that this confusion arises, in part, because of a failure to distinguish between construct validity, a feature of measures relative to a construct, and construct legitimacy, a feature of the construct itself. I propose a prescriptive account of construct (...) validity on the basis of this distinction, then provide evidence for my account through two examples from research psychology. (shrink)

Recent work in the philosophy of scientific concepts has seen the simultaneous revival of operationalism and development of patchwork approaches to scientific concepts. We argue that these two approaches are natural allies. Both recognize an important role for measurement techniques in giving meaning to scientific terms. The association of multiple techniques with a single term, however, raises the threat of proliferating concepts (Hempel, 1966). While contemporary operationalists have developed some resources to address this challenge, these resources are inadequate to account (...) for the full range of complex behaviors of scientific concepts. We adopt show how the patchwork approach’s repertoire of inter-patch relations can expand the resources available to the operationalist. We focus on one especially important type of inter-patch relation: sharing a general reasoning strategy. General reasoning strategies serve two important functions: (1) they bind together distinct patches of scientific concepts, and (2) they provide normative guidance for extending concepts to new domains. (shrink)

Supervised machine learning has found its way into ever more areas of scientific inquiry, where the outcomes of supervised machine learning applications are almost universally classified as predictions. I argue that what researchers often present as a mere terminological particularity of the field involves the consequential transformation of tasks as diverse as classification, measurement, or image segmentation into prediction problems. Focusing on the case of machine-learning enabled poverty prediction, I explore how reframing a measurement problem as a prediction task alters (...) the primary epistemic aim of the application. Instead of measuring a property, machine learning developers conceive of their models as predicting a given measurement of this property. I argue that this _predictive reframing_ common to supervised machine learning applications is epistemically and ethically problematic, as it allows developers to externalize concerns critical to the epistemic validity and ethical implications of their model’s inferences. I further hold that the predictive reframing is not a necessary feature of supervised machine learning by offering an alternative conception of machine learning models as measurement models. An interpretation of supervised machine learning applications to measurement tasks as _automatically-calibrated model-based measurements_ internalizes questions of construct validity and ethical desirability critical to the measurement problem these applications are intended to and presented as solving. Thereby, this paper introduces an initial framework for exploring technical, historical, and philosophical research at the intersection of measurement and machine learning. (shrink)

Precise measurements of well-being would be of profound societal importance. Yet, the sceptical worry that we cannot use social science instruments and tests to measure well-being is widely discussed by philosophers and scientists. A recent and interesting philosophical argument has pointed to the psychometric procedures of construct validation to address this sceptical worry. The argument has proposed that these procedures could warrant confidence in our ability to measure well-being. The present paper evaluates whether this type of argument succeeds. The answer (...) is that it depends on which methodological background assumptions are motivating the sceptical worry to begin with. We show this by doing two things. First, we clarify (a) the different types of well-being theories involved in the science of well-being, and (b) the general methodological dimensions of well-being theorising. Second, we apply these distinctions and argue that construct validation is an unsuccessful response to measurement scepticism if this scepticism is motivated by a form of methodological non-naturalism. In the light of this, the overall point of the paper is that philosophers and scientists, when discussing measurement of well-being, should explicate their deeper methodological commitments. We further suggest that making such explicit commitments might present philosophers with a dilemma. (shrink)

The central task of cognitive neuroscience to map cognitive capacities to neural mechanisms faces three interlocking conceptual problems that together frame the problem of cognitive ontology. First, they must establish which tasks elicit which cognitive capacities, and specifically when different tasks elicit the same capacity. To address this operationalization problem, scientists often assess whether the tasks engage the same neural mechanisms. But to determine whether mechanisms are of the same or different kinds, we need to solve the abstraction problem by (...) determining which mechanistic differences are and are not relevant, and also the boundary problem by distinguishing the mechanism from its background conditions. Solving these problems, in turn, requires understanding how cognitive capacities are elicited in tasks. These three problems, which have been noted and discussed elsewhere in the literature, together form a ‘cycle of kinds’ that frames the central problem-space of cognitive ontology. We describe this cycle to clarify the intellectual challenges facing the cognitive ontologist and to reveal the kind of iterative process by which ontological revision in cognitive neuroscience is likely to unfold. (shrink)

Microaggressions are, roughly, acts or states of affairs that express prejudice or neglect toward members of oppressed groups in relatively subtle ways. There is an apparent consensus among both proponents and critics of the microaggression concept that microaggressions are “subjective.” We examine what subjectivity amounts to in this context and argue against this consensus. We distinguish between microaggressions as an explanatory posit and microaggressions as a hermeneutical tool, arguing that in either case there is no reason at present to regard (...) microaggressions as subjective and that microaggressions in the hermeneutical sense should be regarded as objective. (shrink)