The measurement of diagnostic accuracy is an important aspect of the evaluation of diagnostic tests. Sometimes, medical researchers try to discover the set of observations that are most accurate of all by directly inspecting diseased and not-diseased patients. This method is perhaps intuitively appealing, as it seems a straightforward empirical way of discovering how to identify diseased patients, which amounts to trying to correlate the results of diagnostic tests with disease status. I present three examples of researchers who try to (...) produce definitive diagnostic criteria by directly inspecting diseased and not diseased patients. Despite this method’s intuitive appeal, I will argue that it is impossible to carry out. Before researchers can inspect these patients to discover definitive diagnostic criteria, they must be able to distinguish diseased and not-diseased patients; and they do not know how to do this, because this is what they are trying to discover. I suspect the intuitive appeal of directly inspecting patients makes this difficult to appreciate. To counter this difficulty, I present this problem as a manifestation of ‘Meno’s paradox’, which was described in classical antiquity, and of ‘the problem of nomic measurement’, described more recently. Considering these philosophical problems may help researchers address the methodological issues they face when evaluating diagnostic tests. (shrink)