Peer reviewers at many funding agencies and scientific journals are asked to score submissions both on individual criteria and overall. The overall scores should be some kind of aggregate of the criteria scores. Carole Lee identifies this as a potential locus for bias to enter the peer review process, which she calls commensuration bias. Here I view the aggregation of scores through the lens of social choice theory. I argue that, when reviewing grant proposals, it is in many cases impossible (...) to avoid commensuration bias. (shrink)

Social choice theory is the study of collective decision processes and procedures. It is not a single theory, but a cluster of models and results concerning the aggregation of individual inputs (e.g., votes, preferences, judgments, welfare) into collective outputs (e.g., collective decisions, preferences, judgments, welfare). Central questions are: How can a group of individuals choose a winning outcome (e.g., policy, electoral candidate) from a given set of options? What are the properties of different voting systems? When is a voting system (...) democratic? How can a collective (e.g., electorate, legislature, collegial court, expert panel, or committee) arrive at coherent collective preferences or judgments on some issues, on the basis of its members' individual preferences or judgments? How can we rank different social alternatives in an order of social welfare? Social choice theorists study these questions not just by looking at examples, but by developing general models and proving theorems. (shrink)

Values are incommensurable when they cannot be measured on a single cardinal scale. Many philosophers suggest that incommensurability can help us solve the problems of population ethics. I agree. But some philosophers claim that populations bear incommensurable values merely because they contain different numbers of people, perhaps within some range. I argue that mere differences in how many people exist, even within some range, do not suffice for incommensurability. I argue that the intuitive neutrality of creating happy people is better (...) captured by a version of average utilitarianism. But this view is problematic. So I suggest a version of total utilitarianism that avoids the repugnant conclusion by appealing to incommensurable dimensions of wellbeing. (shrink)