This paper is about a problem that can arise when we try to harness the “wisdom of the crowd” from groups comprised of individuals who exhibit a certain kind of epistemic humility in the way they respond to testimonial evidence. I begin by setting out the problem and then make some initial steps toward solving it. The solution I develop is tentative and may not apply in all circumstances, but it promises to alleviate what seems to me to be a (...) problem of both theoretical interest and practical importance. (shrink)

Artificially intelligent systems will be used to make increasingly important decisions about us. Many of these decisions will have to be made without universal agreement about the relevant moral facts. For other kinds of disagreement, it is at least usually obvious what kind of solution is called for. What makes moral disagreement especially challenging is that there are three different ways of handling it. _Moral solutions_ apply a moral theory or related principles and largely ignore the details of the disagreement. (...) _Compromise solutions_ apply a method of finding a compromise and taking information about the disagreement as input. _Epistemic solutions_ apply an evidential rule that treats the details of the disagreement as evidence of moral truth. Proposals for all three kinds of solutions can be found in the AI ethics and value alignment literature, but little has been said to justify choosing one over the other. I argue that the choice is best framed in terms of _moral risk_. (shrink)