What can we infer from numerical cognition about mathematical realism? In this paper, I will consider one aspect of numerical cognition that has received little attention in the literature: the remarkable similarities of numerical cognitive capacities across many animal species. This Invariantism in Numerical Cognition (INC) indicates that mathematics and morality are disanalogous in an important respect: proto-moral beliefs differ substantially between animal species, whereas proto-mathematical beliefs (at least in the animals studied) seem to show more similarities. This makes moral beliefs more susceptible to a contingency challenge from evolution compared to mathematical beliefs, and indicates that mathematical beliefs might be less vulnerable to evolutionary debunking arguments. I will then examine to what extent INC can be used to flesh out a positive case for mathematical realism. Finally, I will review two forms of mathematical realism that are promising in the light of the evolutionary evidence about numerical cognition, ante rem structuralism and Millean empiricism.