Biologists and philosophers of biology have argued that learning rules that do not lead organisms to play evolutionarily stable strategies (ESSes) in games will not be stable and thus not evolutionarily successful. This claim, however, stands at odds with the fact that learning generalization---a behavior that cannot lead to ESSes when modeled in games---is observed throughout the animal kingdom. In this paper, I use learning generalization to illustrate how previous analyses of the evolution of learning have gone wrong. It has (...) been widely argued that the function of learning generalization is to allow for swift learning about novel stimuli. I show that in evolutionary game theoretic models learning generalization, despite leading to suboptimal behavior, can indeed speed learning. I further observe that previous analyses of the evolution of learning ignored the short term success of learning rules. If one drops this assumption, I argue, it can be shown that learning generalization will be expected to evolve in these models. I also use this analysis to show how ESS methodology can be misleading, and to reject previous justifications about ESS play derived from analyses of learning. (shrink)