Daniel Isaacson has advanced an epistemic notion of arithmetical truth according to which the latter is the set of truths that we grasp on the basis of our understanding of the structure of natural numbers alone. Isaacson’s thesis is then the claim that Peano Arithmetic (PA) is the theory of finite mathematics, in the sense that it proves all and only arithmetical truths thus understood. In this paper, we raise a challenge for the thesis and show how it can be (...) overcome. We introduce the concept of purity for theories of arithmetic: a theory of arithmetic is pure when it only proves arithmetical truths. Then, we argue that, under Isaacson’s thesis, some PA-provable truths—including transfinite induction claims for infinite ordinals and some consistency statements—are seemingly not arithmetical in Isaacson’s sense, and hence that Isaacson’s thesis might entail the impurity of PA. Nonetheless, we conjecture that the advocate of Isaacson’s thesis can avoid this undesirable consequence: the arithmetical nature, as understood by Isaacson, of all contentious PA-provable statements can be justified. As a case study, we explore how this is done for transfinite induction claims with infinite ordinals below ε0. To this end, we show that the PA-proof of such claims employs exclusively resources from finite mathematics, and that ordinals below ε0 are finitary objects despite being infinite. (shrink)