In this paper, we aim to explore connections between a Carnapian semantics of theoretical terms and an eliminative structuralist approach in the philosophy of mathematics. Specifically, we will interpret the language of Peano arithmetic by applying the modal semantics of theoretical terms introduced in Andreas (Synthese 174(3):367–383, 2010). We will thereby show that the application to Peano arithmetic yields a formal semantics of universal structuralism, i.e., the view that ordinary mathematical statements in arithmetic express general claims about all admissible interpretations (...) of the Peano axioms. Moreover, we compare this application with the modal structuralism by Hellman (Mathematics without numbers: towards a modal-structural interpretation. Oxford University Press: Oxford, 1989), arguing that it provides us with an easier epistemology of statements in arithmetic. (shrink)