This paper aims to unite two seemingly disparate themes in the philosophy of mathematics and language respectively, namely ante rem structuralism and inferentialism. My analysis begins with describing both frameworks in accordance with their genesis in the work of Hilbert. I then draw comparisons between these philosophical views in terms of their similar motivations and similar objections to the referential orthodoxy. I specifically home in on two points of comparison, namely the role of norms and the relation of ontological dependence (...) in both accounts. Lastly, I show that insights from this purported connection can address certain objections to both theories respectively. (shrink)