In this paper I discuss Mark Steiner’s view of the contribution of mathematics to physics and take up some of the questions it raises. In particular, I take up the question of discovery and explore two aspects of this question – a metaphysical aspect and a related epistemic aspect. The metaphysical aspect concerns the formal structure of the physical world. Does the physical world have mathematical or formal features or constituents, and what is the nature of these constituents? The related epistemic question concerns humans’ cognitive ability to reach the formal structure of the physical world. Among other things, I explore the interaction of mathematical and non-mathematical cognition in physical discovery.