Modern physics is founded on two mainstays: mathematical modelling and empirical verification. These two assumptions are prerequisite for the objectivity of scientific discourse. Here we show, however, that they are contradictory, leading to the ‘experiment paradox’. We reveal that any experiment performed on a physical system is—by necessity—invasive and thus establishes inevitable limits to the accuracy of any mathematical model. We track its manifestations in both classical and quantum physics and show how it is overcome ‘in practice’ via the concept (...) of environment. We argue that the unravelled paradox induces a new type of ‘ontic’ underdetermination, which has deep consequences for the methodological foundations of physics. (shrink)