In this paper I present two cases, taken from the history of science, in which mathematics and physics successfully interplay. These cases provide, respectively, an example of the successful application of mathematics in astronomy and an example of the successful application of mechanics in mathematics. I claim that an illustration of these cases has a twofold value in the context of the applicability debate. First, it enriches the debate with an historical perspective which is largely omitted in the contemporary discussion. (...) Second, it reveals a component of the applicability problem that has received little attention. This component concerns the successful application of physical principles in mathematical practice. With the help of the two examples, in the final part of the paper I address the following question: are successful applications of mathematics to physics and successful applications of physics to mathematics two sides of the same problem? (shrink)