The relationship between physics and geometry is examined in classical and quantum physics based on the view that the symmetry group of physics and the automorphism group of the geometry are the same. Examination of quantum phenomena reveals that the space-time manifold is not appropriate for quantum theory. A different conception of geometry for quantum theory on the group manifold, which may be an arbitrary Lie group, is proposed. This provides a unified description of gravity and gauge fields as well (...) as generalizations of these fields. A correspondence principle which relates the geometry of quantum physics and the geometry of classical physics is formulated. (shrink)