It is widely believed that some puzzling and provocative remarks that Frege makes in his late writings indicate he rejected independence arguments in geometry, particularly arguments for the independence of the parallels axiom. I show that this is mistaken: Frege distinguished two approaches to independence arguments and his puzzling remarks apply only to one of them. Not only did Frege not reject independence arguments across the board, but also he had an interesting positive proposal about the logical structure of correct (...) independence arguments, deriving from the geometrical principle of duality and the associated idea of substitution invariance. The discussion also serves as a useful focal point for independently interesting details of Frege's mathematical environment. This feeds into a currently active scholarly debate because Frege's supposed attitude to independence arguments has been taken to support a widely accepted thesis (proposed by Ricketts among others) concerning Frege's attitude toward metatheory in general. I show that this thesis gains no support from Frege's puzzling remarks about independence arguments. (shrink)