Are Fregean thoughts compositionally complex and composed of senses? We argue that, in Begriffsschrift, Frege took 'conceptual contents' to be unstructured, but that he quickly moved away from this position, holding just two years later that conceptual contents divide of themselves into 'function' and 'argument'. This second position is shown to be unstable, however, by Frege's famous substitution puzzle. For Frege, the crucial question the puzzle raises is why "The Morning Star is a planet" and "The Evening Star is a (...) planet" have different contents, but his second position predicts that they should have the same content. Frege's response to this antinomy is of course to distinguish sense from reference, but what has not previously been noticed is that this response also requires thoughts to be compositionally complex, composed of senses. That, however, raises the question just how thoughts are composed from senses. We reconstruct a Fregean answer, one that turns on an insistence that this question must be understood as semantic rather than metaphysical. It is not a question about the intrinsic nature of residents of the third realm but a question about how thoughts are expressed by sentences. (shrink)