Wittgenstein's arguments about rule-following and private language turn both on interpretation and what he called our 'pictures' of the mind. His remarks about these can be understood in terms of the conceptual metaphor of the mind as a container, and enable us to give a better account of physicalism.