Quine taught us the difference between a theory’s ontology and its ideology. Ontology is the things a theory’s quantifiers must range over if it is true, Ideology is the primitive concepts that must be used to state the theory. This allows us to split the theoretical virtue of parsimony into two kinds: ontological parsimony and ideological parsimony. My goal is help illuminate the virtue of ideological parsimony by giving a criterion for ideological innocence—a rule for when additional ideology does not count against parsimony. I propose the expressive power innocence criterion: if the ideology of theory one is expressively equivalent to that of theory two, then neither is ideologically simpler than the other. In its favor I offer the argument from accuracy, showing that any account of a theoretical virtue that is supposed to make theories that have it more likely to be true than theories that do not must respect it. Next I consider its ramifications, eliminating rival views and passing judgment on some arguments from parsimony that can be found in the literature. Finally, I consider two objections. First: I address an objection arising from the possibility of languages with a ‘primitive’ operator that allows us to list a theory’s primitives in the object-language. Second: I address an objection raised by Nelson Goodman against attempts to reckon simplicity by expressive power. Both objections fail.