The claim that common sense regards free will and moral responsibility as compatible with determinism has played a central role in both analytic and experimental philosophy. In this paper, we show that evidence in favor of this “natural compatibilism” is undermined by the role that indeterministic metaphysical views play in how people construe deterministic scenarios. To demonstrate this, we re-examine two classic studies that have been used to support natural compatibilism. We find that although people give apparently compatibilist responses, this is largely explained by the fact that people import an indeterministic metaphysics into deterministic scenarios when making judgments about freedom and responsibility. We conclude that judgments based on these scenarios are not reliable evidence for natural compatibilism.