There has recently been an expansion of anti-abortion measures in the United States. Within these various measures there is a divide over certain exceptions: some States permit abortion for pregnancies caused by rape while other States do not. This paper explores the underlying moral justification for such exceptions. I argue that within the dominant moral framework for reproductive ethics these exceptions are incoherent by their own lights. But this is not a defense of an exceptionless anti-abortion position. Rather, because the typical way of making such exceptions is incoherent, this shows why the anti-abortion movement is dangerous: as these incoherencies are acknowledged, this may lead to ever stricter measures being put in place. I end by suggesting that those who are sympathetic to these exceptions should find it easier to move to a pro-choice position rather than to a more extreme, exceptionless one.