Self-control is integral to successful human agency. Without it we cannot extend our agency across time and secure central social, moral, and personal goods. But self-control is not a unitary capacity. In the first part of this paper we provide a taxonomy of self-control and trace its connections to agency and the self. In part two, we turn our attention to the external conditions that support successful agency and the exercise of self-control. We argue that what we call moral security is a critical foundation for agency. Parts three and four explore what happens to agency when moral security is lacking, as in the case of those subject to racism, and those living in poverty. The disadvantages suffered by those who are poor, in a racial minority or other oppressed group, or suffering mental illness or addiction, are often attributed to a lack of individual self-control or personal responsibility. In particular, members of these groups are often seen as irresponsibly focused on short-term pleasures over long-term good, a view underwritten by particular psychological theories of self-control. We explore how narratives about racism and poverty undermine moral security, and limit and distort the possibility of synchronic and diachronic self-control. Where moral security is undermined, the connection between self-control and diachronic goods often fails to obtain and agency contracts accordingly. We close with some preliminary reflections on the implications for responsibility.