In recent years, a new generation of activists has reinvigorated debate over the public commemorative landscape. While this debate is in no way limited to statues, it frequently crystallizes around public representations of historical figures who expressed support for the oppression of certain groups or contributed to their past or present oppression. In this paper, I consider what should be done about such representations. A number of philosophers have articulated arguments for modifying or removing public monuments. Joanna Burch-Brown (2017) grounds her argument for removal in what I call the “honorific” function of such representations—the ways in which they express and tend to cultivate admiration for their subjects. In the first two sections of the paper, I develop a novel argument for modifying these representations based on this insight. I argue that leaving such representations unmodified in the public space tends to undermine the dignity of members of oppressed groups as well as their assurance that society and government are committed to their rights and constitutional entitlements. In the paper’s third section, I develop a “balancing test” for determining whether the relevant moral and pragmatic considerations favor making a particular representation inaccessible to the public, or recontextualizing it for public consumption. Unlike some of the existing philosophical treatments of honorific representations that focus on particular monuments, this balancing test is designed for general application to any honorific representation that satisfies the presumptive case for modification. To conclude, I offer some reasons why weak forms of recontextualization that do not involve altering institutional context may often be an insufficient remedy for the problems I describe.