The notion of epistemic responsibility applied to memory has been in general examined in the framework of the responsibilities that a collective holds for past injustices, but it has never been the object of an analysis of its own. In this article, I propose to isolate and explore it in detail. For this purpose, I start by conceptualizing the epistemic responsibility applied to individual memories. I conclude that an epistemic responsible individual rememberer is a vigilant agent who knows when to engage in different kinds of mental and non-mental actions in order to monitor and update her memories, and who develops and nurture different kinds of virtuous attitudes that guide those actions. These (epistemic) virtuous attitudes are oriented not only towards herself but also towards others. Whereas this conception of epistemic responsibility does not pose a problem to understand shared memories of family members and friends, it may seem suspicious when applied to large-scale collective memories. These memories, which I name historical memories, are memories of events that have a traumatic impact for the community, are permeated by unequal relations of power, keep a complex relationship with historical science, and present other characteristics that distinguish them from individual memories. But despite these differences, the analysis undertaken in this work shows that the general principles that govern the epistemic responsibility of individual and (large-scale) collective rememberers are similar, and are based on similar grounds: pragmatic considerations about the consequences of misremembering or forgetting and a feeling of care. The similarities at the individual and collective scale of the epistemic vigilant attitude that is and should be taken toward our significant past may partially justify the use of the same epithet—“memory”—to refer to these different kinds of representations.