This paper outlines a method to assess the effectiveness of diagrams, from semiotic foundations. In doing so, we explore the Peircian notion of signification, as applied to diagrammatic representations. We review a history of diagrams, with particular emphasis on schematics used for representing systems, and uncover the neglect of semiotic analysis of diagrammatic representations. Through application of category theory to the Peircian triadic model, we propose a set of quantitative quality measures for diagrams, and a framework for their assessment, based (...) on the properties of their encoding, pragmatic and perceptual morphisms. These measures include diagram complexity, utility, aesthetics and expert assessment of semiotic content, together with qualitative feedback. We consider the diagrams as an aid to cognitive processes, rather than a purely communication media. This utility-focused perspective on diagram quality dimensions allows for fresh insights into the creation of effective diagrams. (shrink)