Using art and aesthetics as context, we explore the notion that curiosity and creativity emanate from a single novelty-seeking mechanism and outline support for the idea. However, we also highlight the importance of learning progress tracking in exploratory action and advocate for a nuanced understanding that aligns novelty-seeking with learnability. This, we argue, offers a more comprehensive framework of how curiosity and creativity are related.