I address the question of whether differences in color terminology cause differences in color experience in speakers of different languages. If linguistic representations directly affect color experience, then this is a case of what I call the linguistic penetrability of perception, which is a particular case of cognitive penetrability. I start with some general considerations about cognitive penetration and its alleged occurrence in the memory color effect. I then apply similar considerations to the interpretation of empirical studies of color perception (...) in speakers of different languages. I argue that findings such as differences in categorical perception in speakers of different languages do not show that language affects color experience. They therefore do not support the claim that color experience is linguistically penetrable. But even if we grant that color experience is different in speakers of different languages, I argue that this might still not be a case of linguistic penetration. Finally, I consider some epistemological consequences of the assumption that speakers of different languages have different color experiences. (shrink)