The concept of complementarity, originally defined for non-commuting observables of quantum systems with states of non-vanishing dispersion, is extended to classical dynamical systems with a partitioned phase space. Interpreting partitions in terms of ensembles of epistemic states (symbols) with corresponding classical observables, it is shown that such observables are complementary to each other with respect to particular partitions unless those partitions are generating. This explains why symbolic descriptions based on an ad hoc partition of an underlying phase space description should (...) generally be expected to be incompatible. Related approaches with different background and different objectives are discussed. (shrink)