The operational definition of a weak value for a quantum mechanical system involves the limit of the weak measurement strength tending to zero. I study how this limit compares to the situation for the undisturbed system. Under certain conditions, which I investigate, this limit is discontinuous in the sense that it does not merge smoothly to the Hilbert space description of the undisturbed system. Hence, in these discontinuous cases, the weak value does not represent the undisturbed system. As a result, (...) conclusions drawn from such weak values regarding the properties of the studied system cannot be upheld. Examples are given. (shrink)