This paper argues that most of the alleged straight solutions to the sceptical paradox which Kripke ascribed to Wittgenstein can be regarded as the first horn of a dilemma whose second horn is the paradox itself. The dilemma is proved to be a by‐product of a foundationalist assumption on the notion of justification, as applied to linguistic behaviour. It is maintained that the assumption is unnecessary and that the dilemma is therefore spurious. To this end, an alternative conception of the justification of linguistic behaviour is outlined, a conception that vindicates some of the insights behind Kripke's Wittgenstein's sceptical solution of the paradox. This alternative conception is defended against two objections (both familiar from McDowell's works): (1) that it would imply that for the linguistic community there is no authority, no standard to meet and, therefore, no possibility of error and (2) that it would lead to a kind of idealism.