Suppose I have learned to add by the rule "+2": I produce 2, 4, 6, 8... Now I reach 1000 and write 1002. But someone could claim: the rule I had grasped all along was "add 2 up to 1000, then add 4." How do I know that's not the rule I was following? Any finite sequence of responses is consistent with infinitely many rules. No past application of a rule determines what counts as the correct continuation. The rule itself cannot serve as its own interpretation; an interpretation is just another rule, and we face the same problem again.
The tempting response is: I follow the rule by interpreting it correctly at each step. But this only pushes the problem back. If every application requires an interpretation, and every interpretation is itself a rule requiring interpretation, we generate an infinite regress. Wittgenstein's point is that rule-following cannot be grounded in interpretation: it must eventually reach a bedrock of practice — unreflective, trained response — where justifications run out and we act without reason, from habit and training.
Following a rule is a practice, not a private act of mind. What makes an action the correct continuation of a rule is not the contents of a mental state but participation in a shared practice of training, correction, and response. "Following a rule" — like "obeying a command" — requires an institution. A person who follows a rule "privately" — without any community to correct them, without shared reactions — is not following a rule but only thinking they are. The ground of rules is not logic or interpretation but the bedrock of our forms of life.
The rule-following paradox is developed in §§138–242 of the Investigations, with the crisp statement of the paradox at §201. Kripke's Wittgenstein on Rules and Private Language (1982) made the argument famous under the label "sceptical paradox about meaning," though many readers dispute this is Wittgenstein's own position.
