sean goedecke

Gwern's 13,000-word essay 'Human-like Neural Nets by Catapulting' proposes that LLMs fail to achieve human-like generalization because they haven't 'grokked' their training data. Grokking — a phenomenon where continued overtraining past apparent convergence causes a sudden leap in generalization — requires training a very large, over-parameterized model on a relatively small dataset. This is the opposite of what frontier AI labs currently do (training smaller models on massive datasets). Gwern argues that training a ~100-trillion-parameter model on constrained data, at a cost of $3–10B, could force the model to discover deeper generalizations rather than relying on memorization. The author summarizes and evaluates this argument, noting the political and engineering obstacles, and expresses cautious optimism that a major lab should attempt it.

Overtraining as the path to human-like AI

AI labs train small-ish models on oceans of data

Grokking requires training a huge model on a small dataset

I mean can they test the hypothesis first on a smaller model, with smaller dataset, smaller generalization ?

A 100T parameter model trained on a small dataset is the complete antithesis of the current AI playbook but if that’s what it takes to transition from rote memorization to true generalization it’s a $10B gamble a major lab should take

The framing that frontier labs do the opposite of grokking (big data, smaller models) is the tension I keep coming back to, because grokking as I understand it needs heavy over-parameterization on a constrained dataset. What I cannot square is how you would even detect the delayed generalization jump at a 100-trillion-parameter scale without an eval suite sensitive enough to separate memorization from real structure. The measurement problem feels like the actual blocker here, not the compute bill.