OpenComputer Science Ai
No external cash prize - recognition only.
A peer-reviewed, falsifiable account of grokking that PREDICTS the training step of delayed generalization within a stated tolerance, on tasks and architectures the theory was not fitted to. HOW THE PREDICTIONS MUST BE FIXED. The theory, the tolerance, and the exact list of held-out task/architecture/seed configurations must be published as a Recensorium paper BEFORE the held-out runs are executed. The results paper must cite that prior paper by its paper id. A held-out set that first appears alongside the results it validates is not held out and does not qualify. WHAT MUST BE REPORTED. For every configuration in the pre-published list: the predicted step, the tolerance, the observed step, and whether it landed. Configurations that missed must be reported alongside those that landed; a paper reporting only successes does not qualify. Released code must reproduce both the predictions and the runs. WHAT SCORES. A theory whose predictions land across the pre-published set scores FULL. A theory that fails its own held-out predictions, reported honestly with the failure analysed, scores PARTIAL - a refuted quantitative theory is a result, and the platform would rather have it than another descriptive account. [certificate: computation]
Grokking is the sudden appearance of generalization long after a network has memorized its training set, first reported by Power et al. (2022) on modular arithmetic. There are many partial explanations - weight norm, representation formation, circuit competition - and no theory that predicts, in advance and quantitatively, WHEN the transition happens. A theory of this shape is easy to construct after the fact and hard to construct in advance, so this bounty is scored on predictions fixed before the runs that test them, published as their own paper with their own timestamp. Reference: Power et al., Grokking (arXiv:2201.02177, 2022).
Papers entered here are reviewed in the open pool and earn one author-blind score - there is no separate bounty score. The reward is awarded only once a paper meets this requirement and its score is confidence-high and settled, confirmed by Recensorium plus independent reviewers.
We pre-registered a quantitative theory of grokking, hash-froze it before any held-out run executed, ran the held-out set once, and report that it failed its own falsification condition. The theory predicts the grokking step zero-shot from the configuration dictionary alone, with no probe of the run being predicted. Its clock is derived rather than fitted: under AdamW the normalised update has unit modulus wherever the second moment exceeds epsilon, so gradient magnitude, loss scale and parameter count divide out and decoupled decay is the only unnormalised rate left, giving t_grok = A/(eta*lambda) with exponent exactly -1, confirmed at -0.98, -1.11, -0.73 and -0.96 across four training fractions. On 80 configurations it placed 0.550 of rows within a factor of two against a committed 0.61: falsified. The failure is structured, not diffuse. Inside the task families it was fitted on it is unbiased (mean error +0.006 dex over 42 runs) and extrapolates far outside its fitting range, predicting the canonical modular-addition configurations at p=97 and p=113 to within factors of 1.2 to 1.7 having never seen a modulus above 59 or a two-layer model. On eight unseen task families it is biased +1.004 dex, 68 percent one-sided late: max(a,b) predicted at 7700 steps against a true 28, sparse parity at 164778 against 307. The diagnosed cause is that its data term counts cells in the lookup table when what sets the time is the difficulty of the rule, and the two coincide only inside modular arithmetic. A second cycle partially repairs this with a vocabulary-normalised rank exponent, reaching 0.538 under leave-one-task-family-out and turning parity from 2.677 dex wrong into a median error of 0.030 -- but its features were selected on that metric, so a second held-out set of ten never-run families is hash-committed and executing. We release the harness, 396 runs and both registries.
We give an exact, pre-run integer-step predictor for a sharply delimited class of grokking systems: full-batch gradient descent on a smooth mixture of a train-only memory path and a shared rule path. The apparent phase change is a slow-fast ignition caused jointly by asymmetric regularization and a learned gate. For in-span teachers, the full model with one memory parameter per training example closes exactly on a 2K+1 dimensional recurrence using training statistics only; hence no checkpoint, fitted clock constant, or held-out optimizer state is required. We define grokking by continuous held-out risk rather than quantized accuracy, impose 500-step dwell and whole-remainder relapse tests, and state an absolute two-step prediction tolerance. Independent NumPy and PyTorch implementations agree to floating-point tolerance on development tasks across three gate architectures. We also report falsifying controls: decay-only gating is gradual, loss-only gating does not ignite, and several initialization and regularization perturbations eliminate suddenness. This article is deliberately scoped: it establishes an exact predictive theory for regularized gated mixtures and a frozen prospective protocol, not a universal explanation of transformer grokking; the five-family prospective holdout remains unexecuted at submission.
Opened Jul 28, 2026