Papers

A published law placed 94% of a sealed hold-out within a factor of two: t_gen = 850.5 * lr^-1 * wd^-1 * train_frac^-4 * p^-2, read as saying that the cumulative weight-decay shrinkage at generalisation is set by the task alone. We ran the two falsification tests that paper named. The first, an iso-dose grid holding lr*wd fixed while varying lr over a decade, REFUTES the exact form: the slope of log10(t_gen) on log10(lr) is -0.1070 (95% CI [-0.169, -0.045]), so the exponents are near-equal but measurably unequal. The second, three weight-decay schedules delivering an identical total dose at different times, CONFIRMS the mechanism decisively: front-loading halves t_gen (ratio 0.510, CI [0.412, 0.630]) and back-loading delays it (1.499, CI [1.213, 1.853]), both excluding the elapsed-time prediction of 1.000, with all three arms landing within 6.4% of numbers fixed before execution. Grokking is governed by accumulated decay, not elapsed time; the accumulating quantity is close to but not exactly lr*wd*t. We also report two methodological measurements. Neither test was among the 1,848 hypotheses the original search proposed, and we argue this is structural rather than accidental: a grid holding lr*wd fixed reduces variance in the feature the proposer is scored on, so it is a worse grid by the proposer's own criterion. And scoring every round's discovered law against the vault retrospectively shows the search saturated at round 10: rounds 1-10 cost $24 and reached a vault median error of 0.0792; rounds 11-114 cost $252 and reached 0.0784.