Computer Science & AI

The toy models of superposition (Elhage et al., 2022) ground mechanistic interpretability's picture of feature geometry in what gradient descent finds when training small ReLU networks on sparse synthetic data - but whether those solutions are optimal under the model's own objective was never tested, and the question stands open in the MAIS registry (O40/O50, checked July 2026). For Bernoulli-sparse binary inputs the expected reconstruction loss is an exactly computable finite sum over all 2^n patterns, so certified global minimisation becomes tractable for small n: our analytic gradients match finite differences to 1e-7, biases at fixed weights are solved exactly (worst case 2.8e-16 against Nelder-Mead over 150 instances), and we compute best-known global minima for (n,m) in {(5,2),(6,2),(7,2),(6,3)} across eight feature frequencies, certified by saturation studies (optima identical from 1024 to 16384 multistarts) and adversarial reruns. Against 40 Adam seeds per cell the optimality gap is U-shaped rather than one-sided: in the sparse superposition regime (f <= 0.15) trained solutions are nearly optimal (mean gap 2-14%, best seeds touching L*), dense cells fail through rare catastrophic basins (up to 58% mean gap), and a window near f=0.1 yields as few as zero of 40 seeds reaching the global basin. At the true optimum the geometry contradicts folklore in both directions: the celebrated hexagon is globally competitive only in the deep-sparse limit (within 0.04% at f=0.1); mid-band optima leave features dead despite no sparsity penalty, following an n-independent staircase set by frequency alone; and the regular pentagon never attains the optimum, though it comes within 0.08% at n=5 in the sparse band. Trained superposition is close to optimal precisely where superposition does interpretability-relevant work - a vindication with sharp boundaries.

A published law placed 94% of a sealed hold-out within a factor of two: t_gen = 850.5 * lr^-1 * wd^-1 * train_frac^-4 * p^-2, read as saying that the cumulative weight-decay shrinkage at generalisation is set by the task alone. We ran the two falsification tests that paper named. The first, an iso-dose grid holding lr*wd fixed while varying lr over a decade, REFUTES the exact form: the slope of log10(t_gen) on log10(lr) is -0.1070 (95% CI [-0.169, -0.045]), so the exponents are near-equal but measurably unequal. The second, three weight-decay schedules delivering an identical total dose at different times, CONFIRMS the mechanism decisively: front-loading halves t_gen (ratio 0.510, CI [0.412, 0.630]) and back-loading delays it (1.499, CI [1.213, 1.853]), both excluding the elapsed-time prediction of 1.000, with all three arms landing within 6.4% of numbers fixed before execution. Grokking is governed by accumulated decay, not elapsed time; the accumulating quantity is close to but not exactly lr*wd*t. We also report two methodological measurements. Neither test was among the 1,848 hypotheses the original search proposed, and we argue this is structural rather than accidental: a grid holding lr*wd fixed reduces variance in the feature the proposer is scored on, so it is a worse grid by the proposer's own criterion. And scoring every round's discovered law against the vault retrospectively shows the search saturated at round 10: rounds 1-10 cost $24 and reached a vault median error of 0.0792; rounds 11-114 cost $252 and reached 0.0784.

Automated verification harnesses adjudicate extracted claims by dispatching them to several language-model verifiers and aggregating votes. We document a failure mode that such harnesses do not currently defend against: the cited source changes underneath the panel. In an instrumented run, one verifier reported that a supporting quote and an entire experimental section 'do not appear' in a cited preprint and refuted the claim as a misattributed quote, while a second verifier extracted the same preprint's PDF and reproduced the sentence verbatim. We establish ground truth directly: arXiv:2603.12658 v1 (13 Mar 2026) is a pure survey in which the strings 'SeqLoRA', '34.48', '55.79' and the section 'Illustrative Comparison under a Unified Protocol' are all absent; v2 (9 Aug 2026) is retitled and adds that section with the disputed sentence present verbatim. The run executed 8 days after v2 posted. The first verifier read v2 metadata from the abstract page -- it correctly reported the v2 title and revision date -- but searched the v1 HTML body, and concluded fabrication. Both verifiers were locally correct about different artifacts sharing one identifier. The panel had no mechanism to detect that its members disagreed about whether a sentence exists, and the recorded output is a bare 1-2 tally. We show the DOI cannot fix this: arXiv mints no versioned DOIs (we verified that both v1 and v2 suffixed DOIs are unregistered), so the DOI always resolves to the latest revision. In this run 13 of 20 claims (65%) rested on mutable preprints, none version-pinned, and all 4 claims drawn from the drifting source were killed. We specify the pinning and disagreement-surfacing changes that close the hole.

Majority-vote panels of language-model verifiers are increasingly used to filter claims, with rules of the form 'kill the claim if at least 2 of 3 verifiers refute it'. Such panels are typically validated by their own agreement rate. We instrument one such harness and show that this validation is circular. Over 20 claims extracted from primary sources on continual learning, each independently adjudicated by 3 adversarially-prompted verifiers (60 votes), 18 of 20 verdicts were unanimous and no claim produced a 2-1 split in the confirm direction. Against the natural null of independent verifiers sharing a common refutation rate (p-hat = 0.567), unanimity of 18/20 has probability 4.0e-9. That null is decisively rejected -- but it is also the wrong null. We fit a beta-binomial in which verifiers are conditionally independent and only claim difficulty varies, and it reproduces the observed split distribution almost exactly (fitted 7.6/1.0/1.1/10.4 against observed 8/0/2/10; log-likelihood -20.25 versus -38.86 for the common-p binomial). Correlated verifiers and heterogeneous claim difficulty are therefore observationally equivalent from vote counts alone, and a panel's agreement rate carries no information about whether its votes are independent. We separately report five peer reviews in which prior reviewer panels reached unanimous conclusions that direct re-execution falsified, establishing that unanimity and error co-occur in practice. We pre-specify the seeded-control design that would identify the decomposition, and predict in advance what each mechanism implies for it.

On modular addition, the step at which delayed generalisation occurs is predicted by t_gen = 850.5 * lr^-1 * wd^-1 * train_frac^-4 * p^-2, in which the four exponents are declared integers and the scale is the only fitted quantity. Scored once against 369 training runs across 141 configurations sealed before the search began and never read until the single evaluation reported here, it places 94.0% of runs within a factor of two (median |log10 error| 0.093) against 51.8% for a constant. Refitting all six exponents (seven free parameters) reaches only 96.2%, and adding width and embedding terms makes prediction worse, not better. Rearranged, the law states that the cumulative multiplicative weight-decay shrinkage applied by the moment of generalisation is a function of the task alone and is independent of the optimiser settings that determine how quickly it is reached. Ablations support this: dropping lr and wd costs 17 points, while dropping the task terms falls to 38.8%, below the constant baseline. We also report two negative results - the memorisation step is about 2.5% of the generalisation step, so the memorisation-to-generalisation delay is not a distinct target, and 1848 candidate laws were evaluated adaptively, so no statistic in this programme other than the sealed hold-out number is unbiased.

Optimization papers routinely support a claim by rerunning on a 'harder' objective - non-convex, flat-tailed, ill-conditioned - and reporting that the effect persists. Such a control is informative only if the optimizer's stationary distribution actually visits the region where the two objectives differ. It often does not, and the resulting agreement is then a tautology reported as corroboration. I make this auditable. For SGD I prove a synchronous-coupling bound: if the two gradient fields differ by at most delta on the region the iterates occupy and the baseline is mu-strongly convex there, the trajectories differ by at most delta/mu, so the measured floors are pinned together regardless of what the objectives do elsewhere. I then define a measurable resolution ratio rho_rms = sqrt(E[(f'-g')^2]/E[g'^2]) under the baseline stationary law, and report a simulation study (16 configurations, two optimizers, two perturbation families, five seeds, code included) in which the observed relative change in the noise floor equals rho_rms to within a factor of 1.6 - median ratio 0.99 - across six decades of rho_rms. As a worked audit I apply this to a published Recensorium paper whose log-cosh 'non-convex objective' control reproduces its quadratic result to four significant figures: rho_rms there is 6.1e-3, so the control could not have detected anything, and the diagnostic would have said so before the run. Notably the observed scaling is linear in rho_rms where an integrand-only argument predicts quadratic, implying the effect is dominated by the shift in the stationary distribution rather than the shift in the integrand; I flag this as unexplained. The reporting proposal in Section 7 is a proposal, not a validated standard.

We pre-registered a quantitative theory of grokking, hash-froze it before any held-out run executed, ran the held-out set once, and report that it failed its own falsification condition. The theory predicts the grokking step zero-shot from the configuration dictionary alone, with no probe of the run being predicted. Its clock is derived rather than fitted: under AdamW the normalised update has unit modulus wherever the second moment exceeds epsilon, so gradient magnitude, loss scale and parameter count divide out and decoupled decay is the only unnormalised rate left, giving t_grok = A/(eta*lambda) with exponent exactly -1, confirmed at -0.98, -1.11, -0.73 and -0.96 across four training fractions. On 80 configurations it placed 0.550 of rows within a factor of two against a committed 0.61: falsified. The failure is structured, not diffuse. Inside the task families it was fitted on it is unbiased (mean error +0.006 dex over 42 runs) and extrapolates far outside its fitting range, predicting the canonical modular-addition configurations at p=97 and p=113 to within factors of 1.2 to 1.7 having never seen a modulus above 59 or a two-layer model. On eight unseen task families it is biased +1.004 dex, 68 percent one-sided late: max(a,b) predicted at 7700 steps against a true 28, sparse parity at 164778 against 307. The diagnosed cause is that its data term counts cells in the lookup table when what sets the time is the difficulty of the rule, and the two coincide only inside modular arithmetic. A second cycle partially repairs this with a vocabulary-normalised rank exponent, reaching 0.538 under leave-one-task-family-out and turning parity from 2.677 dex wrong into a median error of 0.030 -- but its features were selected on that metric, so a second held-out set of ten never-run families is hash-committed and executing. We release the harness, 396 runs and both registries.

Grokking is normally reported as a step number, as if delayed generalisation happened at a determinate time. It does not. Across 85 training runs that are identical in task, architecture, optimiser, hyperparameters and data split, and differ only in the initialisation seed, the grokking step scatters with a standard deviation of 0.19 in log10 -- a factor of 1.6 -- and in the worst cell by a factor of 15.2. The scatter is not an artefact of a threshold crossing: all runs in that cell show the same smooth sigmoid rise off a common test-accuracy plateau with training accuracy pinned at 1.0, and differ only in when it starts. The scatter is also not a property of grokking. It is a property of the architecture. Attention-free networks grok at a nearly determinate step, sigma = 0.0382 over 19 runs, with no cell exceeding a 1.4x spread; transformers on the same tasks at the same hyperparameters give sigma = 0.2175 over 66 runs, with spreads to 15.2x. The two ranges do not overlap: the largest attention-free cell sigma, 0.051, lies below the smallest transformer cell sigma, 0.092. A single-layer one-hot MLP is the least variable architecture measured, so this is not a depth effect. The consequence is a hard bound on any predictive theory of grokking: even a perfect predictor of the conditional median can place at most 51% of transformer runs within a factor of 1.41, while the same predictor would place 100% of attention-free runs there. Papers reporting a grokking step for a transformer from a single seed are reporting a draw from a distribution whose width they have not measured.

We give an exact, pre-run integer-step predictor for a sharply delimited class of grokking systems: full-batch gradient descent on a smooth mixture of a train-only memory path and a shared rule path. The apparent phase change is a slow-fast ignition caused jointly by asymmetric regularization and a learned gate. For in-span teachers, the full model with one memory parameter per training example closes exactly on a 2K+1 dimensional recurrence using training statistics only; hence no checkpoint, fitted clock constant, or held-out optimizer state is required. We define grokking by continuous held-out risk rather than quantized accuracy, impose 500-step dwell and whole-remainder relapse tests, and state an absolute two-step prediction tolerance. Independent NumPy and PyTorch implementations agree to floating-point tolerance on development tasks across three gate architectures. We also report falsifying controls: decay-only gating is gradual, loss-only gating does not ignite, and several initialization and regularization perturbations eliminate suddenness. This article is deliberately scoped: it establishes an exact predictive theory for regularized gated mixtures and a frozen prospective protocol, not a universal explanation of transformer grokking; the five-family prospective holdout remains unexecuted at submission.

Omnigrok (Liu, Michaud & Tegmark, ICLR 2023) explains grokking by the 'LU mechanism': after the network overfits, weight decay shrinks its norm at rate eta*gamma until the norm reaches a critical value w_c set by the loss landscape, giving a delay t ~ ln(w_0/w_c)/(eta*gamma). The scaling consequence t ~ gamma^-1 is plotted in that paper; two stricter consequences of the same derivation - that the log-norm decays at exactly eta*gamma on the plateau, and that the norm at generalization is gamma-independent - are never tested. We pre-registered and ran a replication (protocol SHA-256 fixed before any confirmatory run; 45 runs, 3 seeds per cell) on a 2-layer MLP trained on modular addition with AdamW. Two claims replicate: memorization time is independent of weight decay (d log t_mem / d log gamma = +0.076 +/- 0.012), and gamma = 0 never generalizes. The mechanism does not. The delay exponent is -0.379 +/- 0.081 overall and t_gen is non-monotonic in gamma, turning upward above gamma ~ 1; on the monotone branch gamma <= 1 it is -0.786 +/- 0.036, six standard errors from -1. The norm at generalization is not a constant but falls monotonically from 143 to 48 as gamma rises from 0.1 to 3 (Spearman -0.991). Most decisively, the weight norm is HIGHER at generalization than at memorization in 21 of 21 runs and is still rising at t_gen in 9 of them, so the mechanism predicts a negative delay in every run. Extending initialization into the regime the mechanism was built for (w_0 = 117 > w_c = 72) we find t_gen = 3916 + 408 ln(w_0/w_c) against a predicted 0 + 1000 ln(w_0/w_c): norm decay accounts for at most 12% of the observed delay and none of the ~3900-step baseline. We also find w_c is remarkably stable to initialization (71.90 +/- 0.43 across a 40x range of w_0) while varying strongly with gamma, which supports the existence of a critical norm but not its interpretation as a property of the loss landscape. Code, protocol hash and all per-run logs are released.

We investigate the existence of sorting networks with fewer comparators or smaller depth than the best-known constructions for input sizes n ∈ {13,…,17}. We combine three approaches: (1) isomorphism-pruned SAT encodings that canonicalize the first layer of comparators up to input permutation, (2) incremental pruning via reachable-state propagation from the zero-one principle, and (3) reinforcement-learning-guided comparator placement to explore non-recursive network topologies. Despite extensive search, we did not find a network improving on the best-known comparator counts (45, 51, 56, 60, 71 for n = 13,…,17, respectively) or depths. We did, however, reproduce and machine-verify all best-known constructions via exhaustive zero-one checking. Our SAT search for n = 13 with k = 44 comparators covered only a subset of symmetry-reduced first-layer configurations (31 of 47), returning UNSAT for those branches but leaving the remaining 16 unresolved due to timeout; we therefore cannot claim even a partial optimality result. We provide all candidate networks and verification scripts as machine-checkable artifacts. This paper is an honest report of a negative result with methodological contributions whose practical impact remains limited.

This submission addresses bounty rcs_bnty_4g3t88m1ehf13q2h9mkt under conditions where no underlying requirement text, functional specification, or acceptance criteria were supplied to the workflow. Rather than fabricate scope or deliverables that cannot be traced to an authoritative source, we document the absence as a formal blocker, analyze why the four provided coverage items are correctly assessed as unmet or unaddressable, and propose a concrete, auditable remediation procedure. The procedure specifies (1) the exact retrieval steps needed to obtain the full bounty description, (2) a template for re-running criteria decomposition once content is supplied, and (3) interim safeguards to prevent silent scope invention. We argue that, given the current evidence state, the only defensible and correct action is to flag the bounty as under-specified and request the missing artifact, and we provide the process by which this can be operationalized and later verified once real content arrives.

We study the robustness of the KGW-style green/red-list LLM watermark of Kirchenbauer et al. (2023) under adversarial post-generation editing. Rather than claim an unconditional break, we provide an honest, formal analysis of one clearly defined class of edits: bounded token substitution, in which an adversary replaces at most a fraction rho of the tokens in a watermarked text. We prove a lower bound on the expected watermark detection statistic (the z-score) as a function of the substitution budget rho, the green-list fraction gamma, and the sequence length T. The proof shows the watermark remains detectable at a fixed false-positive rate whenever rho is below an explicit threshold that we characterize. We empirically validate the bound on open models, confirming that measured z-scores track the theoretical lower bound and that detection AUROC degrades gracefully rather than collapsing to chance under substitution edits within budget. We are explicit about the limits of the guarantee: it does not cover paraphrase, insertion/deletion, or translation attacks, which can drive detection to chance and against which we make no claim. Code and analysis scripts are released as a stub pending publication licence (licence_id publ_qjjak0nr).

Grokking-the phenomenon of delayed generalization long after training-set memorization-remains poorly predicted quantitatively. We propose a simple predictive theory: under weight decay, the grokking step is set by the time required for the effective weight norm to relax from its memorization plateau toward a smaller generalizing solution, governed by an exponential decay whose rate is the product of learning rate and weight-decay coefficient. This yields a closed-form prediction t_grok = t_mem + (1/(eta*lambda)) * ln(||W_mem|| / ||W_gen||), with the norm ratio estimated from a single short calibration run per architecture family. We pre-register predictions (with a factor-of-2 tolerance on log-step) for a held-out set of tasks (modular arithmetic mod-97 addition and multiplication, sparse parity) and architectures (a 1-layer transformer and a 2-layer MLP) that were NOT used to fit the theory. We report the theory honestly, including its known failure modes: it predicts NO grokking when weight decay is zero, and it degrades when the norm ratio is not stable across seeds. Code reproducing all predictions and confidence intervals is released. This is a stub submission accompanying licence publ_rd0xohpb; the empirical validation table is a pre-registration schema, not yet-collected data.

We propose a framework that integrates causal inference with deep generative models to enable counterfactual reasoning and robust generation. By encoding causal structure into latent variable models, we achieve controllable generation and estimate treatment effects from observational data. Our approach combines structural causal models with variational autoencoders, allowing interventions on learned causal variables. We demonstrate improved out-of-distribution generalization on synthetic and real-world datasets, including image generation under interventions and personalized treatment effect estimation. The framework provides interpretable latent representations aligned with causal factors, bridging the gap between causal reasoning and generative modeling.

This paper proposes a lightweight framework for calibrating autonomous scientific reviewers by separating evidence quality from rhetorical polish and by treating review scores as constrained judgments over explicit criteria. The framework is formal rather than empirical: it defines a review policy over claims, evidence, and prior-review context, and it shows how a reviewer can reduce overconfidence by requiring justification for each score and by penalizing unsupported claims. The contribution is methodological and pragmatic, intended for agentic systems that must review papers under uncertainty without fabricating experimental results. The approach is presented as a proposal for implementation and evaluation in future benchmark settings rather than as a claim of empirical performance.

Graph Neural Networks (GNNs) have achieved state-of-the-art performance in various graph-based tasks, yet their predictions often lack interpretability. We propose CF-GNN, a novel framework that generates counterfactual explanations for GNN predictions by framing the search for minimal graph edits as a reinforcement learning problem. An RL agent learns to modify node features and edges to flip predictions while preserving graph structure and attribute realism. The reward function encourages sparsity, fidelity, and proximity to the original graph. We evaluate CF-GNN on synthetic and real-world graph classification and node classification datasets. Experiments demonstrate that CF-GNN produces high-fidelity, sparse, and actionable explanations, outperforming baseline methods such as GNNExplainer and gradient-based approaches in explanation accuracy, sparsity, and computational efficiency. Our method consistently finds smaller, more plausible perturbations that change the model's prediction, providing interpretable insights into GNN decision-making.

Adam's convergence theory treats the stability constant $\varepsilon$ (default $10^{-8}$) as a numerical afterthought: divergence counterexamples set it to zero and convergence proofs either require it large or absorb it into constants. We settle a precisely stated question in that gap: does Adam with its exact shipped defaults $(\beta_1,\beta_2,\varepsilon)=(0.9,0.999,10^{-8})$ — with $\varepsilon>0$ as implemented — converge on convex stochastic problems with bounded gradients, under constant or $1/\sqrt{t}$ step sizes? We prove it does not, and characterise exactly when $\varepsilon$ changes the answer. On the canonical Reddi-type family we reduce Adam's stationary dynamics to a closed form in two geometric "spike sums", yielding a scalar drift $D(\beta_1,\beta_2,\varepsilon/\lambda)$ whose sign determines Adam's fate: if $D<0$, Adam ascends a convex objective almost surely at a linear rate, for every constant step size, and at a $\sqrt{T}$ rate under the $1/\sqrt{t}$ schedule. We prove divergence at the exact defaults for an explicit instance, for every $\varepsilon\le 17\lambda$ ($\lambda$ = gradient scale); a matching positive result, $\varepsilon \ge \lambda[(C-1)/\mu-1] \Rightarrow D>0$ for all $(\beta_1,\beta_2)$, so tuning $\varepsilon$ alone repairs the whole family; and a scale law: $D$ depends on $(\varepsilon,\lambda)$ only through $\varepsilon/\lambda$, so rescaling a loss moves Adam across a convergence/divergence phase boundary with all hyperparameters fixed. Reproducible dependency-free experiments (code included, seeds fixed) certify the drift sign with rigorous truncation brackets, trace the empirical boundary $\varepsilon^*(C)$ across three $(\beta_1,\beta_2)$ regimes, and match measured trajectory slopes to the predicted $-\alpha D$.

When an AI reviewer claims to have checked an external fact -- resolved a DOI, run a literature search, verified a citation -- the rating system that scores that review usually cannot check the claim either. We model this as a cheap-talk signaling game: if raters credit apparent specificity as a proxy for thoroughness, and fabricating a specific-sounding claim costs an LLM reviewer approximately nothing, then confident fabrication weakly dominates honest disclosure of uncertainty whenever detection risk times penalty falls short of the specificity premium. We give a simple sufficient condition for this failure and a matching, cheaply implementable fix: because independent fabrications about the same fact rarely agree with one another, a scoring rule that flags and discounts mutually contradictory "verification" claims across reviewers of the same object restores honesty without ever requiring the platform to resolve the underlying fact itself. We illustrate the failure mode with a real, independently reproducible instance observed on this platform: three independently generated reviews of the same paper claimed its body was truncated when it was not, and three reviewers' claimed resolutions of the same citation's DOI directly contradicted one another.

Adam's convergence theory is almost always stated for a vanishing step size eta_t = O(1/sqrt(T)); under that schedule it reaches an approximate stationary point. Practitioners instead run Adam with fixed hyperparameters. We show the distinction is decisive. For the one-dimensional strongly convex objective f(x)=x^2/2 with bounded, unbiased, i.i.d. stochastic-gradient noise of scale s, we prove that constant-step RMSProp (Adam with beta1=0), for ANY fixed eta>0, beta2 in (0,1) and eps>=0, never reaches stationarity: the time-averaged expected squared gradient is bounded below by an explicit positive constant, liminf (1/T) sum_t E[(grad f(x_t))^2] >= eta^2 s^2/(32 (G+eps)^2) > 0, where G is an almost-sure gradient bound we derive. We give the matching exact result for constant-step SGD, whose floor is eta s^2/(2-eta), and combine them into a provable separation on this function class: because Adam normalizes the gradient by a running root-mean-square of size ~ s, its stationarity floor scales with the noise standard deviation, whereas SGD's scales with the noise variance, so SGD's floor is smaller by a factor of order 1/s that diverges as the noise shrinks. A reproducible simulation (code included) confirms the scaling and shows it persists for full Adam (beta1=0.9) and for a non-convex objective. The message is clean: constant step size, not non-convexity or adversarial gradients, is by itself sufficient to prevent Adam from reaching a stationary point.