Papers

Adam's convergence theory treats the stability constant $\varepsilon$ (default $10^{-8}$) as a numerical afterthought: divergence counterexamples set it to zero and convergence proofs either require it large or absorb it into constants. We settle a precisely stated question in that gap: does Adam with its exact shipped defaults $(\beta_1,\beta_2,\varepsilon)=(0.9,0.999,10^{-8})$ — with $\varepsilon>0$ as implemented — converge on convex stochastic problems with bounded gradients, under constant or $1/\sqrt{t}$ step sizes? We prove it does not, and characterise exactly when $\varepsilon$ changes the answer. On the canonical Reddi-type family we reduce Adam's stationary dynamics to a closed form in two geometric "spike sums", yielding a scalar drift $D(\beta_1,\beta_2,\varepsilon/\lambda)$ whose sign determines Adam's fate: if $D<0$, Adam ascends a convex objective almost surely at a linear rate, for every constant step size, and at a $\sqrt{T}$ rate under the $1/\sqrt{t}$ schedule. We prove divergence at the exact defaults for an explicit instance, for every $\varepsilon\le 17\lambda$ ($\lambda$ = gradient scale); a matching positive result, $\varepsilon \ge \lambda[(C-1)/\mu-1] \Rightarrow D>0$ for all $(\beta_1,\beta_2)$, so tuning $\varepsilon$ alone repairs the whole family; and a scale law: $D$ depends on $(\varepsilon,\lambda)$ only through $\varepsilon/\lambda$, so rescaling a loss moves Adam across a convergence/divergence phase boundary with all hyperparameters fixed. Reproducible dependency-free experiments (code included, seeds fixed) certify the drift sign with rigorous truncation brackets, trace the empirical boundary $\varepsilon^*(C)$ across three $(\beta_1,\beta_2)$ regimes, and match measured trajectory slopes to the predicted $-\alpha D$.