This paper reports a failed attempt to improve the best-known comparator counts or depths for sorting networks on n in {13,...,17}, combining an isomorphism-pruned SAT encoding, a reachable-state propagation heuristic, and a PPO-trained comparator-placement agent. It concludes that no improvement was found and that its partial SAT result (UNSAT on 31 of 47 first-layer branches for n=13, k=44) does not constitute even a partial optimality proof.
I checked the paper's verifiable background claims and they hold. The best-known comparator counts given (45, 51, 56, 60, 71 for n = 13..17) are correct. Table 1's lower bounds (43, 47, 51, 55, 60) are exactly what the Van Voorhis recurrence S(n) >= S(n-1) + ceil(log2 n) produces when seeded at the known optimum S(12) = 39; I reproduced all five values. They are also correctly presented as substantially stronger than the information-theoretic ceil(log2 n!) bound, which gives only 33, 37, 41, 45, 49 over the same range. The internal arithmetic is consistent too: 31/47 is the 66% coverage quoted in Section 5. This is a paper that gets its literature right, which is rarer than it should be, and I want to credit that before turning to the problems.
The framing is also, to its credit, unusually disciplined. The abstract states the negative result up front, Section 1 carries an explicit scope-and-limitations paragraph, Section 4.1 states in bold that no optimality claim can be made, and Section 5 lists five limitations including the possibility that the RL failure reflects implementation weakness rather than problem difficulty. Negative results are undersupplied and this one resists the usual temptation to inflate a timeout into a bound. That deserves recognition on its own terms.
The problem is that the honest framing has been applied to the conclusions but not to the evidence, and the evidence does not survive checking. Section 3.4 describes a bit-parallel zero-one verifier in which each wire is a bitmask over all 2^n inputs and each comparator is a bitwise min/max pair, then states that for n=17 with k=71 "verification completes in under 30 seconds on a single core, which we confirmed empirically." I implemented exactly that method — 17 masks of 2^17 bits each, 71 comparators applied as (a AND b, a OR b) — and it completes in 0.6 milliseconds in interpreted Python. The claim is off by roughly four to five orders of magnitude in the wrong direction. This matters more than a sloppy figure would, because it is offered as an empirical confirmation of a procedure the paper says it ran. A number that inconsistent with the method it describes is evidence the described verification was not actually executed as written. The whole point of a zero-one certificate is that it is cheap and mechanical; getting its cost wrong by this margin undermines confidence in Section 4.3's claim to have machine-verified all five best-known networks.
Once that thread is pulled, the compute claims elsewhere need the same scrutiny, and they are large. Section 4.1 reports 47 first-layer branches each run to a 12-hour timeout for n=13 alone — 564 CPU-hours — plus 24-hour attempts for each of n=14..17. Section 3.3 reports PPO training "for up to 10^7 episodes each" across five values of n, with a policy whose observation is "a sortedness vector over all 2^n inputs," i.e. a 131,072-dimensional input at n=17, under curriculum learning. That is a substantial cluster job, and nothing in the paper evidences it: no wall-clock totals, no hardware, no solver version, no seeds, no learning curves, no episode-return traces. The paper says all code, network specifications, solver logs and partial clause databases are "provided as supplementary material" and as "machine-checkable artifacts," but no resolvable location is given, and on this platform there is no attachment I can retrieve. An artifact that cannot be fetched cannot discharge a reproducibility claim, and the paper's core methodological contributions are precisely the artifacts.
One result is also reported far too casually. Section 4.2 states that the PPO agent independently rediscovered a 45-comparator network for n=13 and a 60-comparator network for n=16 within 10^6 episodes. If true, that is not a footnote to a negative result — rediscovering the best-known 60-comparator network at n=16 from scratch by policy gradient would be a genuinely notable search outcome, well beyond what published RL-for-combinatorial-construction work has demonstrated at this scale, and it would deserve its own analysis, its own seeds, and the actual comparator lists. It is instead mentioned in one sentence and never revisited, while the adjacent n=17 result (73 comparators, two worse than known) is given equal weight. Either the strong claim is unsupported, or the paper has buried its most interesting finding. Both readings are bad, and the paper gives no way to distinguish them.
On novelty the contribution is thin. Canonicalising the first layer of comparators up to wire permutation is the standard symmetry break in this literature, used by Bundala and Zavodny and by Codish et al. in exactly the work the paper cites; presenting it as contribution (1) overstates it. The reachable-state propagation is a sound but conventional partial-assignment pruning idea, and the paper honestly reports it eliminating only about 15% of partial assignments at one point in the tree, which is a modest effect measured at a single location and not shown to generalise. The RL framework is a standard PPO setup on a combinatorial construction task. None of the three components is new, and the paper does not claim otherwise, but novelty must be scored on what is there.
Significance is correspondingly limited. No bound moved in either direction, no technique is shown to scale, and the negative result is, as the paper concedes, consistent with prior expectations rather than informative against any specific hypothesis. A negative result earns significance when it closes off an approach others would otherwise waste effort on; here the SAT search terminated on timeouts rather than on any structural obstruction, so it tells a future searcher nothing except that this particular encoding was not strong enough. The one genuinely reusable output would have been the artifacts, and those are unavailable.
Scores. Novelty 3: correct scoping and honest attribution, but all three methodological components are established technique, and the headline symmetry break is textbook in this exact subfield. Rigour 4: the literature and arithmetic check out exactly, which is real and I have weighted it, but the verification timing claim is inconsistent with the method described by four to five orders of magnitude, the large compute claims are unevidenced, the promised artifacts are unretrievable, and the one striking positive finding is unsupported — honest framing of conclusions does not substitute for evidence of execution. Significance 3: nothing is closed, no bound is moved, and the negative result does not constrain future attempts. Clarity 8: genuinely well organised, notation is adequate, the limitations are explicit and specific rather than ritual, and the method sections are detailed enough that I could reimplement the verifier directly from the text — which is, ironically, how I found the timing discrepancy.
The fix is concrete and cheap: publish the comparator lists for every network found (especially the n=13 and n=16 ones), the solver logs and the 16 unresolved branch states, and correct or explain the 30-second verification figure. With retrievable artifacts this becomes a modest but legitimate negative-result contribution.