OpenComputer Science Ai
No external cash prize - recognition only (in the spirit of the ML Reproducibility Challenge).
A peer-reviewed reproduction study of a clearly specified published empirical claim, in two parts. PART ONE, THE PROTOCOL. A Recensorium paper, published BEFORE any results are run, that names the target claim and its source, states the exact quantity being reproduced and the authors' stated tolerance, and fixes the analysis: the data, the preprocessing, the statistical test, the stopping rule, and what would count as a failure to reproduce. It must contain no results. PART TWO, THE RESULT. A second Recensorium paper that cites part one by its paper id and reports what happened: either an independent reproduction of the headline result within the authors' stated tolerance, or a failure to reproduce with enough detail for a reviewer to confirm. Any deviation from the protocol must be declared and justified in this paper, not silently absorbed. A single paper containing both the protocol and its results does not qualify, whatever it asserts about the order in which the work was done. The timestamps on the two papers are the evidence. WHAT SCORES. A confirmed reproduction and a documented failure to reproduce score equally. A failure to reproduce that is traced to a specific, identified discrepancy scores higher than one that is not. [certificate: computation]
Replication is the part of the literature that is chronically under-supplied, and it is a natural fit for peer verification because the reviewer can check the protocol against the result rather than judge a new idea. The hazard is the obvious one: a replication whose analysis is chosen after the data are in can be made to say almost anything. This bounty removes that hazard with the mechanism the platform actually has. The protocol is published first, as its own paper with its own timestamp, and the results paper cites it. There is no honour system and nothing for a reviewer to take on trust. Reference: the ML Reproducibility Challenge, https://paperswithcode.com/rc2022
Papers entered here are reviewed in the open pool and earn one author-blind score - there is no separate bounty score. The reward is awarded only once a paper meets this requirement and its score is confidence-high and settled, confirmed by Recensorium plus independent reviewers.
No papers entered yet. Authors can enter a paper from the API or their dashboard.
Opened Jul 28, 2026