Source-linked AI summary
When a Data Artifact Isn't a Shortcut: Causal Auditing of Synthetic RLVR Corpora
Esther Xin
TL;DR
When correct options come from real corpus text and distractors are model-generated, provenance may become a statistical cue alongside correctness, raising the question of whether policies can rely on it. The audit reports exploitation gaps per policy, overall and by domain; a larger treatment-arm gap supports the hypothesis, while an equal or smaller gap argues against it at the tested budget. 0.021 versus 0.027: the unmodified-data policy’s exploitation gap was not larger than the control’s, despite a detectable surface signal.
Problem
When correct options come from real corpus text and distractors are model-generated, provenance may become a statistical cue alongside correctness, raising the question of whether policies can rely on it.
Source: §1 · p. 1Method
The audit reports exploitation gaps per policy, overall and by domain; a larger treatment-arm gap supports the hypothesis, while an equal or smaller gap argues against it at the tested budget.
Source: §3.4 · p. 4Results
Takeaways & Limitations
The authors recommend pairing cheap characterization with causal intervention and reporting whether distractor-generation mechanisms vary by domain.
Source: §5 · p. 5The audit uses a modest 60-step budget on one GPU and one seed per arm, so it does not establish whether longer training would produce a shortcut.
Source: §6 · p. 5
Abstract
from arXiv · showhide
Several recent pipelines build RLVR training data by masking a span of real corpus text and asking a language model to invent plausible wrong answers around it. The correct option is therefore genuine human prose; every distractor is synthetic. Correctness and provenance become entangled, and a policy could in principle learn the second instead of the first. We audit that possibility in GooseReason-0.7M. First we ask whether the asymmetry is visible at all: a classifier reading only five surface statistics (never the meaning) reaches AUROC 0.562 over 315,499 options, barely above chance. The aggregate hides something, though. Code sits at 0.416, below chance, and manual inspection explains why: code distractors turn out to be single-operator mutations of the gold answer rather than freely written alternatives, so the two classes are nearly identical by construction. Detecting a signal is not the same as showing a model uses it, so we then run an intervention. We build a paraphrase-matched control corpus, hold training-set size identical across arms, and train two policies under one fixed budget. The exploitation gap does not favour the unmodified-data arm: 0.021 against 0.027 for the control. Under our budget, in other words, a detectable artifact went unexploited. We think that dissociation, along with the domain-specific construction finding, is worth knowing for anyone curating corpora of this kind, and we release the audit as a mostly CPU-only protocol.
1 Introduction
The paper asks whether synthetic RLVR data creates provenance cues that policies can exploit, separating cue detection from evidence of policy use. Its audit finds a weak surface signal but no larger exploitation gap for the unmodified-data policy under the tested budget.
Motivation
- The study targets an unexamined distinction: whether real-text gold options and model-written distractors leave a provenance signal beyond the intended correctness signal.A policy could in principle use statistical residue such as token rarity, rhythm, or syntax instead of reasoning.Source: §1 · p. 1
Approach
- The audit separates surface-only characterization from a paraphrase-matched intervention that tests whether policies rely more on authenticity-correlated cues.The two arms use matched difficulty and training-set size under one budget.Source: §1 · p. 1
Findings
- A surface-only classifier is near chance overall (AUROC 0.562, n = 315,499); math and STEM are similar (0.584 and 0.583), while code is below chance (0.416), attributed to minimal mutations.Source: §1 · p. 2
- The unmodified-data policy’s exploitation gap is 0.021 versus 0.027 for the matched control, so the treatment gap is not larger.The intervention removes the provenance asymmetry while holding difficulty and training-set size fixed.Source: §1 · pp. 1–2
Contributions
- The paper contributes a cheap semantics-blind diagnostic, a GooseReason-0.7M audit, and a causal protocol for testing whether detected asymmetries are exploited.Its evidence shows that detection and exploitation can come apart at the tested budget.Source: §1 · p. 2
2 Related Work
The related work distinguishes authenticity cues in option text from other candidate shortcuts and from forensic provenance detection. The paper positions its intervention as a complementary audit of whether a measurable corpus artifact is actually used.
Synthesizing RLVR data from unverifiable text
- The paper’s audit is orthogonal to downstream-accuracy evaluations of masked-span synthesis, which do not characterize distractor statistics.A dataset can improve downstream accuracy and still carry a construction asymmetry worth documenting.Source: §2 · p. 2
Distractor design and position bias
- Weak distractors and answer-position bias are distinct shortcut axes from whether option text carries a real-versus-generated signature.The paper presents its protocol as complementary to audits of distractor strength and position.Source: §2 · p. 2
Provenance detection
- Provenance detection here asks whether corpus construction leaves a trace separating option classes, not whether a model has synthetic-data training history.The cited forensic work asks the latter question.Source: §2 · p. 2
Shortcut learning and verifier reliability
- Shortcut-learning research establishes artifact exploitation, while this paper targets the converse case of a measurable artifact that appears unused.Its target is a corpus artifact, rather than a verifier or reward flaw.Source: §2 · pp. 2–3
3 Method
The method audits GooseReason-0.7M in two stages: it first tests whether surface features distinguish real and generated options, then intervenes with matched controls to test policy reliance. Domain-specific transformations preserve the code items’ injected faults while neutralizing stylistic authenticity cues.
3.1 Data and construction mechanism
- GooseReason-0.7M items restore a masked reasoning-critical span as the gold option and use model-generated plausible-but-wrong alternatives as distractors.The corpus spans mathematics, code, and general STEM reasoning.Source: §3.1 · p. 3
3.2 Stage 1: surface-feature characterization
- Five meaning-blind statistics, including rare-token rate, burstiness, entropy, and POS-tag divergence, feed classifiers predicting option provenance.The classifiers use a stratified held-out split and report AUROC overall and by domain.Source: §3.2 · p. 3
3.3 Stage 2: causal intervention
- The control corpus transforms gold and distractor options alike; math and STEM receive semantic paraphrases, while code receives restyling to preserve injected faults.A manual audit found one-token gold-code mutations in all ten sampled code items, so generic paraphrasing could repair the fault.Source: §3.3 · p. 3
3.5 Reward, training, and compute
- Treatment and control policies train with identical per-domain training-set caps, using GRPO and a rule-based exact-match reward.Both fine-tune from Qwen3-1.7B with 4-bit quantized low-rank adapters.Source: §3.3 · p. 3, §3.5 · p. 4
3.4 Evaluation protocol
- Both policies face original, neutralized, and adversarial test variants; the artifact exploitation gap compares neutralized with adversarial accuracy.A larger treatment-arm gap is the hypothesized signature; an equal or smaller gap is evidence against it at the tested budget.Source: §3.4 · pp. 3–4
4 Results
The treatment-trained policy shows no larger overall exploitation gap than the control, while surface-only provenance detection is weak overall and uneven across domains. In code, a manual audit links below-chance classification to distractors that are near-identical to gold answers by construction.
4.1 A weak signal, unevenly distributed
- 0.562 overall AUROC is close to chance; math and STEM are similar at 0.584 and 0.583, while code is the only domain below chance.The code result reflects one-token mutations of gold answers, making the classes nearly identical in surface statistics by construction.Source: §4.1 · p. 4, Table 1 · p. 4, Figure 1 · p. 4
4.2 No exploitation under RLVR training
- 0.021 for treatment versus 0.027 for control: the overall exploitation gap does not favour greater shortcut reliance after training on unmodified data.The domain breakdown likewise shows no treatment gap substantially exceeding control; math has a negative gap in both arms at similar magnitudes.Source: §4.2 · pp. 4–5, Table 3 · p. 4, Figure 2 · p. 5
5 Discussion
The discussion distinguishes detecting provenance signals from showing that policies exploit them, and finds that the code-domain construction mechanism requires domain-specific inspection. It recommends pairing characterization with intervention and treating authenticity as distinct from other shortcut axes.
5 Discussion
- A detectable authenticity signal did not produce a measurable increase in shortcut reliance for the original-data policy versus the control.The authors recommend pairing characterization with causal intervention, where compute permits, before concluding a detected asymmetry needs engineering around.Source: §5 · p. 5
- Code distractors were found to be bug-injection mutations rather than free rewrites, a domain-specific mechanism not captured by one aggregate statistic.The finding emerged by combining the below-chance classifier result with inspection of individual items.Source: §5 · p. 5
- Authenticity is a separate shortcut axis from distractor strength and answer position, so this protocol complements rather than replaces those audits.The discussion notes that a policy may exploit weakness or position bias without showing an authenticity-shortcut gap.Source: §5 · p. 5
6 Limitations
The conclusions are bounded by a modest training budget, limited replication and per-domain samples, one dataset construction family, and multiple-choice evaluation. The paper leaves open whether the pattern holds in other settings.
6 Limitations
- The 60-step RLVR budget is below production scale, so the null gap does not rule out shortcuts emerging with longer training.Each arm has one seed, and 300 test items per domain are insufficient to pin down very small per-domain effects.Source: §6 · p. 5
- The audit covers one corpus and masked-span-and-distractor construction family, leaving generalization to other construction methods open.The reward and evaluation target multiple-choice RLVR; the same asymmetry could behave differently or not arise in free-form or code-execution settings.Source: §6 · pp. 5–6
7 Conclusion
The conclusion reports a detectable authenticity signal but no measurable increase in shortcut reliance under the tested matched-control comparison. It presents the released audit materials as a basis for applying the protocol to other corpora and larger budgets.
7 Conclusion
- The code-domain signal arose from near-identical bug-injection distractors, while the original-data policy did not measurably increase shortcut reliance against the matched control.The authors release the characterization protocol, paraphrase-matched control construction, and per-domain evaluation splits for audits on other synthetic RLVR corpora and larger budgets.Source: §7 · p. 6
A Reproducibility Details
The reproducibility details report the Stage 2 training, evaluation, and compute configuration, including exactly matched training-set sizes across treatment and control. The settings are released with the code for reproduction or extension.
A Reproducibility Details
- Table 4 reports the Stage 2 training, evaluation, and compute configuration, including matched per-domain sample sizes across arms.Source: A Reproducibility Details · p. 6
- The table caption states that train-set size is matched exactly to avoid confounding authenticity with data quantity.Source: Table 4 · p. 7
- All settings are released with the code so the run can be reproduced or extended to a larger compute budget.Source: Table 4 · p. 7