PaperlayerView exact revision on arXiv

Source-linked AI summary

Same Answer, New Box: How Much of RLVR's Out-of-Domain Gain Is Answer Format?

Zongye Lyu

arXiv:2610.20639v1cs.LGcs.CL

TL;DR

Out-of-domain RLVR gains are often read as evidence of transfer, but answer extractors can count a change in answer placement as a gain. The paper re-scores the same base and RL outputs with a fixed extractor, finding that 82% of reported gains across four released 7B SimpleRL pairs are format gain.

  • Problem

    Out-of-domain RLVR benchmark gains are read as signs of transfer, but extractors can report gains when RL changes answer placement without improving the answers.

    Source: §1 · p. 1

  • Method

    The paper re-reads outputs from released base–RL pairs with a fixed extractor that accepts stated answers in either the box or text, then compares its gain with the harness-reported gain.

    Source: §1 · pp. 2–3

  • Results

    82% of the out-of-domain multiple-choice gain reported for four released 7B SimpleRL pairs is format gain.

    Source: §1 · p. 2

  • Takeaways & Limitations

    A reported gain partly depends on the extractor that reads the answers, and stored outputs can be rescored to measure this without rerunning RL.

    Source: §1 · p. 3, §7 · p. 10

    The 82% is not an estimate of published release scores because the authors did not reproduce those evaluations.

    Source: §6 · p. 9

Abstract

from arXiv · show

Reinforcement learning with verifiable rewards (RLVR) is credited with gains beyond the domain its rewards check, such as higher MMLU accuracy after training on mathematics. Such accuracies are usually read out of the model's text by an answer extractor. A model that learns to write its answer where the extractor looks can therefore score higher without answering better. We measure how much of the gain reported by extractors copied from public code arises this way. We generate outputs from a base model and its RL model and score them twice: with those extractors, and with one extractor of our own that reads a stated answer whether it is in the answer box \boxed{} or in the text. On four released 7B SimpleRL base-RL pairs, 82% of the out-of-domain multiple-choice gain that these extractors report disappears under ours: it is a gain in answer format, not in answers. The base models already state most of their answers in text, and RL moves them into the box, where these extractors look first. Independent LLM readers, in place of our extractor, put the figure at 77%. With a prompt that itself asks for the box, our extractor puts it at 79%. On a 32B pair trained with another recipe, almost none of the reported gain is format gain: its base model boxes its answers when asked, and its RL model rarely does when not. A reported gain is partly a property of the extractor that read it, and re-scoring stored outputs with a second extractor shows how much, without re-running RL.

1 INTRODUCTION

The paper asks how much out-of-domain RLVR gains reflect better answers rather than answer placement, and re-scores the same outputs with a fixed extractor. Across four released 7B SimpleRL pairs, most of the reported gain is attributed to format.

1 INTRODUCTION

  • On ARC-Challenge, the harness extractor reports 30.6 points of gain, while the fixed extractor reads 9.8 from the same outputs.Both models answer B on the illustrated item; the extractor that reads only the box misses the base model’s stated answer.Source: §1 · p. 1, Figure 1 · p. 2
  • The method re-reads outputs from released base–RL pairs with a fixed extractor and compares its gain with gains from harness extractors copied from public code.The measurement uses stored outputs and requires no RL run.Source: §1 · pp. 2–3
  • 82% of the out-of-domain multiple-choice gain reported for four released 7B SimpleRL pairs is format gain.The base models state most answers in text, while RL moves them into the box where the harness extractors look first.Source: §1 · p. 2
  • Independent LLM readers find 77% format gain, and the result shifts only a few points under other extractor choices, prompts, or leave-one-out comparisons.Pooling every comparison with a nonzero reported gain lowers the share to 75%.Source: §1 · p. 3
  • These are readings of generated checkpoint outputs, not reproduced published scores; on one 32B pair with gains to split, almost none of the gain is format gain.The 82% result therefore is not an estimate for the releases’ published numbers.Source: §1 · p. 3

2 SPLITTING A REPORTED GAIN

The paper separates a harness-reported gain into the gain a fixed extractor reads and the remaining format gain. It defines the fixed extractor and restricts pooled shares to comparisons where the estimate is interpretable.

2.1 TWO GAINS FROM THE SAME OUTPUTS

  • Format gain is the harness-reported gain minus the fixed-extractor gain; the format share is that difference divided by the reported gain.The fixed extractor applies the same rules to both models, so its gain is the gain in answers as it reads them.Source: §2 · p. 3, §2.1 · p. 3
  • For pooled results, gains are summed before division, weighting comparisons by their reported gain so near-zero gains cannot dominate.The pooled format share is the sum of format gains divided by the sum of reported gains.Source: §2.1 · p. 3

2.2 THE FIXED EXTRACTOR

  • The fixed extractor reads only explicitly stated answers, and an item with no found answer counts wrong for either model rather than being dropped.It finds answers in 87% of base outputs and 88% of RL outputs on the comparisons behind the 82%.Source: §2.2 · pp. 3–4

2.3 WHICH COMPARISONS WE POOL

  • Pooling requires a clearly positive harness gain and fixed-extractor answers in at least half of each model’s outputs, with extraction rates within 20 points.The 95% intervals resample items within benchmarks and do not cover variation between pairs or benchmarks.Source: §2.3 · p. 4

3 SETUP

The setup compares eligible released base–RL pairs across benchmarks, harness extractors, and two prompts. Four pairs provide the out-of-domain multiple-choice comparisons pooled for the 82% result.

3 SETUP

  • Eligible pairs use a named public base model trained with RLVR, without intervening supervised fine-tuning or distillation.The study includes seven such pairs from three releases; additional larger pairs and a pilot are reported separately.Source: §3 · p. 4
  • The central evaluation covers out-of-domain multiple-choice MMLU, MMLU-Pro, and ARC-Challenge, with added benchmarks spanning the other domain-and-answer-form combinations.These include free-form out-of-domain tasks, free-form in-domain mathematics, and in-domain multiple-choice tasks; items were frozen before generation.Source: §3 · pp. 4–5
  • Harness extractors come from six public repositories and look for answers in different places, including the box, text, or an Answer line.The paper uses pinned copies and applies each multiple-choice extractor to all three multiple-choice benchmarks.Source: §3 · p. 5
  • Each model answers under a plain prompt and a prompt requesting a single letter inside the box, allowing comparison when the prompt itself asks for that format.Some checkpoints’ default system message already requests a boxed final answer, including under the nominally plain prompt.Source: §3 · p. 5
  • Four of the seven pairs show a clear out-of-domain multiple-choice gain readable by the fixed extractor and supply the 82% pool.The DeepSeek-Math-7B pair has no such gain to split, while the fixed extractor cannot read the Qwen2.5-1.5B base outputs reliably.Source: §3 · p. 5

4 RESULTS

Across out-of-domain multiple-choice benchmarks, most gain reported by harness extractors disappears when the same outputs are read with a fixed extractor. The difference reflects base models stating answers in text while RL models more often put them in \boxed{}, and the result persists under alternative choices and independent readers.

4.2 RL MOVES THE STATED ANSWER INTO \boxed{}

  • 26.6% of base-model outputs contain a \boxed{}, versus 58.3% of RL-model outputs; most remaining base answers are stated in text.An extractor reading only the box therefore misses more base answers; the difference persists on items where neither model reaches the token limit.Source: §4.2 · pp. 5–6

4.1 MOST OF THE OUT-OF-DOMAIN MULTIPLE-CHOICE GAIN IS FORMAT GAIN

  • 82% of the harness-reported gain on MMLU, MMLU-Pro, and ARC-Challenge for four released 7B SimpleRL pairs is format gain.Harness extractors report an average 22.5-point gain per comparison, versus 4.0 points when the fixed extractor reads the same outputs.Source: §4.1 · p. 5, Figure 2 · p. 6

4.2 RL MOVES THE STATED ANSWER INTO \boxed{}

  • On MATH-500 and GSM8K, 96% to 97% of both models’ readable outputs contain a \boxed{}, and format gain is close to zero.This contrasts with the out-of-domain multiple-choice pattern.Source: §4.2 · p. 6

4.3 MOST OF THE GAIN REMAINS FORMAT GAIN UNDER OUR OTHER CHOICES AND UNDER INDEPENDENT READERS

  • 75% to 87% of reported gain remains format gain across the tested choices, while independent LLM readers put the share at 77%.The share is 79% with the boxed-instruction prompt and 86% with the plain prompt; the response sample is too small to determine whether 77% and 82% differ.Source: §4.3 · p. 7

4.4 A REPORTED GAIN DEPENDS ON ITS EXTRACTOR

  • 15 points is the median spread between harness extractors’ gains on identical outputs across pair, benchmark, and prompt combinations.On Figure 1’s outputs, two box-only extractors report 30.6 points, while Qwen2.5-Math’s extractor reports 1.0 point.Source: Figure 5 · p. 8, §4.4 · p. 8
  • 56% of the gain on Qwen2.5-Math’s extractor comparisons is format gain, versus 85% and 83% for two extractors that read only the box.Across seven model–extractor combinations, higher base-model extraction rates are associated with lower format shares, mostly driven by Qwen2.5-Math’s extractor.Source: §4.4 · p. 8
  • Of 17 out-of-domain multiple-choice results reported by 18 surveyed recent releases, 13 were computed by extracting answers from generated text.One result used log-likelihoods, and three did not state how they were computed.Source: §4.4 · p. 8

5 RELATED WORK

Prior work examines RL’s transfer, evaluation sensitivity, formatting during training, and extractor behavior. This paper distinguishes its focus by measuring how much a reported gain changes when the same outputs are read by another extractor.

5 RELATED WORK

  • Transfer studies ask what RL adds across domains, while this paper asks how much of a measured gain the evaluation extractor produces.Yue et al. (2025) examine pass@k; Hu et al. (2026) and Huan et al. (2025) study domain transfer.Source: §5 · p. 8
  • Prior evaluation studies report sensitivity to implementation choices and parsing, while input-format controls and test-time adaptation change what is compared.This paper holds outputs fixed and changes only the extractor.Source: §5 · p. 9
  • Training studies find that format-only rewards and formatting features can raise strict accuracy, while this paper measures the split on released checkpoints and out-of-domain multiple choice.The cited work includes RL rewards for producing \boxed{} and analyses separating content from formatting effects.Source: §5 · p. 9
  • Extractor studies show that rule-based methods can miss equivalent answers and LLM extractors can err, but extractor accuracy alone does not quantify its share of a particular gain.That share depends on how the two models differ in where they write their answers.Source: §5 · p. 9

6 LIMITATIONS

The reported 82% is based on outputs and prompt–extractor pairings generated for this study, not a decomposition of published gains. Its scope is also limited by extractor behavior, prompt choice, model family, and the fixed extractor’s possible failures.

6 LIMITATIONS

  • The 82% does not estimate the SimpleRL-Zoo paper’s published gains, because this study generated its own outputs and did not reproduce that evaluation.The published paper used its own prompt and evaluation script; the authors compare published accuracies with their readings of the same pairs in an appendix.Source: §6 · p. 9
  • Nearly all of the gain behind 82% comes from harness extractors that read the box first; reading text letters as well reduces the format share.Running Qwen2.5-Math’s extraction function without its letter-reading step gives a pooled share of 85%.Source: §6 · p. 9
  • The four 7B pairs come from one release family and three base models, with Qwen base models carrying nearly all reported gain.Whether the share holds in other model families remains open.Source: §6 · p. 9
  • The format share depends on the prompt: a pilot prompt requesting a boxed letter yielded 31%, versus 88% and 86% under the study’s two prompts.Under the pilot prompt, the base model stated no answer readable by the fixed extractor in 45% of outputs, so much of the measured gain reflected stating an answer at all.Source: §6 · p. 10
  • On the one Qwen2.5-32B DAPO pair, format share was 2.1% over 9 comparisons, but that pair cannot isolate scale from recipe or base-model effects.The base boxed answers when asked; the RL model rarely did when not asked.Source: §6 · p. 10
  • The fixed extractor can disagree with harness extractors even on the sign of a change; the LLM-based independent check included no human readers.A Llama-3.1-8B pilot was excluded from the 82% after the fixed extractor reported a 19-point loss.Source: §6 · p. 10

7 CONCLUSION

The paper concludes that some credit for out-of-domain RLVR gains can belong to the program reading the answers. It reports a large format share for four 7B pairs, but little for one 32B pair, and recommends documenting and re-scoring outputs; the analysis artifacts are also described.

7 CONCLUSION

  • On four released 7B SimpleRL pairs, 82% of reported out-of-domain multiple-choice gain is format gain, while almost none is format gain on the one 32B pair.The paper recommends naming the extractor, reporting its base-model extraction rate, and re-scoring the same outputs with an extractor that reads answers in boxes or text.Source: §7 · p. 10

REPRODUCIBILITY

  • The ancillary files include analysis code, frozen items, results, reader responses, and hashes for all 264 generation files; eight files are included with the arXiv version.The remaining 256 generation files are available from the author, and file hashes link generations to the item set.Source: REPRODUCIBILITY · p. 10

AI USE STATEMENT

  • The authors used generative AI for hypothesis refinement, methodological feedback, result interpretation, prose editing, figure creation, literature and checkpoint searches, and pipeline execution.They state that AI did not generate synthetic datasets or translate, and that they reviewed the work and re-derived every reported number from raw results.Source: AI USE STATEMENT · pp. 10–11

A HOW THE POPULATION NARROWS

The appendix traces how the study’s candidate population becomes the comparisons behind its main result. The resulting 46 comparisons are concentrated in three Qwen pairs, while the pool is restricted by eligibility and evaluation criteria.

A HOW THE POPULATION NARROWS

  • The grid counts comparisons, each defined as one base–RL pair on one benchmark under one prompt, scored by one harness extractor.Table 2 presents the grid in counts, with Table 1 naming what each row counts.Source: A HOW THE POPULATION NARROWS · p. 15, Table 2 · p. 15
  • Table 3 traces the reduction from generation runs to the 46 main-result comparisons, with each row following from the prior row except the pairing step.The text explains that second row; Table 3 groups are those of Table 19.Source: Table 3 · p. 15, A HOW THE POPULATION NARROWS · p. 15
  • The 12 checkpoints form 7 pairs, each with 16 benchmark–prompt combinations; multiple-choice and free-form combinations contribute different numbers of extractor comparisons.Each pair contributes 88 comparisons.Source: A HOW THE POPULATION NARROWS · p. 15
  • Neither Qwen2.5-1.5B pair has comparisons meeting the extraction-rate criterion because the base model reaches the token limit on almost every item.The pool rests on one release family because the qualifying pairs from other labs were set aside or excluded.Source: A HOW THE POPULATION NARROWS · pp. 15–16
  • All 46 main-result comparisons have positive format gain, and 43 have 95% intervals excluding zero; the three Qwen pairs carry 98.5% of summed reported gain.The comparisons span 20 combinations across benchmarks, prompts, and harness extractors.Source: Table 3 · p. 15, A HOW THE POPULATION NARROWS · p. 16
  • The candidate screen excluded most nonqualifying releases because RL began from distilled, instruction-tuned, or fine-tuned checkpoints rather than a named public base model.The two larger qualifying pairs were set apart before generation; the DAPO Qwen2.5-32B pair was kept outside the pool as a larger pair.Source: A HOW THE POPULATION NARROWS · p. 16

B THE FIXED EXTRACTOR AND ITS VARIANTS

The study’s fixed extractor reads explicitly stated answers across boxes and text, then compares how stricter or looser rules affect the format-share estimate. The results show sensitivity to unequal extraction rates between models and to the prompt used to elicit answers.

B THE FIXED EXTRACTOR AND ITS VARIANTS

  • For multiple-choice items, ω checks six answer positions in order, stops at the first position present, and takes that position’s last occurrence.For free-form items, it prioritizes the last boxed answer, then specified answer phrases; if none is found, it abstains and the item is scored wrong.Source: B THE FIXED EXTRACTOR AND ITS VARIANTS · p. 16
  • The strict variant reads only the last box and admits no comparisons because it reads under half of one model’s outputs or the models’ extraction rates differ by more than 20 points.Its format gain is zero by construction, so it cannot evaluate the format share on admitted comparisons.Source: B THE FIXED EXTRACTOR AND ITS VARIANTS · p. 17
  • Across the 18 comparisons admitted by all three non-strict variants, the maximal variant exceeds ω by 0.0184 (95% CI 0.0004 to 0.0357).Within this family, the share moves by less than 0.02 and ω gives the lowest value, but the interval’s lower end is too close to zero to establish the direction on its own.Source: B THE FIXED EXTRACTOR AND ITS VARIANTS · p. 17, Table 4 · p. 17
  • The permissive pilot extractor reports a 0.5535 format share versus ω’s 0.8805 when it reads 80.4% of base and 99.8% of RL outputs.It credits RL with a 14.2-point fixed-extractor gain, while ω finds 3.8 points; the unequal extraction rates admit this pilot comparison.Source: B THE FIXED EXTRACTOR AND ITS VARIANTS · p. 17, Table 5 · p. 18
  • Under the pilot’s box-requesting prompt, the format share is 0.311, compared with 0.881 and 0.864 under the grid’s two prompts.Only 54.8% of base outputs and 75.8% of RL outputs were read under the pilot prompt, so it misses the extraction-rate criterion by one point and is excluded from the main result.Source: B THE FIXED EXTRACTOR AND ITS VARIANTS · pp. 17–18
  • Across the 46 main-result comparisons, ω abstains on 13.1% of base outputs and 12.0% of RL outputs on average, but individual comparisons show larger differences.In the Figure 1 comparison, ω finds no answer in 56 base outputs and 7 RL outputs; the authors retain comparisons with such abstentions.Source: B THE FIXED EXTRACTOR AND ITS VARIANTS · p. 18

C INDEPENDENT READERS

The authors checked ω against independent LLM readers on a declared sample of model responses. The readers’ estimate was close to ω’s, but the prespecified test classified the comparison as inconclusive.

C INDEPENDENT READERS

  • The check sampled 2,720 responses from 34 generation runs, with two readers independently reading each response and a third resolving disagreements or missing readings.Readers were blinded to the key, model identity, and extractor readings.Source: C INDEPENDENT READERS · p. 18
  • On the 46 main-result comparisons, the readers’ format share is 0.771 (95% CI 0.696 to 0.849), versus 0.822 under ω; the declared test is inconclusive.The difference is −0.0508, with an interval from −0.1259 to +0.0266.Source: C INDEPENDENT READERS · p. 19
  • The first two readers agree on 98.2% of the 2,719 responses both returned, and the tie-break reader resolves every remaining disagreement or missing reading.The release also contains a blind sample of 120 responses for a human check, which the authors have not run.Source: C INDEPENDENT READERS · pp. 19, 21
  • Where ω reads a letter, the readers choose the same letter on 98.6% of responses in both groups.Most disagreements are ω abstentions; where it abstains, readers find the key more often for RL than base responses, 52.2% versus 39.1%.Source: C INDEPENDENT READERS · p. 19

D ROBUSTNESS IN FULL

The robustness checks find that the pooled format share varies with comparison inclusion rules and extractor choice, while leaving out individual pairs, benchmarks, or harness extractors changes it less. The appendix also details how harness extractors handle outputs and how scoring choices can affect results.

D ROBUSTNESS IN FULL

  • The three comparison-inclusion rules move the share by at most 0.073, and the two alternatives to the main rule lower it.The 54 comparisons added by the second rule have a summed reported gain of −280.6 points and format gain of −253.0, giving a ratio of 0.902.Source: D ROBUSTNESS IN FULL · pp. 21–22, Table 9 · p. 22
  • Five extraction-rate settings place the share between 0.7933 and 0.8222, with the main setting giving the highest share.Without an extraction-rate criterion, some admitted comparisons have very low base-model read rates; when the chat-template rule is also dropped, one base model’s outputs are read at only 3.6–9.2%.Source: D ROBUSTNESS IN FULL · p. 22, Table 10 · p. 22
  • Leaving out any one pair, benchmark, or harness extractor moves the share by at most 0.032, 0.016, or 0.031, respectively.Leaving out one prompt moves it to 0.7879 or 0.8557; the two prompts carry about half of the summed reported gain each.Source: D ROBUSTNESS IN FULL · pp. 22–23, Table 12 · p. 23
  • The two box-only extractors give shares of 0.854 and 0.835 on their own comparisons, while Qwen2.5-Math’s extractor gives 0.562.The two box-only extractors use the same algorithm; together, box-first extractors carry 97.6% of the summed reported gain.Source: D ROBUSTNESS IN FULL · p. 23, Table 13 · p. 24
  • Six comparisons have a format share above one because ω reads a small negative gain where the harness extractor reads a positive one.They contribute 7.8% of summed reported gain and are retained because dropping comparisons by their share would select on the outcome.Source: D ROBUSTNESS IN FULL · p. 23, Table 14 · p. 24
  • On the 18 comparisons both admit, the external Qwen2.5-Math extractor and ω have pooled shares within 0.03, but their comparison-level shares differ by 0.25 on average.The pooled agreement reflects offsetting differences; the extractors disagree on the sign of the RL gain in 5 of 18 comparisons.Source: D ROBUSTNESS IN FULL · pp. 23, 25, Table 15 · p. 25

E THE HARNESS EXTRACTORS

  • The default lm-evaluation-harness MMLU task scores option likelihoods without extracting generated text, unlike its generative MMLU variant and default MMLU-Pro and BBH tasks.The study transcribed extractors from repository files and tested executable versions against those copies.Source: E THE HARNESS EXTRACTORS · pp. 25, 27
  • First-character scoring can credit a whole response as a multiple-choice answer: Open-Reasoner-Zero’s extractor does so when no box is found.Across 33,000 outputs, it credits a correct answer this way on 6.99% of base outputs and 4.61% of RL outputs, lowering the reported gain and format share.Source: E THE HARNESS EXTRACTORS · p. 27

F THE OTHER GROUPS

This appendix compares format shares across three groups beyond out-of-domain multiple choice. The out-of-domain estimate is comparatively stable across rules, while other groups and the domain contrast remain uncertain.

F THE OTHER GROUPS

  • Out-of-domain free-form format gain remains untested: only two qualifying comparisons come from one combination, with a 14.8-point extraction-rate gap.That combination’s reported gain is 6.4 points, while the fixed extractor reads −2.2.Source: F THE OTHER GROUPS · p. 27
  • In-domain multiple-choice share shifts by up to 0.346 when one pair or benchmark is omitted, exceeding the group’s own interval.The share moves from 0.369 to 0.829 when one pair is left out and from 0.465 to 0.771 when one benchmark is left out.Source: F THE OTHER GROUPS · p. 27
  • The out-of-domain minus in-domain multiple-choice contrast is +0.107 (95% CI −0.057 to 0.271) under the significant-gain rule, but its direction depends on the rule.Under the other rules, the contrast is 0.593 and 0.519; the authors say the data cannot establish whether format gain is specific to out-of-domain evaluation.Source: F THE OTHER GROUPS · p. 27
  • 0.07 is the out-of-domain multiple-choice share’s movement across three inclusion rules, versus more than 0.5 for each other group.The appendix attributes the wider movements to the other groups’ instability under the rules.Source: F THE OTHER GROUPS · p. 27, Table 20 · p. 29
  • In-domain free-form’s −0.144 share mixes extractor results; three extractors report gains below the fixed extractor’s, while three have no qualifying comparisons.The shortfall over the fixed extractor’s gain is 20.6% for Qwen2.5-Math’s, 23.5% for lm-evaluation-harness’s flexible GSM8K rule, and 26.8% for verl’s flexible rule.Source: F THE OTHER GROUPS · p. 30, Table 21 · p. 30
  • Two larger contrasts have intervals excluding zero but are not reported as findings because their groups move by an order of magnitude more under the rules.The appendix also reports an out-of-domain multiple-choice minus free-form contrast of −0.5216 (95% CI −1.6519 to 0.6087).Source: F THE OTHER GROUPS · p. 30

G BEYOND THE GRID: LARGER PAIRS AND THE LLAMA PILOT

The appendix tests the measurement beyond the main grid using two larger pairs and a Llama pilot. The 32B pair shows almost no format gain, while the pilot illustrates how extraction rates can fall even when conditional accuracy does not.

G BEYOND THE GRID: LARGER PAIRS AND THE LLAMA PILOT

  • 0.021 is the format share for Qwen2.5-32B with DAPO: +2.0 format-gain points over +94.0 reported-gain points.Across prompt choices or after removing any one harness extractor, the share is between −0.04 and 0.10; the authors qualify the main result as applying to models of 7B and below under their declaration.Source: G BEYOND THE GRID: LARGER PAIRS AND… · p. 31, Table 23 · p. 31
  • The 32B pair’s near-zero format share contrasts with its small difference in answer location; scale, RL recipe, and lab differ from the main grid and cannot be separated by this pair.The authors also report a Qwen2.5-Math extractor-only share of 0.436, between the declared cutoffs, with two comparisons carrying the share.Source: G BEYOND THE GRID: LARGER PAIRS AND… · p. 31
  • Under the boxed-instruction prompt, the 32B base model writes boxed answers in 99.2% of out-of-domain multiple-choice outputs, versus 42.2% for the base models of the four main-result pairs.Under the plain prompt, the DAPO model writes boxed answers in 14.8% of outputs, versus 54.8% for the four pairs’ RL models.Source: G BEYOND THE GRID: LARGER PAIRS AND… · p. 31
  • Mistral-Small-24B has only three qualifying comparisons, below the declared five-comparison floor, so its 0.560 share is reported but not used.The authors treat it as having no gain to decompose on their outputs, not as evidence about the share; SimpleRL-Zoo reports MMLU gains they did not reproduce.Source: Table 22 · p. 30, G BEYOND THE GRID: LARGER PAIRS AND… · p. 31
  • On the Llama pilot, the fixed extractor reads 90% of base versus 68% of RL ARC-Challenge outputs and 70% versus 38% on MMLU.MMLU-Pro is excluded because the extractor reads only 6% of base outputs; neither model writes a readable boxed answer, even when asked.Source: G BEYOND THE GRID: LARGER PAIRS AND… · pp. 31–32, Table 24 · p. 32
  • The Llama pilot’s fixed-extractor score is 19 points lower for RL, but conditional accuracy is 75.5% versus 73.8%, a +1.7-point difference.The authors interpret this pilot as an extraction-rate collapse, not a loss of capability; other harness extractors report either no change or a rise on the same outputs.Source: G BEYOND THE GRID: LARGER PAIRS AND… · p. 32

H WHAT THE SHARE TRACKS

The appendix examines which features of the base models and extractors track format share, then reviews how released evaluations produce out-of-domain scores and documents reproducibility. Across combinations, base-model boxing rate is associated with share, but the evidence does not support prediction beyond the studied combinations.

H WHAT THE SHARE TRACKS

  • Across 20 combinations, format share falls as base-model boxing rate rises (r = −0.434, 95% CI −0.687 to −0.082), but repeated rates make the interval narrower than the design supports.Across 46 comparisons, the correlation is weaker (r = −0.275); the association is across combinations and does not rank base models.Source: H WHAT THE SHARE TRACKS · p. 32
  • On 13 matched comparisons using Qwen2.5-Math-7B, SimpleRL-Zoo and SimpleRL-Zero shares differ by 0.024, but no interval shows the gap exceeds sampling noise.The authors describe this recipe-effect evidence as exploratory.Source: H WHAT THE SHARE TRACKS · pp. 32–33
  • Within out-of-domain multiple choice, change in boxing rate has near-zero correlation with share (r = −0.03), and no within-group dose–response is detectable at this grid’s size.The positive correlation across groups is a between-group contrast, not evidence of a within-group relation.Source: H WHAT THE SHARE TRACKS · p. 33, Table 25 · p. 33
  • Across 20 base-model/extractor entries, the extractor accounts for 78.5% of extraction-rate variance, and changing extractors moves the rate by a median 72.7 points versus 30.15 for changing base models.The table reports the same direction when empty returns are excluded.Source: H WHAT THE SHARE TRACKS · p. 33, Table 26 · p. 34
  • Across seven pooled combinations, share’s correlation with base-model boxing rate is r = −0.858, but the seven points do not support prediction beyond those combinations.Without the two Qwen2.5-Math extractor combinations, the correlation is −0.461 over five points; the interval for the full correlation treats dependent points as independent.Source: H WHAT THE SHARE TRACKS · p. 34

I HOW RELEASED RLVR AND REASONING-MODEL CHECKPOINTS REPORT OUT-OF-DOMAIN MULTIPLE CHOICE

  • Of 17 out-of-domain multiple-choice results from 13 releases, 13 are generated, one uses log-likelihood, and three are unstated; the 18 releases reviewed are not a systematic sample.The count establishes how results were produced, not the size of format gain in those releases.Source: I HOW RELEASED RLVR AND… · p. 34, Table 27 · p. 35
  • Published SimpleRL-Zoo accuracies are not split by this study: its outputs use different items, prompts, decoding, and extractors, and no single reading reproduces the published accuracies.For Qwen2.5-Math-7B, published MMLU accuracies are 38.0% and 56.6%; splitting those published gains would require outputs generated under the release’s own prompt and decoding.Source: I HOW RELEASED RLVR AND… · pp. 34–35, Table 28 · p. 35

J COST AND REPRODUCIBILITY

  • The 192 grid generation runs took 29.7 GPU-hours, while scoring after generation runs on a CPU; frozen item hashes and output hashes support reproducibility and re-scoring.The appendix also records decoding, precision, item counts, and seeded intervals; ancillary files include analysis code and results.Source: J COST AND REPRODUCIBILITY · pp. 35–36
Loading 2610.20639v1…