PaperlayerView exact revision on arXiv

Source-linked AI summary

Correct Choices, Failed Sequences: Choice2Seq Benchmarks Correctness Retention Across Output Interfaces

Tianyou Lai, Wentao Yue, Zhiying Deng, Qilei Li

arXiv:2610.16191v1cs.AI

TL;DR

Correct whole-answer selection may not survive sequential construction, and aggregate interface accuracy can obscure which successes persist. Choice2Seq pairs independent selection and construction on the same problems and candidates; results show large, nonuniform gaps, alongside improvements from candidate support and indexed evidence.

  • Problem

    Aggregate accuracy across interfaces obscures which correctly selected items remain correct under sequential construction, making item-level paired comparisons important.

    Source: §1 · p. 1

  • Method

    Choice2Seq compares independent whole-answer selection and local construction on the same problems and candidates, using SCR to measure Serial correctness among Native-correct items.

    Source: §1 · p. 1, §3.1 · p. 3

  • Results

    Large Native–Serial gaps appear across models; Jev reaches 99.5% Native accuracy on lexical retrieval but only 2.1% SCR.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Candidate-supported options improve construction, while indexed evidence improves copying fidelity even for incorrect selections, and dialogue retention differs by model.

    Source: §1 · p. 2

    The primary tasks characterize short, constrained construction rather than unrestricted long-form or multilingual generation, and interface contrasts do not isolate a single decoding mechanism.

    Source: Limitations · p. 9

Abstract

from arXiv · show

A model may select the correct answer yet fail to construct it through a sequence of local decisions. We introduce Choice2Seq, a diagnostic benchmark that measures correctness retention across complete-answer selection and sequential construction in independent runs on the same problems and candidate sets. Sequence Composition Rate (SCR) measures the fraction of correctly selected items that are also answered correctly through local construction. Jev achieves 97.1% and 99.5% selection accuracy on arithmetic and lexical retrieval, yet retains only 33.0% and 2.1% of those correct choices under serial construction, respectively. Substantial gaps also emerge for StartLux and Qwen, including when Qwen emits characters directly. Restricting local options to candidate-supported characters improves construction, while explicit position indexing improves copying fidelity, even for incorrect selections. These interventions distinguish task correctness, plan fidelity, and output compliance. Matched dialogue-state evaluations reveal that local field construction largely preserves Jev's correct states but reduces Qwen's mean-turn accuracy, highlighting task- and interface-dependent differences. Sequence-level confidence and request-cost analyses further characterize completed outputs.

1 Introduction

Choice2Seq asks whether correct whole-answer choices survive independent sequential construction on matched tasks, a question that aggregate accuracy can obscure. Its paired measure and targeted controls distinguish retention from copying fidelity and output compliance.

1 Introduction

  • Prior interface-level accuracy comparisons obscure which correctly selected items remain correct under sequential construction, while faithful copying can preserve an error.The paper motivates item-level pairing and controlled comparisons with distinct evaluation criteria.Source: §1 · p. 1
  • Choice2Seq pairs independent complete-answer selection and sequential construction on the same problems and candidate sets, with SCR measuring retained correctness among Native-correct items.The paper asks how reliably correct selection carries over to local construction and how retention varies across interfaces.Source: §1 · p. 1
  • Jev’s Native accuracy reaches 97.1% on arithmetic and 99.5% on lexical retrieval, yet its SCR falls to 33.0% and 2.1%, respectively.Large Native–Serial gaps also appear for StartLux and when Qwen emits characters directly.Source: §1 · p. 2
  • Candidate-supported local choices improve construction, while indexed evidence improves copying fidelity even for incorrect selected answers.These controls distinguish task correctness, copying fidelity, and output compliance.Source: §1 · p. 2
  • Poor retention is not universal: Jev largely preserves correct dialogue states, whereas Qwen loses mean-turn accuracy.The introduction also notes that symbolic tasks show high retention.Source: §1 · p. 2

2 Related Work

Related work spans structured decisions, output constraints, construction and readout, and confidence or trajectory evaluation. Choice2Seq connects these threads through paired interface outcomes and controls that separate correctness from copying and format compliance.

Structured decisions and output interfaces

  • Choice2Seq pairs complete-answer and local outcomes on identical problems and candidates, extending the comparison to matched dialogue fields.The related-work discussion frames the benchmark alongside structured-decision and output-constraint evaluations.Source: §2 · pp. 2–3

Construction and readout

  • Choice2Seq probes prefix feedback through fresh position queries with external state and uses supplied-answer controls to separate copying fidelity from task correctness.Matched digit–letter operations test representation dependence.Source: §2 · p. 3

Confidence and trajectory outcomes

  • Choice2Seq tests whether scores aggregated from local choices predict completed-string exact match and reports dialogue outcomes beyond terminal success.Its confidence comparisons use length-only references and calibration uses prevalence baselines.Source: §2 · p. 3

3 Evaluation Protocol

The protocol compares whole-answer selection with independently executed local construction, then uses paired outcomes and targeted controls to examine correctness retention. Tasks, interfaces, evidence, models, and statistical procedures define the scope of those comparisons.

3 Evaluation Protocol

  • Choice2Seq compares whole-answer selection with local construction on paired tasks, using paired outcomes to identify retained, lost, and newly correct items.The protocol distinguishes correctness retention from diagnoses of its limits.Source: §3 · p. 3

3.1 Interfaces and outcomes

  • Native selects a complete candidate, while Serial builds one character at a time from its own prefix; SCR is Serial correctness conditional on a correct Native choice.Paired tables report both-correct, Native-only, Serial-only, and both-wrong outcomes; copying controls separately score fidelity and task correctness.Source: §3.1 · p. 3

3.2 Tasks and candidate geometry

  • The formal test uses 384 items per domain, and arithmetic–lexical pairs match candidate ambiguity along the correct-prefix path without equalizing computation or retrieval difficulty.Tasks include arithmetic and English noun retrieval; a symbolic panel and isomorphic digit–letter operations extend the comparisons.Source: §3.2 · p. 4

3.3 Controlled interventions

  • Interventions vary local choice support, singleton filling, evidence representation, and rendering; dialogue arms compare joint, independent, and serial field construction.Trie keeps completed strings within the original candidate set, while indexed evidence adds a position table and Direct also supplies the queried character.Source: §3.3 · p. 4

3.4 Models and statistical analysis

  • Model comparisons distinguish service-returned choices, probability-based readouts, direct generation, and position queries, with decoding contracts and Native denominators kept distinct.Analysis uses item or source-cluster resampling, corrected McNemar tests, dialogue-level bootstrap, and specified exploratory GEE fits.Source: §3.4 · pp. 4–5

4 Correctness Retention Across Interfaces

Across tested tasks and models, whole-answer accuracy does not consistently carry over to local construction, and marginal accuracy can hide which items were lost or gained. The observed gap also depends on the evaluated interface and prompt conditions.

4 Correctness Retention Across Interfaces

  • Jev’s high whole-answer accuracy coexists with lower Serial accuracy, especially in lexical retrieval; every original paired decrease survives Holm correction.Candidates alone yield near-chance selection, supporting a role for problem evidence; Native and Serial rankings also differ for direct Qwen versus Jev on arithmetic.Source: §4 · p. 5
  • Qwen3-8B gains nine semantic-field successes overall but retains six of 28 Native successes, losing 22 and gaining 31.Marginal accuracy can therefore obscure replacement of previously correct answers; Jev’s similar marginal accuracies accompany high overlap.Source: §4 · p. 5
  • On the same Qwen3.5-27B checkpoint and 96 parents per domain, WholeGenerate solves almost all items, while Serial position queries lose many successes despite valid character output.The contrast concerns the evaluated external construction interface; prompt-state representation, assistant prefill, and rendering also vary.Source: §4 · p. 5
  • Across three templates, direct Qwen Serial EM ranges from 7.3–28.1% in arithmetic and 39.6–56.2% in lexical retrieval, while Native accuracy remains higher in both domains.Jev’s arithmetic retention decreases with length, while lexical retention is low even for short targets; the broad pattern persists under prompt, order, and repeat perturbations.Source: §4 · p. 5

5 Diagnosing Correctness and Fidelity

Across output interfaces, correctness on complete-answer selection does not guarantee retention during sequential construction. Candidate support, evidence representation, output contracts, and task structure each shape construction or copying outcomes.

5.1 Candidate support and prefix context

  • 82.0% TrieCall EM, versus 24.0% UnionCall and 1.8% Full, on matched Jev lexical items; TrieCall prevents cross-candidate recombinations while querying every position.Every valid Trie output belongs to the candidates, whereas only 26.3% of UnionCall outputs do.Source: §5.1 · p. 6
  • TrieFill reduces calls from 5.5 to about two per item while retaining eight feasible outputs; queried depth and branch placement affect Jev more than alphabet size on twelve-character reversals.FullFill improves arithmetic accuracy while preserving full choices at ambiguous positions; Union and Trie restrict choices, and Fill skips singletons.Source: §5.1 · p. 6
  • Gold prefixes improve character accuracy, especially lexically, without comparable complete-string gains; arithmetic Serial exceeds the Independent-position-product reference at most lengths.The arithmetic TeacherForced string-success difference has an interval spanning zero; lexical outcomes remain low under generated prefixes.Source: §5.1 · pp. 6–7

5 Diagnosing Correctness and Fidelity

  • Jev’s arithmetic SCR declines with length and lexical SCR remains near zero, while direct-output Qwen retains more lexical than arithmetic answers.Figure 2 covers 64 paired items per domain and length; SCR is conditioned on Native-correct items.Source: Figure 2 · p. 7

5.2 Evidence access and plan fidelity

  • A position table derived from supplied answers improves position readout and complete-string Serial fidelity for Jev and StartLux across every fixed-criterion model/domain comparison.Indexing changes addressability, salience, and prompt length together.Source: §5.2 · p. 7
  • Indexed copying improves fidelity even for incorrect selected plans, but faithfully copying an incorrect plan still fails gold EM.The table distinguishes fidelity to freshly selected plans from indexed outputs’ correctness against the task target.Source: §5.2 · p. 7, Table 4 · p. 7

5.3 Option binding and output contracts

  • Qwen’s output-contract results vary by model and evidence: Qwen3-8B favors characters in both directions, while Qwen3.5’s corrected differences favor codes.For Qwen3.5, matched generation favors indexed character output but codes with direct lexical evidence; natural Serial can retain content errors despite valid character output.Source: §5.3 · p. 8

5.4 Task and representation effects

  • Matched COPY/REVERSE tasks show no consistent letter-versus-digit Serial advantage, while Jev retains most correct named and binary semantic fields.The natural arithmetic–lexical contrast combines solving with candidate geometry and evidence access; queried depth and branch placement also contribute to construction accuracy.Source: §5.4 · p. 8

6 Reliability of Completed Outputs

Completed-output reliability depends on more than correctness: confidence, request cost, termination, and accuracy across dialogue states are evaluated separately. Field construction largely preserves Jev’s correct states but lowers Qwen’s mean-turn accuracy.

6.1 Confidence, cost, and stopping

  • 13/2,347 lexical successes (0.55%; exact 95% interval 0.30–0.95%) leave AUROC/AP gains over length and the Brier difference from prevalence-only uncertain.At 10% coverage, Product precision is 3.8% [1.7, 6.4]; calibration error persists.Source: §6.1 · p. 8
  • Serial adds requests, tokens, and recorded request time relative to Native plus program return; the common-alphabet track separately scores premature or missing END and cap exhaustion.These are distinct from strict content EM.Source: §6.1 · p. 8

6.2 Field construction and downstream state

  • Jev retains 420 of 429 Joint-correct turns under Serial, with a mean-turn difference interval spanning zero; Qwen has a corrected Serial mean-turn loss.Serial–Joint final-state difference intervals span zero for both models, and local field construction has higher per-turn request costs.Source: §6.2 · p. 8
  • Rollout has lower mean-turn and final-state accuracy than Oracle despite identical trajectory EM, which remains zero after the first error.The comparison tests gold versus predicted prior state.Source: §6.2 · p. 8

7 Conclusion

The paper asks whether correctness under whole-answer selection carries over to independently executed local construction. Its findings support interface-aware evaluation that tracks retained successes and separates task correctness, copying fidelity, and output compliance.

7 Conclusion

  • Choice2Seq compares paired tasks and examines conditions preserving correctness; candidate support, evidence representation, and generation protocols shape construction outcomes.High selection accuracy alone does not guarantee retention, and copying fidelity can improve without improving task correctness.Source: §7 · p. 8

Limitations

The evaluation covers short, constrained construction and specific dialogue, confidence, and runtime settings. Several comparisons do not isolate a single mechanism or generalize beyond the reviewed population.

Limitations

  • Primary tasks use lengths three to eight and eight displayed candidates, characterizing short, constrained construction rather than unrestricted long-form or multilingual generation.A twelve-character symbolic panel extends the length range; off-path branching, computation, and vocabulary remain task-specific.Source: Limitations · p. 9
  • Natural-generation comparisons treat continuous responses, position queries, and reconstructed character prefills as distinct configurations, without attributing differences to one internal decoding mechanism.Indexing comparisons may change addressability and salience together; human judgments support the reviewed population, while resource-level conclusions cover the full panel.Source: Limitations · p. 9
  • Dialogue results use known-service intents and categorical fields with state-specific endpoints rather than character SCR; confidence and cost findings apply to evaluated mixtures, prompts, and runtime configurations.Fresh-plan fidelity comparisons use 30 plans per model/domain.Source: Limitations · p. 9

Ethical Considerations

The methods define paired units and output conditions, construct screened arithmetic and lexical candidate sets, and describe the data sources and privacy practices. They also specify representation, confidence, and inference procedures.

A.1 Design and independent units

  • The independent unit is a parent item, shared symbolic source, semantic fact record, or complete dialogue; positions and repeated conditions remain grouped within it.Paired observations identify Native–Serial overlap, while marginals alone bound it.Source: §A.1 · p. 12
  • On selected-plan parents, Qwen arithmetic FullFill and UnionFill solved 36/96 items, retaining 35/94 and 36/94 Native successes, respectively.In the named semantic Qwen cell, 22 losses and 31 gains yielded a net gain of nine.Source: §A.1 · p. 12

A.2 Arithmetic and candidate construction

  • Arithmetic products are stratified by operand structure and carry, and primary candidate sets match lexical counts of candidates compatible with each gold prefix.Each natural pair shares the same K vector along the gold path, with K(0)=8 and K(1)=2.Source: §A.2 · pp. 12–13

A.3 Lexical sources and review

  • Lexical targets are screened for sense, candidate ambiguity, and frequency, with final quotas conditioned on available sources and review eligibility.The primary sources are WordNet 3.0 and wordfreq 3.1.1; short eligible nouns are scarce under the selection requirements.Source: §A.3 · pp. 13–14
  • Both reviewers selected the intended answer and judged it uniquely acceptable for all 576 lexical-bank items; 384 eligible items were selected by frozen quotas.A separate calibration bank retained 2,347 of 2,400 targets after screening and review.Source: §A.3 · p. 13

A.4 Output conditions and stopping

  • Native selects among eight candidates; Independent, Serial, and TeacherForced differ in whether they receive a position, generated prefix, or gold prefix, while copy conditions assess fidelity to a supplied plan.The separate 51-choice track includes END and scores exact completion under a cap of L+3 requests.Source: §A.4 · p. 14

A.5 Representation and readout controls

  • Representation controls vary symbolic encodings, named or binary fields, position evidence, option-name bindings, and code-versus-character output contracts.Fresh-plan copying compares raw and indexed evidence across Independent and Serial controls, with indexing computed from the model's selected string, including errors.Source: §A.5 · p. 14

A.6 Confidence and statistical inference

  • Confidence analyses score completed strings using path probabilities and calibration metrics, while inference uses clustered or paired resampling and prespecified corrections.The primary six Native/Serial exact McNemar tests share a Holm correction at 0.05.Source: §A.6 · pp. 14–15

A.7 Models and reproducibility

The study pins model and software versions, preserves detailed request and response records, and documents how costs are estimated. Reproducibility materials include the frozen items, schedules, and offline reconstruction.

A.7 Models and reproducibility

  • Jev, StartLux-Decision-4B, Qwen3-8B, and Qwen3.5-27B are evaluated with pinned versions, checkpoints, and specified decoding and output constraints.The study records checkpoint, tokenizer, and source hashes alongside responses.Source: §A.7 · p. 15
  • Logs retain prompts, option orders, prefixes, references, selections, available probabilities, model identity, timing, errors, and retries.Frozen schedules interleave domains and conditions within collection windows.Source: §A.7 · p. 15
  • Costs count logical requests, input tokens, available physical forwards, and recorded request times, excluding loading and orchestration overhead.Jev charges are estimated at a frozen USD 0.042 per million input tokens; reproducibility materials include frozen items and complete requests and responses.Source: §A.7 · p. 15

B.1 Prefix feedback and supplied targets

The section examines prefix feedback, copying, and output interfaces through matched and controlled evaluations. Results show that fidelity, task accuracy, and format validity depend on the construction and evidence conditions.

B.1 Prefix feedback and supplied targets

  • Indexing improves fidelity to fresh selected plans for both models and domains, but arithmetic fidelity can exceed gold exact match when incorrect plans are copied faithfully.The figure uses 30 fresh plans per domain and model, with 95% item-bootstrap intervals.Source: §B.1 · p. 15, Figure 4 · p. 16

B.2 Three workflows on the same selected-plan parents

  • Jev arithmetic copying was faithful on 40/60 selected plans but gold-correct on 38/60, while program return preserved the selected choice perfectly without another call.The workflows share parents, but Native and Serial solve independently.Source: §B.2 · p. 16

B.3 Natural whole-answer generation

  • Whole-answer generation improves Qwen3-8B accuracy while changing which items succeed; Serial legal-code exact match is zero in both domains.The table compares 384 items per domain.Source: Table B1 · p. 16

B.4 Digit and letter representations and semantic controls

  • On matched COPY/REVERSE bases, letter–digit contrasts span both directions, while extra option-name conflict losses remain for StartLux arithmetic and Qwen3-8B lexical readout.Paired intervals are 95%; asterisks mark Holm-corrected p < 0.05 within comparison families.Source: §B.4 · p. 16, Figure 5 · p. 17

B.6 Matching generation contracts

  • Jev preserves most correct semantic decisions under both named and binary encodings, with SCR reported as retained over Native-correct counts.The matched generation-contract panel compares code and character outputs under identical evidence and order; malformed responses fail.Source: §B.6 · pp. 16–17, Table B2 · p. 17

B.5 Readout at a single position and option binding

  • Indexed readout approaches perfect accuracy for Jev and StartLux from lower raw accuracy, while Qwen3-8B legal-code accuracy remains lower.The panel uses 60 supplied-target parents per domain.Source: §B.5 · p. 16, Figure 6 · p. 17

B.7 Distribution, stopping and cost

  • For Qwen3-8B, format validity stays high despite accuracy differences; indexing favors characters in both domains, whereas direct lexical evidence favors codes.The separate 51-choice track scores END behavior, content, and request-cap outcomes.Source: Figure 7 · p. 18, §B.7 · p. 18

C Dialogue Evaluation

Dialogue evaluations compare access to prior states and construction of current fields. Gold prior states improve some accuracy measures, while local field construction has model-dependent effects on mean-turn accuracy.

C.1 Prior-state access

  • The 96 dialogue prefixes cover four, six, or eight user turns across ten services, and state selection requires changes and a nonempty categorical value.HistoryRecompute receives the transcript and schema; Rollout uses its previous predicted state, while Oracle uses the gold prior state.Source: §C.1 · p. 18

C.2 Matched current-state construction

  • The current-state comparison uses Joint, Independent, and Serial field construction on the same visible history and schema; Serial additionally receives earlier field predictions for that turn.Neither local arm receives Joint's output.Source: §C.2 · pp. 18–19
  • Gold prior states improve mean-turn and final-state accuracy over predicted prior states without changing trajectory exact match.Differences are reported as Oracle minus Rollout with paired-dialogue and response-component intervals.Source: Table C2 · p. 19, Table C1 · p. 19

D.1 Shared Native-correct population

  • Local field construction reduces Qwen mean-turn accuracy, while the Jev contrast spans zero.Mean-turn accuracy weights dialogues equally; final-state and trajectory exact match score terminal states and complete prefixes, respectively.Source: Table C4 · p. 20
  • Trajectory accuracy can conceal differences in later states; gold prior states improve mean-turn and final-state accuracy over predicted prior states while trajectory accuracy remains unchanged.Exploratory intervals resample dialogue groups linked by shared responses.Source: Figure 8 · p. 20

D.2 Repeated Native observations

  • Repeated Native requests and Serial trajectories are collected for 72 parents, with choice stability and probability stability treated as distinct endpoints.The three requests and trajectories per parent remain grouped by parent.Source: §D.2 · p. 20

D.3 Natural task robustness under complete trajectories

This section examines natural-task robustness across complete trajectories, using robustness and paired-outcome analyses to distinguish correctness retention from faithful reproduction and to show what aggregate accuracy can conceal. It also reports semantic-review and generation-budget checks and specifies the natural-task, continuous-generation, field-construction, and readout contracts used in the evaluations.

D.3 Natural task robustness under complete trajectories

  • On matched dialogues, Qwen3.5-27B shows a corrected mean-turn loss from local field construction while Jev’s interval spans zero; final-state intervals span zero for both models.Intervals are 95% dialogue bootstrap intervals.Source: Figure 9 · p. 20

D.4 Direct output natural prompt robustness

  • Jev leads all seven arithmetic settings and five lexical settings, while two lexical settings have no successes; worst-template retention remains well below Native accuracy.The setting averages are not expectations over an unspecified prompt distribution.Source: §D.4 · p. 22, Table D4 · p. 22

D.6 Matching complete feasible outputs and controller calls

  • Jev’s lexical-minus-arithmetic EM was −26.3 points [−30.2,−22.4] under Full and 2.6 [−2.9,7.8] under TrieCall; the TrieCall interval spans both negative and positive contrasts.The study notes that TrieCall changes feasible outputs and that all local conditions remain below Native.Source: §D.6 · p. 22, Figure 12 · p. 24

D.7 Crossed controller factors on longer strings

  • Candidate-supported local construction improves Serial success while leaving a gap to Native; arithmetic gains combine option restriction with deterministic filling, whereas lexical gains retain a model request at every position.The position union can leave the complete candidate set, so its constraint differs from Trie.Source: §D.5 · p. 22, Figure 11 · p. 23

D.8 Solving through continuous and reconstructed-prefix outputs

  • PrefixContinue’s initial complete suffix is correct on 94/96 arithmetic and 92/96 lexical items, but zero lexical items complete the repeated reconstruction sequence; the exact 95% upper limit is 3.8%.Invalid or empty subsequent characters fail and enter the next state as an invalid marker.Source: §D.8 · p. 24

D.13 Crossed prompt and option order robustness

  • Indexing raises copying fidelity, yet Jev copies every indexed incorrect target, showing that faithful copying can preserve errors.The analysis uses supplied correct and incorrect targets; its error-enriched population measures conditional fidelity rather than fresh-plan success prevalence.Source: §D.12 · p. 26, Figure 16 · p. 28

D.15 Independent lexical calibration study

  • Conditional exact 95% SCR upper limits for zero independent lexical successes are 0.67% (0/546) and 0.79% (0/468) at lengths seven and eight; only thirteen successes overall make fine-grained selection estimates imprecise.The independent lexical score-minus-length intervals span zero, so pooled discrimination does not establish incremental screening utility beyond length.Source: §D.15 · p. 29

D.16 Paired correctness turnover

  • Semantic fields have lower Native coverage and larger conditional differences, with Qwen named-field accuracy gains coexisting with substantial losses of previously correct items.Marginal accuracies alone do not identify this overlap.Source: §D.16 · p. 29

D.17 Lexical output membership and semantic review

  • No reviewed lexical pair received unanimous noun-and-definition acceptance, leaving strict closed-set EM unchanged; accepting uncertain “pop” raises Jev UnionCall from 92/384 (24.0%) to 93/384 (24.2%).The review covered sampled pairs; unindexed outputs outside the sample retain unresolved semantic status.Source: §D.17 · p. 32

D.18 Observed generation budgets

  • All 1,056 position-query requests stop within two tokens, and whole-generation requests never hit the cap; four PrefixContinue requests reach it, but some format failures occur below it.The audit rules out position-budget truncation in this panel, but the comparison changes prompt-state representation, assistant prefill tokenization, and repeated requests jointly.Source: §D.18 · pp. 32–33

E.1 Formal content and stopping instructions

  • The formal content track queries each absolute answer position without END, while the stopping track permits END among its local options.Serial construction supplies the current prefix; TeacherForced substitutes the gold prefix.Source: §E.1 · p. 34

E.2 Complete natural task paraphrases and candidate support

  • Natural paraphrases change the instruction while retaining the controller’s problem, candidates, length, position, and prefix; Position union Serial filters local options to candidate-supported characters.Both Serial arms use the problem and full candidate set without receiving the fresh Native selection.Source: §E.2 · p. 34

E.3 Continuous solving and prefix continuation

  • WholeGenerate and PrefixContinue receive the original problem, eight candidates, and target length, but PrefixContinue appends the generated prefix and consumes only its first suffix character per request.Both allow 32 new tokens; PrefixContinue’s state omits absolute position and current-prefix fields.Source: §E.3 · p. 35

E.4 Matched field construction contract

  • Matched field-construction arms reuse fixed field and value orders; Serial additionally includes its actual preceding field predictions in partial_current_state.Joint requests all fields, Independent requests only the current field, and Serial requests the current field.Source: §E.4 · p. 35

E.5 Full crossed readout contracts

  • The T0–T2 readout templates preserve the target and criterion bindings while varying wording emphasis and whether the response is a choice code or criterion character.The contracts require only the specified single-character response, without explanation or extra formatting.Source: §E.5 · pp. 35–36, Table E2 · p. 36

E.6 Complete request examples

  • The request examples show that identifying the correct criterion can still fail the output contract when the model emits its bound code instead of the required character.Responses are scored without case repair, whitespace trimming, or code mapping.Source: §E.6 · p. 36, Table E3 · p. 37
Loading 2610.16191v1…