PaperlayerView exact revision on arXiv

Source-linked AI summary

When Should Enterprise Policy Decisions Be Learned, and When Should They Be Reasoned?

Hyojung Han, Ryangkyung Kang, Youngeun Kim, Seung-Hun Jeon

arXiv:2610.17643v1cs.AI

TL;DR

Enterprise agents must choose a policy action before responding, yet generated action objects and test-time reasoning are often bundled together. The paper separates these factors and evaluates operational paths and a sealed benchmark with computed labels. Reasoning drives the sealed benchmark’s exact-action generalization gap, while several format contrasts remain unresolved or are not detected.

  • Problem

    Enterprise agents must choose whether to answer, seek information or confirmation, call tools, escalate, or refuse before producing a response, while output format and test-time reasoning are often bundled together.

    Source: §1 · p. 2

  • Method

    The paper separates generated JSON actions from constrained single-token decisions and varies whether the model reasons first, using a sealed benchmark whose labels come from a hidden executable policy.

    Source: §1 · p. 2

  • Results

    On the sealed benchmark, reasoning drives the exact-action generalization gap; with reasoning off, no format effect was detected, while the reasoning-on format contrast remains unresolved.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results distinguish hidden pre-decision reasoning from emitting an action object: the former matters on the sealed benchmark, while no action-output advantage was measured in the clean reasoning-off comparison.

    Source: §1 · p. 2

    The benchmark uses one constructed policy family, so performance on real enterprise policies remains unvalidated.

    Source: §11 · p. 19

Abstract

from arXiv · show

An enterprise agent that answers under a written policy must choose an action before it says anything: answer, ask for a missing fact, ask for confirmation, call one or several tools, escalate, or refuse. Holding one 27B Korean enterprise model fixed, we vary whether the decision is a generated JSON object or a single constrained token, and whether the model may reason first. On 2800 in-distribution items we did not detect an accuracy difference between the two deployed paths (93.46% vs 94.14%), while they differ about tenfold in restart stability (3.00% of actions flip vs 0.29%); they also differ in reasoning, so this is an operational comparison. On a sealed benchmark of 300 items whose labels come from a hidden executable policy, reasoning is worth +17.3 points [+12.3, +23.0] on exact action. With reasoning off, the format difference was 0.0 points [-4.3, +4.0], a failure to detect an advantage for either interface, not equivalence; with reasoning on it is unresolved. Against a token-matched random control, boundary-targeted supervision moves the two covered boundaries by +17.5 and +20.0 points and leaves a held-out boundary unchanged. In a 3-seed training x reasoning factorial, the sealed-rendering interaction is -4.9 [-8.1, -1.7], consistent with partial substitution. All supervision was Korean. On independently rendered English versions, the model is 0.82 points lower [-2.68, +0.93] with reasoning on, the same order as re-rendering in Korean; the reasoning-off gain from targeted supervision is not stable in either language, so we do not claim transfer. The original text stays private as the confirmatory set. An open-model re-rendering made after these results reproduced the reasoning effect and the English verdict; the interaction kept its sign but did not replicate under item-clustered intervals. We release the model and that re-rendering with gold labels.

1 Introduction

The paper separates action-output format from test-time reasoning to study enterprise policy decisions. Reasoning matters on the sealed benchmark, while the in-distribution comparison concerns two operational paths that differ in both format and reasoning.

1 Introduction

  • On the sealed benchmark, reasoning drives the exact-action generalization gap; its labels are computed by a hidden executable policy, not written.With reasoning off, the format comparison detects no effect; with reasoning on, the contrast remains unresolved.Source: §1 · p. 2
  • Boundary-targeted supervision improves decisions on explicitly covered boundaries, while the untouched boundary shows no detected transfer.Source: §1 · p. 2
  • In the 3-seed factorial, the training-by-reasoning interaction is negative and stable on the sealed rendering, but not stable on the open re-rendering.The authors therefore base the substitution finding on the sealed set alone.Source: §1 · p. 3
  • On independently rendered English items, no English degradation was detected at the pre-registered audit’s resolution, but reasoning-off gains were unstable in both languages.Source: §1 · p. 3

2 The decision task and three interfaces

  • The study distinguishes action-output generation from reasoning generation: emitting the action object is separate from producing hidden pre-decision tokens.The cleanly identified reasoning-off comparison detects no measurable advantage for action-output generation.Source: §1 · p. 2, §2 · p. 3

3 In distribution, we detect no accuracy difference between the cheap and deployed paths

  • On the 2800-item full split, the deployed path scores 93.46% and the cheap path 94.14%, with no detected accuracy difference.These are operational paths: the deployed path reasons and emits JSON, while the cheap path uses one constrained token without reasoning.Source: §3 · p. 4, Table 1 · p. 4
  • Restart flips average 3.00% for the deployed path and 0.29% for the cheap path, with non-overlapping reported ranges.The ranges are 2.54–3.36% and 0.18–0.50%, respectively; end-to-end latency reductions for the cheap path are upper bounds from this serving setup.Source: §3 · p. 4

4 A benchmark nobody labelled

The authors replace unreliable human/model adjudication with a sealed benchmark whose gold actions are derived from a hidden executable policy. Its construction includes checks for rendering fidelity and shortcut risks, while the authors note limits to its ecological validity.

4 A benchmark nobody labelled

  • Cross-vendor adjudication did not converge: κ = 0.47 on 1,193 items and κ = 0.40 on 853, with roughly 30% abstained in both runs.On the intended answer-versus-tool boundary, adjudicators chose call tool on 3 of 348 items versus answer on 182.Source: §4 · p. 5
  • A deterministic executor maps latent policy factors to one action under a fixed priority order and emits a proof of the rule and factors behind it.The authors enumerated the factor space and retained assignments with exactly one derivable action.Source: §4 · p. 5
  • Natural-language items were checked for answer leakage, counterfactual consistency, and rendering fidelity, while selection used structural difficulty rather than model performance.Recovery and challenge gates reject items when extracted factors mismatch the generating assignment or an alternative reading supports another action.Source: §4 · p. 5
  • 300 items across seven layers, including 30 counterfactual pairs and 15 invariance pairs, were sealed before evaluation.A twelve-item checklist was re-run from the sealed artifact.Source: §4 · p. 6
  • The majority-class baseline scores 29.7% overall, with per-layer scores ranging from 26.7% to 60.0%.CALL TOOL is the most frequent gold action.Source: §4 · p. 6
  • Lexical probes exceeded their layer ceilings on two layers, and all 15 invariance pairs came from the dialect layer where a lexical shortcut existed.The checklist checks answer leakage but not lexical shortcuts; the authors say future versions should gate on both before sealing.Source: §4 · p. 6
  • The benchmark supports defensible labels and controlled counterfactuals, but constructed policies do not establish ecological validity for real company documents.The confirmatory rendering remains private; an open-model re-rendering was produced after results were known.Source: §4 · p. 6

5 On unseen boundaries, reasoning drives the generalisation gap

A sealed-benchmark factorial separates reasoning from output format: reasoning improves exact action, while the reasoning-off format contrast detects no advantage and the reasoning-on contrast remains unresolved. Boundary-targeted supervision improves covered boundaries, but transfer to the held-out boundary is not established.

5 On unseen boundaries, reasoning drives the generalisation gap

  • The initial sealed comparison appeared to favor JSON (85.3% versus 70.7%), but it confounded reasoning with format, prompting the full two-by-two test.The initial JSON arm reasoned; the constrained single-token arm did not.Source: §5 · p. 6
  • For invariance pairs, the reasoning effect is unresolved: with only 15 pairs, one pair changes the score by 6.7 pp.The authors describe this endpoint as uninformative for effects smaller than a large one.Source: §5 · p. 8
  • Reasoning adds +17.3 pp [+12.3, +23.0] on exact action; with reasoning off, format is 0.0 pp [−4.3, +4.0], while the reasoning-on format contrast remains unresolved.The reasoning-on arms use different prompts, so their format contrast is confounded.Source: §5 · pp. 8–9, Figure 2 · p. 9
  • With reasoning off, call tool-only recall favors the single token by +12.4 pp [+4.5, +20.2] on this rendering, but the difference does not persist on the open re-rendering.With reasoning on, both formats exceed 93.3% and no difference was detected.Source: §5 · p. 9
  • A confidence router reaches 85.7% while reasoning on 50% of items, versus 88.0% with reasoning always on and 70.7% with it off; unseen-boundary errors are often confident.The confidence threshold was chosen on in-distribution data before the sealed set was opened.Source: §5 · pp. 9–10
  • Reasoning improves counterfactual both-correct performance by +43.3 pp [+23.3, +63.3]; the reasoning-off format estimate is −10.0 pp and inconclusive.This endpoint requires both members of a counterfactual pair to be correct.Source: Figure 3 · p. 10

6 Is the reasoning requirement a supervision gap?

  • Against token-matched random supervision, targeted training raises any-tool recall from 60.2% to 70.1%, meeting the registered gate, though seed-pair gains are uneven.Overall accuracy rose by +4.8 pp, below both preregistered effect-size thresholds; the in-distribution non-regression check passed.Source: §6 · pp. 10–11
  • Targeted supervision improves the covered answer/tool and tool/confirmation boundaries by +17.5 pp and +20.0 pp; the held-out boundary shows no established transfer.The missing-information boundary moved by +2.2 pp with an interval containing zero; its control arm was already near the top of its range.Source: §6 · p. 11, Figure 4 · p. 12

7 Learned or reasoned

In the sealed-benchmark factorial, reasoning adds less after boundary-targeted supervision than after random supervision, consistent with partial substitution. The configuration comparison also shows that no single-pass option reaches the always-reasoning reference.

7 Learned or reasoned

  • Both trained arms were evaluated with reasoning on and off through one harness, across three seeds, with weights as the only changed factor.The estimands, intervals, and stability rule were fixed before the twelve evaluations.Source: §7 · p. 12, Table 5 · p. 15
  • 79.2% for targeted supervision remained below the 88.0% always-reasoning reference; these are competing configurations, not causal deltas.The configurations use different decision mechanisms and harnesses, so the values should not be subtracted across tables.Source: §7 · pp. 12, 14, Figure 5 · p. 13, Table 4 · p. 14
  • A common ceiling could explain at most 3.1 pp of the 4.9 pp interaction under a simple compression model, but the design cannot establish the ceiling.The remaining −1.8 pp is tied to an indeterminate contrast, so partial substitution is a description of joint behavior, not a mechanism.Source: §7 · pp. 13–14
  • −4.9 pp [−8.1, −1.7] interaction: reasoning adds less after targeted than random supervision, consistent with partial substitution.The interaction had the same sign in all three seed pairs; the result describes joint behavior in this operating range, not a mechanism.Source: §7 · pp. 12–13, Figure 6 · p. 14
  • The practical reading is to spend data on nameable boundaries when deployment requires one pass, and compute when reasoning is affordable.The targeted arm remained 8.8 pp below the always-reasoning reference, and reasoning added less after targeted supervision than after random supervision.Source: §7 · pp. 13–15

8 Two robustness checks that need no new runs

Robustness checks show that some conclusions depend on metric definitions and benchmark layers, while the English audit finds no detectable degradation but does not establish transfer of the supervision effect. The post-hoc open re-rendering reproduces some findings, but not the interaction’s registered stability verdict.

8 Two robustness checks that need no new runs

  • Depending on the denominator, targeted-over-random call-tool recall ranges from +2.8 to +15.7 pp; the five shortcut-clean layers preserve the main effects but shift the format estimate to −0.8 pp.The pre-registered any-tool recall comparison is +9.9 pp; these checks recompute stored predictions and were not pre-registered.Source: §8 · pp. 15–16, Table 6 · p. 16

9 Zero-shot transfer across policy languages

  • The English audit freshly rendered the same latent scenarios rather than translating Korean text, using 286 items that passed gates in every rendering.The common subset scored 1.5–2.2 pp higher than the full benchmark, so absolute accuracies are not comparable across tables.Source: §9 · p. 16, Table 7 · p. 18
  • With reasoning on, English was −0.82 pp [−2.68, +0.93] versus Korean, with no detectable degradation; the targeted-supervision gain was not stable in either language.The reasoning-off gain was indeterminate in both languages, so the audit does not support a transfer claim.Source: §9 · pp. 16–17, Table 7 · p. 18

10 An open re-rendering as a robustness check

  • The public re-rendering is post hoc, and its gates disproportionately removed hard multi-tool and clarification scenarios, leaving a benchmark 3.9 pp easier.The original rendering remains private and confirmatory; the public version is a robustness check, not independent confirmation.Source: §10 · pp. 17–18
  • Table 8’s replication grades are a post-hoc descriptive taxonomy; revising its rule changed four verdicts among 27 graded findings.The authors report both first-pass and final counts so the revision can be checked.Source: §10 · pp. 18–19
  • The open rendering reproduced the reasoning effect and English verdict, but the interaction did not replicate under item-clustered intervals.Its interaction estimate kept a negative sign but reached zero; format effects and shortcut probes also varied with rendering.Source: §10 · p. 19

11 Limitations

The claims are bounded by evaluation on one Korean model and one constructed policy family, plus unresolved design limitations in key comparisons. The related-work discussion positions the paper’s contribution as separating action-output format, reasoning, and their interaction in a bounded policy task.

11 Limitations

  • Every result uses one 27B Korean model and one constructed policy family; performance on these policies may not generalize to real company documents.The paper identifies evaluation on real enterprise policies with owner-defensible labels as its most important next validation.Source: §11 · p. 19
  • The reasoning-on format comparison is confounded because the two arms use different reasoning prompts; only the reasoning-off format contrast is cleanly identified.The in-distribution path comparison also differs in reasoning as well as format, and restart stability was not measured under matched reasoning.Source: §11 · pp. 19–20
  • Targeted and random training arms were token-matched, not label-matched; the call-tool class-prior shift remains a possible explanation for some gains.A post-hoc label-matched arm argues against class prior as a sufficient explanation but does not conclusively reject it.Source: §11 · p. 20
  • Item-resampling intervals omit retraining uncertainty, and the two-by-two cells are single runs with run-level noise not captured by those intervals.Only the no-reasoning cells are free of the stated restart-variability concern.Source: §11 · p. 20
  • The public re-rendering is post hoc and selectively easier, so it is a robustness check rather than independent confirmation.Its gates disproportionately removed hard multi-tool and clarification scenarios.Source: §11 · p. 20

12 Related work

  • The paper’s stated novelty is factorially separating action-output interface and test-time reasoning, then measuring their interaction on an executable policy benchmark.It distinguishes this from work on constrained structure generation and from work measuring training or reasoning resources separately.Source: §12 · pp. 21–22

13 Two post-hoc experiments, one of which failed

The paper’s final claims distinguish operational-path comparisons from format effects, and qualify results on targeted supervision, reasoning, and English renderings. Post-hoc controls probe the targeted advantage but leave its mechanism unresolved.

13 Two post-hoc experiments, one of which failed

  • The label-matched control did not eliminate the targeted arm’s average advantage, but one seed pair flipped sign and left T − C1 indeterminate.T exceeded C1 by +5.8 pp averaged over seeds, yet the seed-level instability also prevents isolating contrastive boundary construction as the causal mechanism.Source: §13 · p. 23
  • Call-tool recall rose from 59.2% and 56.2% in C0 and C1 to 74.9% in T, alongside a higher predicted call-tool rate, without higher precision.The post-hoc, three-seed analysis cannot separate better boundary detection from a shift toward predicting call tool more often.Source: §13 · p. 23, Figure 7 · p. 24
  • The matched-reasoning format experiment was void because changing the output instruction changed reasoning length and caused readout collapse in the alternate wiring.The positive control also failed: planted conclusions were followed on only 10.3% of single-token and 50.0% of JSON items, below the registered 90% threshold.Source: §13 · p. 24

14 What we claim

  • 94.14% vs 93.46% accuracy and 0.29% vs 3.00% restart flips compare two operational paths, not output formats alone.The paths differ in both reasoning and output interface; the accuracy comparison is a failure to detect a difference, not evidence of equivalence.Source: §14 · p. 25
  • Reasoning moves exact action from 70.7% to 88.0%; with reasoning off, the format difference is 0.0 pp [−4.3, +4.0].The reasoning-on format effect remains unresolved because the arms do not share a reasoning prompt; the post-hoc control did not remove that confound.Source: §14 · p. 25
  • +17.5 pp and +20.0 pp on the two covered boundaries exceed a token-matched random control, while the held-out boundary shows no detected transfer.A post-hoc label-matched control argues against class prior as a sufficient explanation, but seed instability neither conclusively rejects that hypothesis nor establishes the construction effect.Source: §13 · p. 23, §14 · p. 26
  • The training × reasoning interaction is −4.9 pp [−8.1, −1.7], consistent with partial substitution on the sealed rendering.The interpretation concerns joint behavior in this operating range, not a mechanism; the open re-rendering interaction was not stable.Source: §14 · p. 26
  • With reasoning on, English is −0.82 pp [−2.68, +0.93] from Korean, but the reasoning-off supervision gain is not stable in either language.The result is not an equivalence claim and does not cover English-authored enterprise policies.Source: §14 · p. 26

15 Conclusion

The conclusion limits the findings to one Korean enterprise model and a constructed policy family, emphasizing real-policy validation and matched-reasoning format comparison as unresolved needs. The released artifacts support checking results, but the confirmatory text and training corpus remain unavailable.

15 Conclusion

  • Evaluation on real enterprise policies and a comparison of output interfaces under matched reasoning are the two validations the paper most needs.The harness could not construct matched reasoning, so the reasoning-on format effect remains unresolved.Source: §15 · p. 26

Artifacts

  • The original 300-item sealed rendering remains private, while the public open re-rendering contains 248 common items and the original set is verifiable by content hash.Released materials include latent scenarios, executable gold labels and proofs, the generator, and a standalone verifier.Source: Artifacts · p. 27
  • The English audit uses independent renderings of the latent scenarios, but those items are no longer a blind test set for future cross-lingual claims.Future cross-lingual claims require a freshly sealed benchmark.Source: Artifacts · p. 27
  • The released boundary checkpoint is a merged full-weight model with its pointer head, selected on the in-distribution evaluation set rather than the sealed benchmark.The merge affects 496 targeted text-tower tensors and leaves the vision tower and remaining weights byte-identical.Source: Artifacts · p. 27
  • The training corpus is not released because it derives from internal and licensed sources that cannot be redistributed.Released renderings were produced with open-weight models; commercial models only served as gate judges and wrote none of the released text.Source: Artifacts · p. 27

A Measurement index

The measurement index catalogs each main-text result table and figure with its checkpoint, interface, sample size, runs, uncertainty method, and harness. Its entries distinguish evaluation configurations and denominators across in-distribution, sealed, training, and rendering comparisons.

A Measurement index

  • Main-text result tables and figures are indexed with checkpoint/interface, sample size and denominator, runs or seeds, uncertainty, and harness.Seed-averaged paired intervals use a 10,000-replicate item-clustered bootstrap, moving all three seeds of a resampled item together.Source: A Measurement index · p. 31
  • The in-distribution entry compares the deployed JSON path with a constrained single-token path across 2,800 items and restart pairs.Accuracy uses 2,414 clean-split and 2,800 full-split items; restart flips count items whose action differs between two restarts.Source: A Measurement index · p. 31
  • The sealed reasoning × format experiment uses 300 items, one run per cell, and a paired item bootstrap; the training × reasoning factorial uses three seeds per arm.The supervision comparison uses a pointer-head interface with three seeds per arm, and distinguishes any-tool-required n=118 from call-tool-only n=89.Source: A Measurement index · pp. 31–32
  • The index warns that some entries mix harnesses, and absolute latencies are not comparable across sections because hardware and serving conditions differ.The configuration ladder mixes serving and pointer-head results; the supervision seeds were trained across H200 NVL and B200 hardware, with paired arms on the same hardware.Source: A Measurement index · p. 32
  • The cross-lingual audit reports 286 items in the common intersection of four sets, while the open re-rendering replication uses 248 open-common items and re-scores existing checkpoints.The open audit lists 300 items for the original full set; its existing-checkpoint re-scoring on open rendering is the same as the source tables.Source: A Measurement index · pp. 32–33

B Reproducibility details

The reproducibility details document uncertainty calculations, models, prompts, action readouts, and serving configurations. They also identify several unrecorded implementation details that constrain exact reconstruction.

B Reproducibility details

  • Seed-averaged paired intervals use a 95% percentile interval from a 10,000-replicate item-clustered bootstrap with seed 12345.All seeds of each resampled item move together; the statistic is the seed-averaged paired difference.Source: B Reproducibility details · p. 33
  • Every arm starts from Qwen3.8-27B-Human-KO-Enterprise-v0.1 in bf16, and the sealed benchmark is identified by a content hash.The Korean open re-rendering is separately identified by its content hash over 248 items.Source: B Reproducibility details · p. 33
  • Prompts are single user messages without a system message; JSON arms use the Figure 9(a) contract, while single-token arms append the Figure 9(b) suffix.English-frame cells use an English contract and suffix that translate the Korean versions line for line while leaving action names untranslated.Source: B Reproducibility details · p. 33, Figure 9 · p. 34
  • Reasoning is toggled per request, and single-token decisions select among action codes from top-20 log-probabilities at the first position.With reasoning on, the single-token path generates reasoning first and then reads one token; the pointer head instead scores seven actions from the encoded state.Source: B Reproducibility details · pp. 34–35
  • Training arms share arguments, while data differ; vLLM LoRA serving required adapter-key rewriting after an initial no-match attempt was discarded.The discarded attempt produced output identical to the base model.Source: B Reproducibility details · p. 35
  • Absolute latencies are not comparable across sections because hardware and serving conditions vary, and several implementation details were not recorded.Unrecorded items include the reasoning-on prompt tail, chat-template hash, pointer-head latency, and scores for space-prefixed letter codes.Source: B Reproducibility details · p. 35

C Measurement caveats and audit trail

The audit qualifies how several reported measurements and comparisons should be read. In particular, some endpoints are unresolved, and the English audit does not identify which checkpoint stage preserved English decision ability.

C Measurement caveats and audit trail

  • The English audit supports transfer only for the released checkpoint; without English evaluation of the base or intermediate checkpoint, the preservation stage is unknown.The English benchmark also may under-represent English-specific policy formulations and discourse conventions.Source: C Measurement caveats and audit trail · pp. 35–36
  • The reasoning-on interaction is descriptive of this operating range, not an identified mechanism, because a ceiling could explain up to 3.1 of the 4.9 pp effect.Both reasoning-on cells are near 88.7%; the paper says a ceiling is not excluded.Source: C Measurement caveats and audit trail · p. 36
  • Several reported endpoint differences are below the data’s resolution, including +2.2 pp on missing information and +4.4 pp on invariance pairs, and are not interpreted.One counterfactual pair represents 3.3 pp and one invariance pair 6.7 pp.Source: C Measurement caveats and audit trail · p. 36
  • The in-distribution accuracy comparison lacks a paired test, and its split-specific differences point in opposite directions, so neither direction is established.The full split favors single token by 0.68 pp; the clean split favors generated JSON by 0.11 pp.Source: C Measurement caveats and audit trail · p. 36
  • The measurement record has additional design qualifications: restart sets differed, latency p95 is noisy, and none of three seed-noise endpoints met the registered band.The latency sample was 40 items per restart; observed seed noise was 1.0–13.5 pp, above the registered 1.5–3.0 pp band.Source: C Measurement caveats and audit trail · pp. 36–37
  • The two call-tool recall values use different denominators—89 gold call-tool items versus 118 tool-requiring items—so they are not comparable across sections.The paper notes that the affected endpoint passed and that both denominator choices are defensible.Source: C Measurement caveats and audit trail · p. 36

C.1 Matched-state diagnostic (pre-registered, void)

The pre-registered matched-state diagnostic was void because its planted-wrong positive control failed. Its post-hoc readouts differed under planted-wrong traces, but the paper treats that observation as descriptive rather than registered evidence.

C.1 Matched-state diagnostic (pre-registered, void)

  • The diagnostic was void after G1 failed, so its −0.3 pp primary difference is not interpretable as a format effect or evidence of equivalence.The failed positive control means the readout cannot establish whether the interfaces read the same state the same way.Source: §C.1 · p. 37
  • Under planted-wrong traces, readouts followed the planted action at 10.3% for single token and 50.0% for JSON, a post-hoc observation only.On real traces, the corresponding rates were 85.7% and 86.0%.Source: §C.1 · p. 37
  • The diagnostic used unnatural retained reasoning and teacher-forced scoring, which removes the parse failures of free generation.Even a valid run would have compared choices from a fixed state rather than deployed generation.Source: §C.1 · p. 37

D Changes in v2 •

Version 2 clarifies which comparisons are unresolved or stable, expands the per-class and reproducibility material, and corrects several measurement labels and intervals. The open re-rendering remains a replication check rather than the confirmatory set.

D Changes in v2 •

  • The reasoning-on output-format effect is unresolved, while the training-by-reasoning interaction is described as stable only on the sealed rendering.The revised intervals weaken the reasoning-off targeted-supervision gain in both languages and the interaction does not replicate on the open re-rendering.Source: D Changes in v2 • · pp. 37–38
  • The revision adds a worked benchmark example and clarifies the executor’s scope and the recovery and challenge gates where they first appear.It also adds a measurement index, reproducibility appendix, and training, serving, hardware, and latency details.Source: D Changes in v2 • · pp. 37–38
  • The paper standardizes interval estimation with an item-clustered bootstrap and corrects labels distinguishing call-tool-only recall from any-tool recall.The same exact-action contrast now carries the same interval across tables, and the registered Section 6 verdict is unchanged.Source: D Changes in v2 • · p. 38
  • The open re-rendering uses 248 common items and was generated after the original results, whereas the sealed private rendering remains the primary analysis set.The table reports paired contrasts with item-bootstrap intervals and descriptive robustness grades.Source: Table 8 · p. 39
  • A post-hoc label-matched control matches the treatment arm’s action-count histogram, with token budget within 0.26% overall, but its token totals are not exact.Its boundary-layer accuracy uses 120 items and excludes the policy-versus-principle layer; its call-tool-only recall uses 89 gold call-tool items.Source: Table 9 · p. 40
  • The revision adds a per-class analysis of call-tool precision, recall, F1, and predicted rate, plus per-action precision and recall across all seven actions.The per-class view is post-hoc and based on stored pointer-head predictions from three seeds on 300 sealed items.Source: D Changes in v2 • · p. 38, Table 10 · p. 40, Table 11 · p. 41

Related papers

Loading 2610.17643v1…