PaperlayerView exact revision on arXiv

Source-linked AI summary

When Top-K Misses the Decision: Tool-Call Drift in Multi-Teacher On-Policy Distillation

Jiabin Shen, Guang Chen, Chengjun Mao

arXiv:2607.07050v4cs.CLcs.LG

TL;DR

Top-K distillation can preserve teacher probability mass while omitting tokens that determine behavioral branches, leaving multi-teacher supervision unable to correct tool-call entry. This paper audits and restores decision-critical support, showing that support omission drives over-calling in Qwen3.5 and recurs across model families.

  • Problem

    Top-K distillation can preserve probability mass while omitting tokens needed to calibrate discrete behavioral decisions such as tool calls versus direct responses.

  • Method

    The paper combines multi-teacher on-policy distillation with support audits, matched token restoration, placebo controls, and teacher/student support-union interventions.

  • Results

    Support omission was causally linked to Qwen over-calling and reproduced across Llama-3.1, where support union reduced over-calling from 28.8% to 11.1%.

  • Takeaways & Limitations

    Support-aware correction can reduce over-calling, but indiscriminate restoration trades restraint against required-call recall and dialogue success.

  • Takeaways & Limitations

    The matched causal study is scoped to Qwen3.5 and teacher-top-32, while Llama-3.1 lacks a second matched restoration study.

Abstract

from arXiv · show

Top-K teacher logits make on-policy distillation tractable, but probability mass is not decision support. In a two-teacher tool-use setting, vanilla generalized knowledge distillation raises tool-call recall but also over-calls on direct-answer examples. With Qwen3.5-9B, the response teacher's top-32 retains 99.99% of its probability mass yet contains the behavior-switch tool-call token on only 0.4% of 500 unique response prompts; even top-256 covers only 52.2%. Student logits for tokens omitted from the truncated support receive zero direct gradient, so the tool teacher reinforces entry while the response teacher usually cannot oppose it; a wrong entry then amplifies divergence along the generated trajectory. Matched restoration connects this blind spot to behavior: restoring the tool-call token at every supervised response position lowers full-generation over-calling from 14.2+/-2.1% to 3.7+/-0.5% across three seeds, but also lowers call recall by 12.4 points. A non-tool placebo changes over-calling by only 0.95 points, whereas a teacher/student top-32 support-union intervention reaches 7.4+/-0.6% over-calling with 87.0+/-2.0% call recall. The pattern recurs with Llama-3.1-8B's native JSON protocol: the response teacher omits the JSON-entry token through top-256 despite 99.977% retained mass, whereas the tool teacher ranks it first on every audited tool prompt; the matched support-union intervention lowers over-calling from 28.8+/-0.8% to 11.1+/-1.2%. These results identify decision-critical support omission as a causal mechanism in the primary Qwen setting and support it across model families. They also trace a restraint-capability trade-off among support-, loss-, and decoding-level interventions, motivating support-aware audits of compressed distillation. Code and aggregate artifacts are available at https://github.com/shen-jiabin/topk-support-opd.

1 Introduction

Top-K distillation can preserve nearly all teacher probability mass while omitting low-probability tokens that determine tool-call behavior, creating a one-sided gradient blind spot. Audits, matched restorations, placebo controls, and cross-model replication connect this support omission to over-calling and motivate behavior-aware support audits.

  • Behavioral consequence: Vanilla GKD improves tool-call recall but increases calls on direct-answer examples, with wrong mode entry amplifying teacher–student divergence after the branch.The shift also persists in multi-turn interaction as more calls, repeated calls, and tool-call loops.
  • Matched restoration: 14.2% to 3.7%: restoring <tool call> at every supervised response position sharply reduces full-generation over-calling across three seeds.First-position restoration reduces entry and loops, but calls migrate later; all-position restoration removes that escape route.
  • Matched placebo control: 0.95 points versus 10.47 points: a probability-matched non-tool placebo changes over-calling far less than exact all-position restoration.The control rules out generic support-cardinality expansion at the matched teacher-probability scale.
  • Cross-model replication: 28.80±0.80% versus 10.25%: under Llama-3.1-8B-Instruct, vanilla GKD over-calling exceeds the mixed-SFT reference while call recall is 97.27±0.33% versus 91.90%.The response teacher retains 99.977% top-32 probability mass while omitting the first JSON-entry token on all 500 audited response prompts.

2 Problem Setup

The setup trains a student on its own or dataset trajectories using routed tool and response teachers under a top-32 support-conditioned GKD objective. It defines decision-critical omissions as low-probability teacher tokens that can change discrete behavior but receive zero direct gradient when omitted, especially mode-entry tokens that select tool-call trajectories.

  • Training objective: Each example is routed to either a tool teacher or a response teacher, and on-policy distillation evaluates their distributions along the student-generated continuation.With probability λ = 0.8, continuations come from the current student; otherwise, training uses dataset trajectories.
  • Training objective: The teacher API supplies K = 32 tokens, and the loss compares temperature-scaled teacher and student distributions after restricting and renormalizing over the teacher’s top-K support.Unless stated otherwise, both distributions use τ = 0.9 before support restriction and renormalization.
  • Support restriction: For any omitted token v ∉ S(K), the student logit has zero direct loss gradient: ∂L/∂z_i,v = 0.The sequence loss averages supervised target positions, while a fixed supervised format anchor preserves the structured-call protocol on dataset-source trajectories.
  • Teacher routing: Both teachers share a base model but specialize in structured tool-call targets versus direct-response targets while supervising the final assistant turn.Qwen uses the special token <tool call> for tool-mode entry; Llama uses native JSON beginning with {"name.
  • Decision-critical omissions: A decision-critical omission combines low teacher probability with student probability sufficient to change behavior, despite full-vocabulary correction being blocked by truncated support.Mode-entry tokens can select an entire tool-call trajectory; the tool teacher usually supplies a positive entry update, while the response teacher often omits the token and loses the negative update.

3 Diagnosing Decision-Support Omission

Teacher-only top-K support omits decision-critical behavior-switch coordinates despite near-complete retained probability mass, creating one-sided supervision whose errors amplify after mode choice. Matched all-position restoration causally suppresses over-calling by closing delayed-call escape routes, while reducing call recall and tool-call quality.

  • Trajectory asymmetry: 55.45 times higher per-token JSD occurs in direct-answer mismatches on tool-tagged rollouts, which contribute 55.5% of the squared-divergence proxy.The reverse response→tool mismatch is more common at 11.6% but only 2.03 times its aligned baseline.
  • Matched interventions: 3.73±0.51% full-generation over-calling follows all-position restoration, versus 13.48±1.27% after first-position support.Delayed calls fall to 0.65%, showing that extending restoration across supervised response positions closes the temporal escape route.
  • Support omission: 52.2% of response prompts contain <tool call> at K = 256, despite approximately 99.99% retained mass at K = 32.The response teacher ranks <tool call> at median 236.5, and appending its exact logit restores full-vocabulary direction with one token.
  • Trade-off: 12.40 points of call recall and 2.31 points of BFCL tool-call quality are lost under all-position support relative to first-position support.The intervention is diagnostic rather than a recommended training recipe because stronger suppression improves restraint while reducing required-call performance.
  • Cross-family replication: 97.93% of Llama-3.1-8B teacher/student top-32 unions contain the JSON-entry token, with mean entry descent −0.04720 versus full-vocabulary −0.04702.The response teacher’s coverage remains zero through K = 256 despite 0.999968 retained mass, while the tool route explicitly reinforces entry.

4 Mechanism-Guided Interventions

The section organizes mechanism-guided interventions into support construction, loss calibration, and decoding calibration, which target distinct stages of the distillation pipeline. Support union restores student-relevant coordinates, loss shapers modify retained signals, and entry bias changes deployment thresholds without retraining.

  • Intervention layers: Support construction changes which token coordinates are distilled, whereas loss calibration changes retained-signal strength and decoding calibration moves the deployed entry threshold.The three layers need not select the same operating point.
  • Support construction: With K = 32, teacher/student support union averages 38.88±0.08 tokens, including 6.88±0.08 student-only additions, across three primary Qwen runs.The intervention restores coordinates where the student is currently competitive without hard-coding the tool-call token or response-only scope.
  • Support construction: The support union increases returned support by 21.5% relative to teacher top-32, while average steady-state step time rises from 106.4 to 123.5 seconds (16.0%).Peak-memory differences remain within run-to-run variation, and the method requires teacher logits for student-selected token IDs.
  • Loss calibration: Hard Clip, Global Reweight, and Soft Clamp reshape token-level divergence but do not change which token coordinates enter the distilled support.They respectively cap extreme losses, attenuate batch-relative divergences, and softly compress batch-adaptive extremes while retaining nonzero gradients.
  • Decoding calibration: Entry bias replaces the first assistant-token tool-entry logit ℓtool with ℓtool + b, where b ≤0, without retraining or changing later-token logits.The bias is selected on APIGen-MT validation and frozen for test and strict multi-turn evaluation.

5 Experiments

Experiments show that support-aware intervention reduces vanilla GKD’s over-calling while retaining call recall, improves boundary separation, and mitigates compounding errors in multi-turn interaction. Cross-family evaluation tests whether these effects persist under Llama’s native JSON protocol.

  • APIGen-MT decision results: Vanilla GKD reaches 91.5% call recall but over-calls on 14.2% of should-respond examples; support union reaches 7.4% over-calling, 87.0% call recall, and 89.8% decision accuracy.The main comparison targets retaining GKD’s call-recall gain while reducing over-calling.
  • First-token operating points: 0.9760±0.0006 support-union boundary AUC exceeds vanilla GKD’s 0.9692±0.0023 and localized compression’s 0.9710±0.0011.Support union also reaches 8.72±0.74% response-entry error and 90.55±1.00% call-entry recall.
  • Transfer evaluation: Support union reaches 81.53±0.93% BFCL overall and 68.27±1.11% When2Call MCQ, but support correction is not a uniform capability solution.These are matched mean changes of +2.55 and +2.70 points over vanilla; Mixed SFT remains stronger on BFCL, and GKD variants remain below Base/Mixed SFT on When2Call overall.
  • BFCL multi-turn diagnostic: 1.118 calls per turn and 8.1% Loop@3 under support union improve on vanilla’s 1.515 calls and 15.1% Loop@3, but dialogue exact success is only 4.1%.Turn-checkpoint replay exposes the same restraint-capability trade-off.
  • Cross-family replication: Under Llama-3.1-8B’s native JSON protocol, vanilla is call-heavy: it has 5.37 points higher call recall, 18.55 points higher over-calling, and 6.59 points lower decision accuracy than Mixed SFT.The mixed-SFT comparison is descriptive because its objective and schedule differ; matched GKD comparisons further test support-aware correction.

6 Related Work and Limitations

Prior work addresses selective distillation, sparse logit support, multi-teacher routing, loss shaping, and tool-use drift, while this paper identifies decision-token support omission as a distinct mechanism. Its causal evidence is scoped to Qwen3.5 and the implemented teacher-top-32 objective, with partial independent reproduction on Llama-3.1 and public artifacts.

  • Sparse and truncated logit support: Sparse-logit methods correct top-K distribution bias through tail estimation or aggregate tail weighting, motivating support-aware analysis.Sparse Logit Sampling preserves the full gradient in expectation with importance-sampled tail logits, while Tail-Aware Distillation amplifies the full-vocabulary tail.
  • Multi-teacher routing and conflict: Multi-teacher OPD has studied routing, specialization, and conflict, whereas this work holds routing fixed and isolates directional incompleteness in the routed loss.The two teachers need not expose the same decision token in their truncated supports.
  • Tool-use behavior drift: This work distinguishes supervised multi-teacher OPD drift from prior tool overuse, collapse, and repeated-call findings by linking behavior to omitted structural tokens.A one-sided update moves the shared call/respond boundary when one teacher removes a low-probability structural token from its support.
  • Limitations: The matched causal chain is scoped to Qwen3.5 and the implemented teacher-top-32 objective, while Llama-3.1 provides partial independent reproduction rather than a second matched restoration study.Llama-3.1 reproduces a call-heavy vanilla operating point, response-side support omission, and matched support-aware correction under a native JSON protocol.
  • Artifact availability: Training code, analysis scripts, evaluation harnesses, aggregate metrics, and plotting scripts are publicly available.Artifacts are available at https://github.com/shen-jiabin/topk-support-opd.

7 Conclusion … A.2 Tool-call target rendering

The conclusion identifies decision-critical support omission as the mechanism behind one-sided supervision and tool-call drift in compressed multi-teacher distillation. Support-aware restoration changes behavior across model families, while the experiments and appendices emphasize jointly auditing behavior, required calls, data splits, and rendered targets.

  • 7 Conclusion: Top-K distillation can retain nearly all probability mass while omitting the behavioral-branch token, allowing the tool teacher to reinforce entry without direct response-teacher correction.Wrong entry then amplifies divergence along the generated trajectory.
  • 7 Conclusion: 14.2% to 3.7%: all-position support restoration reduces final over-calling across three seeds but also reduces call recall and supported-category dialogue exact success.First-position restoration moves entry decisions while allowing calls to migrate later.
  • 7 Conclusion: 7.4% over-calling with 87.0% call recall: the student-aware support union uses 38.9 support tokens on average, unlike the teacher-probability-matched non-tool placebo.It improves single-turn separation but lowers required-turn coverage and overall dialogue exact success.
  • 7 Conclusion: 28.8% over-calling versus 10.3%: with Llama-3.1-8B, vanilla GKD exceeds the separately trained mixed-SFT reference, while support union reaches 11.1%.The native JSON-entry token is absent from response-teacher support through K = 256 despite 99.977% top-32 mass.
  • 7 Conclusion: Compressed OPD should audit decision-critical support, full-generation behavior, and task-required calls together because probability-mass coverage alone does not certify behavior preservation.Support-aware correction changes distilled coordinates, while loss- and decoding-level baselines trace other points on the restraint–capability frontier.
  • A.1 Training data and supervision: 15,419 tool-call examples and 15,245 response examples form the filtered GKD training splits from APIGen-MT, using disjoint conversation-level train, validation, and test partitions.The split uses 5,000 conversations with seed 42: 3,500 training, 500 validation, and 1,000 test.
  • A.1 Training data and supervision: Only the final assistant turn is supervised, and tool calls are pre-rendered as XML-style text for the ms-swift/Megatron backend.The rendering uses tags such as <tool call>, <function=...>, and <parameter=...> to prevent unrendered objects from corrupting the learned schema.
  • A.2 Tool-call target rendering: Structured tool-call examples are rendered as text before training, with illustrative targets containing <tool_call>, function, and parameter tags.Other frameworks may serialize the same structured call differently.

A.3 Main training configuration … B.5 Mode-mismatch replay and decision-support audit

The paper fixes a shared multi-teacher GKD configuration, defines mode-entry and training-process diagnostics, and audits mode mismatch through paired replay and full-vocabulary support analysis. These measurements show how truncated teacher support and trajectory mode entry affect divergence and decision-boundary behavior.

  • A.3 Main training configuration: All loss-comparison variants and support interventions use identical data, teachers, training length, batch settings, and seed-specific configurations, differing only in their modifier or support construction.Exact restoration and its placebo append one matched coordinate, while support union probes student-selected coordinates.
  • A.3 Main training configuration: λ = 0.8 student-rollout sampling is combined with a supervised anchor applied only to dataset-source trajectories across GKD variants.The supervised anchor is used for structured-output stability.
  • B.1 Evaluation metrics: Decision accuracy measures correct tool-use mode entry, while over-calling, call recall, and respond recall separately quantify erroneous calls and detection performance on should-call and should-respond examples.The balanced APIGen-MT-derived decision test makes decision accuracy the average of call recall and respond recall; schema validity is evaluated separately.
  • B.2 Training process metrics: Top-K JSD and squared-divergence proxy ratios are computed on renormalized teacher-top-32 support, with values below one indicating larger average response-example divergence or proxy divergence.The proxy is scale-sensitive rather than a parameter-gradient norm and reduces first-order effects from response length and batch composition.
  • B.2 Training process metrics: First-supervised-token diagnostics record student <tool call> probability, log-odds margins, and top-1 status to measure entry pressure, while full-generation evaluation captures later behavior.These metrics distinguish decision-boundary behavior at entry from downstream generated behavior.
  • B.3 Aggregate sanity diagnostics: 31 paired diagnostic steps per method retain one tool-call and one response sample every ten steps, with paired bootstrap intervals used for aggregate sanity diagnostics.The sparse full-sequence diagnostics do not establish a stable direction of teacher dominance because every reported interval includes one.
  • B.4 Token-level concentration diagnostics: Multi-seed concentration diagnostics report the JSD share carried by the largest 1%, 5%, and 10% of supervised tokens, alongside adjusted-to-raw squared-divergence shrinkage and threshold-event frequency.Values are mean±std over seeds 42, 44, and 60; lower concentration means fewer extreme tokens dominate token-level JSD mass.
  • B.5 Mode-mismatch replay and decision-support audit: 26.73 [14.87, 67.91] is the tool-teacher mismatched-to-aligned per-token JSD ratio, with 99.5% of paired prompts showing higher mismatched JSD after forced mode entry.The paired continuation intervention forces <tool call> for one trajectory and masks it for a response entry in the other, while later student generation remains normal.

B.6 Matched support-scope training … C.1 Additional APIGen-MT results

Matched support-scope restoration shows that broader temporal coverage changes full-generation behavior mainly after the first token, reducing over-calling but also call recall and exact success. Matched placebo, student-aware support union, and Soft Clamp further expose a restraint-capability trade-off without improving overall dialogue exact success.

  • B.6 Matched support-scope training: First-position restoration reduces mean response entry error from 14.47% to 3.40% but lowers mean call entry from 91.72% to 80.57%.First-token accuracy changes by only −0.04 points, while AUC changes from 0.9692 to 0.9712.
  • B.6 Matched support-scope training: 805 first-position calls (13.42%) become 184 (3.07%) on should-respond outputs, while delayed calls rise from 47 (0.78%) to 625 (10.42%).The no-call count changes only from 5,148 (85.80%) to 5,191 (86.52%), indicating call relocation rather than elimination.
  • B.6 Matched support-scope training: All-position support reduces APIGen over-calling by 10.10, 8.35, and 10.80 points across seeds 42, 44, and 60, while call recall loses 12.75, 10.70, and 13.75 points.E1 remains close to first-position support, so the complete-generation gap opens after the first token.
  • B.6 Matched support-scope training: All-position overall dialogue exact success is 3.38±0.50%, versus 7.04±0.31% for vanilla and 5.83±1.01% for first-position support.Empty-ground-truth no-call accuracy rises to 61.49±4.54%, while required-turn call coverage falls to 73.24±4.62%.
  • B.7 Matched placebo and student-aware support: Teacher/student overlap averages 25.12±0.08 tokens, so the student-aware support union averages 38.88±0.08 tokens rather than 64.The union appends student-top-32 tokens missing from teacher top-32 and queries their teacher logits.
  • B.7 Matched placebo and student-aware support: The matched non-tool placebo does not reproduce exact restoration, while the student-aware union gives a less conservative full-generation point and improves threshold-free E1 separation.None improves overall dialogue exact success.
  • B.7 Matched placebo and student-aware support: A response-side logit penalty (λ = 0.15) reduces APIGen over-calling from 14.20±2.08% to 12.93±1.14%, producing only a small operating-point shift.Exact restoration is characterized as a diagnostic intervention rather than a recommended objective.
  • C.1 Additional APIGen-MT results: Soft Clamp reduces over-calling relative to vanilla GKD while preserving decision accuracy, at the cost of lower call recall.Qualitative should-respond examples show Soft Clamp staying on the response side of the behavior boundary.

C.2 Inference-time tool-entry bias counterfactual · C.3 BFCL and When2Call tables

Inference-time scalar bias closely reproduces Soft Clamp’s tool-entry reduction but leaves residual generation-quality gaps, while support union and cost-sensitive evaluation show that preferred interventions depend on protocol and deployment costs. Strict BFCL and When2Call evaluations provide separate aggregate comparisons, with support union leading the displayed GKD overall mean in When2Call.

  • C.2 Inference-time tool-entry bias counterfactual: 11.2% versus 11.1% response entry error, but validation-tuned bias has lower call recall and accuracy than Soft Clamp.Call recall is 89.2% versus 90.4%, and accuracy is 89.0% versus 89.7%.
  • C.2 Inference-time tool-entry bias counterfactual: 8.7% response entry error with 90.6% call-entry recall and 0.9760 AUC, achieved by support union.These results are consistent with student-aware support improving the operating point.
  • C.2 Inference-time tool-entry bias counterfactual: ρ ∈{0.25, 0.5} favors vanilla GKD, whereas ρ ∈{1, 2, 4} favors support union on fixed APIGen methods.On E1, support union is lowest through ρ = 2; costaware bias becomes lowest at ρ = 4, with mean risks of 8.86% and 7.06%, respectively.
  • C.2 Inference-time tool-entry bias counterfactual: 14.5±1.5% first-token entry error and 14.3±2.2% full-generation over-calling characterize vanilla GKD, versus 11.1±0.3% and 9.0±0.2% for Soft Clamp.The disagreement between entry error and full-generation over-calling is protocol-specific because E1 and full generation use different backends.
  • C.2 Inference-time tool-entry bias counterfactual: Strict BFCL multi-turn comparisons use the same local first-token-bias decoder for every row, and rows should be compared only within Table 20.The main BFCL diagnostic uses vLLM, so backend, batching, and template differences can change absolute values.
  • C.2 Inference-time tool-entry bias counterfactual: Scalar bias nearly matches Soft Clamp on call frequency and loop rate but leaves worse mean non-tool-final and invalid-call rates.It is a strong entry-level counterfactual, while residual gaps under other inference-time calibration protocols remain open.
  • C.3 BFCL and When2Call tables: The support union has the highest displayed GKD overall mean in When2Call, while the Base reference remains higher overall.BFCL and When2Call table values are percentages, with GKD rows reported as mean±std over three seeds.

D Secondary Qwen3.5-4B Student-Scale Check

A Qwen3.5-4B student-scale check with fixed 9B teachers tested whether loss-calibration effects persist at smaller scale, before support-level interventions. Global Reweight and Soft Clamp reduced over-calling and Loop@3 rates versus vanilla, but neither improved overall dialogue exact success.

  • Experimental scope: The experiment replaced the Qwen3.5-9B student with Qwen3.5-4B while retaining fixed 9B teachers, testing loss calibration rather than support-aware correction.The experiment predates support-level interventions and is not evidence for support-aware correction.
  • Loss-calibration results: 12.8±0.7% over-calling fell to 9.5±0.8% with Global Reweight and 9.2±1.2% with Soft Clamp.These results preserve the loss-shaping scale check at the smaller student scale.
  • Loss-calibration results: 8.9±1.4% and 9.2±2.6% Loop@3 rates for Global Reweight and Soft Clamp were below vanilla’s 15.7±3.2%, but neither raised overall dialogue exact success.The rates are reported for Global Reweight and Soft Clamp, respectively, versus vanilla.

E Llama-3.1 Cross-Family Replication … G Reproducibility Checklist

The Llama-3.1 replication reproduces decision-critical support omission: the response teacher retains nearly all probability mass while omitting the JSON-entry coordinate, whereas the tool teacher directly reinforces it. The matched comparisons use fixed training conditions and three seeds, with artifacts and evaluation materials stored for reproducibility.

  • E Llama-3.1 Cross-Family Replication: Llama-3.1-8B-Instruct uses native assistant JSON for tool-call SFT, native tool-response roles, and two-epoch mixed, response-only, and tool-only checkpoints.Mixed SFT is a separate supervised reference; Vanilla GKD, Soft Clamp, and support union initialize from the original Instruct checkpoint.
  • E Llama-3.1 Cross-Family Replication: Llama single-turn GKD results report mean±sample standard deviation over three seeds, and all GKD variants remain below mixed SFT on When2Call.Base and SFT are single references, while the one-sided SFT teachers verify intended routing extremes.
  • E Llama-3.1 Cross-Family Replication: 99.977% retained mass still omits the JSON-entry coordinate on every response prompt, while the tool teacher contains it on every tool prompt.The student-aware union recovers nearly the full-vocabulary correction with 40.49 coordinates on average.
  • E Llama-3.1 Cross-Family Replication: 105,952.5 is the median response-teacher entry rank, spanning 10,381–127,884, while union entry coverage reaches 99.2%, 97.6%, and 97.0% for seeds 42, 44, and 60.The tool teacher ranks the entry token first on all 500 tool prompts.
  • F.1 Vanilla GKD: Vanilla GKD uses the original token divergence d_i without additional calibration as the reference for testing whether loss modifiers reduce behavior imbalance.It is the baseline for the loss-modifier comparisons.
  • F.2 Hard Clip: c = 0.5 defines Hard Clip, which truncates extreme token losses by removing marginal gradients above a fixed threshold.It is a practical contrast rather than a mechanism-matched control and affects a different number of tokens from Soft Clamp.
  • F.3 Global Reweight: Global Reweight uses α = 0.3, z_max = 3.0, w_min = 0.25, and w_max = 2.0 to reweight all tokens by batch-relative divergence.Soft Clamp instead leaves non-extreme tokens unchanged.
  • F.4 Soft Clamp: Soft Clamp uses k = 3.0 with a detached dynamic threshold C = stopgrad(k mean_i(d_i)); capped tokens retain a nonzero gradient scaled by C/d_i.Detaching the threshold prevents gradients from flowing through the batch mean.

H BFCL Multi-turn Loop Diagnostic

The BFCL multi-turn diagnostic tests whether stronger tool-call priors reduce usability in interactive tool environments, using simulated observations and dialogue continuation. Excluding unsupported missing-function cases raises vanilla dialogue exact success to 9.39%, but fewer loops still do not improve endpoint correctness.

  • Diagnostic setup: The diagnostic evaluates 800 BFCL v4 multi-turn tasks and 3,136 user turns with fixed simulated tool observations returned after each emitted call.The harness continues until a non-tool final answer or the maximum step limit.
  • Interaction diagnostics: The analysis measures tool-call frequency, looping, maximum-step hits, repeated calls, termination, and overall dialogue exact success.Because the harness stops after five tool-call steps, Loop@5 and Max-step are numerically identical.
  • Correctness criterion: A dialogue passes only when required-turn executions match reference observations and environment state, while empty-ground-truth turns contain no executable tool call.Both conditions must hold across the complete dialogue.
  • Results: 9.39% vanilla dialogue exact success follows exclusion of the unsupported missing-function category, alongside 35.57% turn-level protocol success.The passage reports that fewer loops still do not improve endpoint correctness.
Loading 2607.07050v4…