PaperlayerView exact revision on arXiv

Source-linked AI summary

ASA: Backbone-Training-Free Representation Engineering for Tool-Calling Agents

Youjin Wang, Run Zhou, Yingjie Ma, Rong Fu, Jiani Liang, Shuaishuai Cao, Min Huang, Tao Fang, Liangming Pan

arXiv:2602.04935v3cs.SEcs.AI

TL;DR

Domain-specific tool calling is brittle because readable tool-use intent does not reliably become parser-valid execution. ASA addresses this gap with a backbone-training-free, inference-time controller that combines shared boundary and domain-local steering with signed gating, improving executable tool calling while leaving schema-realization errors unresolved.

  • Problem

    Domain-specific tool calling remains brittle under evolving interfaces, and readable hidden-state tool-use evidence does not guarantee parser-valid execution.

  • Method

    ASA is a backbone-training-free inference-time controller using shared boundary steering, domain-local residual steering, and a probe-guided signed gate.

  • Results

    ASA improves executable tool-use behavior on NESTFUL and BFCL by reducing missing-tool failures and improving first-call and sequence accuracy.

  • Takeaways & Limitations

    Intent signals benefit from control, but fine-grained schema realization, nested binding, and argument precision remain challenging.

  • Takeaways & Limitations

    The experiments do not fully isolate each controller component, and ASA does not fully solve post-boundary schema realization.

Abstract

from arXiv · show

Adapting LLM agents to domain-specific tool calling remains notably brittle under evolving interfaces. Prompt and schema engineering is easy to deploy but often fragile under distribution shift and strict parsers, while continual parameter-efficient fine-tuning improves reliability at the cost of training, maintenance, and potential forgetting. We identify a critical Lazy Agent failure mode where tool necessity is nearly perfectly decodable from mid-layer activations, yet the model remains conservative in entering tool mode, revealing a representation-behavior gap. We propose Activation Steering Adapter (ASA), a training-free, inference-time controller that performs a single-shot mid-layer intervention and targets tool domains via a router-conditioned mixture of steering vectors with a probe-guided signed gate to amplify true intent while suppressing spurious triggers. On MTU-Bench with Qwen2.5-1.5B, ASA improves strict tool-use F1 from 0.18 to 0.50 while reducing the false positive rate from 0.15 to 0.05, using only about 20KB of portable assets and no weight updates.

1 Introduction

Domain-specific tool calling remains brittle because models can recognize tool needs yet fail to produce valid calls under changing schemas and strict execution. ASA addresses this representation–behavior gap with training-free hidden-state control and improves tool-use reliability.

  • Motivation: Tool calls can fail through missed invocation, invalid tools, malformed schemas, or omitted nested arguments, especially across changing domains.Domain-specific interfaces vary in tool names, schemas, argument meanings, and calling conventions.
  • Motivation: Prompting and parameter-efficient adaptation improve tool use but remain sensitive to interface changes or require retraining and regression testing.Prompt-only methods are affected by wording and context length, while tuning methods incur maintenance costs.
  • Intent–Execution Gap: Readable tool-use evidence mainly predicts crossing the parser-recognizable boundary into tool mode, not complete executable success.Execution still requires valid tools, arguments, formats, and trajectory-level constraints.
  • Approach: ASA is a backbone-training-free inference controller that combines shared boundary steering, domain-local residual steering, and a probe-guided signed gate.The gate can open the tool boundary, suppress spurious crossing, or abstain under uncertainty.
  • Results: ASA reduces missing-tool failures and improves executable accuracy on NESTFUL and BFCL, while fine-grained schema realization and nested argument binding remain residual bottlenecks.The error analysis identifies boundary-entry and trajectory-continuation failures as the main corrected categories.

2 Related Work

Prior work improves tool use through elicitation, adaptation, constrained decoding, and activation steering. ASA targets the stricter problem of parser-valid execution by controlling the hidden-state boundary between answering and tool calling.

  • Tool-use models: ReAct, Toolformer, ToolLLM, Gorilla, and later function-calling models expand API use through elicitation, self-supervision, scaling, or specialization.These approaches establish tool use across increasingly broad API collections and model settings.
  • Adaptation and decoding: Prompting, constrained decoding, supervised tuning, reinforcement learning, and parameter-efficient adapters improve invocation or format compliance but add adaptation or robustness considerations.LoRA and QLoRA reduce adaptation cost without eliminating the need to fit additional parameters.
  • Representation control: Activation steering shifts intermediate representations without changing prompts or backbone weights and has been extended to conditional behavior and agent policies.ASA applies this inference-time control paradigm to parser-defined tool execution.
  • Diagnostics: Pre-generation probes can decode tool decisions, and tool identity can be linearly read and steered in fixed-menu settings.These findings motivate Probe&Prefill and Tool-identity steering as controls for ASA.
  • Positioning: ASA asks whether readable tool-use evidence guarantees boundary entry and correct execution under strict evaluation, and the paper shows that it does not.The target is stricter than merely eliciting tool use.

3 Problem Setup

The paper models tool calling as parser-defined execution over hidden representations, separating boundary entry from post-entry validity. This setup distinguishes tool necessity, parser triggers, schema realization, and domain-specific representations.

  • Execution model: An input instruction x produces an autoregressive output y, with an oracle label indicating whether a tool call is required.The label distinguishes Tool-Necessary inputs from cases where a direct answer is sufficient.
  • Execution model: A deterministic parser maps generated text to a binary trigger event indicating whether the model enters tool mode.Boundary entry is defined by the parser-visible trigger rather than by latent intent alone.
  • Execution model: Strict schema realization requires boundary entry plus valid formatting, tool names, arguments, and applicable task-level or multi-turn correctness.The objective improves strict success on Tool-Necessary inputs while preserving no-trigger behavior on Non-Tool inputs without modifying model weights.
  • Research question: The central question is whether pre-generation tool-use evidence readable from hidden states is sufficient for parser-valid execution.This question motivates separating trigger behavior from post-entry validity.
  • Representations: Representations are taken from the final non-padding prompt token at a selected Pre-LN residual-stream layer and standardized using training-split statistics.The standardized representation uses the training mean and standard deviation.
  • Domains and probes: A domain denotes a tool-schema family sharing tool names, argument semantics, and executable output conventions.The domain is not defined as a topical category.
  • Domains and probes: Probe scores are trained and selected on non-test splits and rank Tool-Necessary examples into diagnostic top-ρ buckets.These buckets compare readable evidence, parser-visible entry, and strict executable success.

4 Evidence for the Intent–Execution Gap

Mid-layer representations contain readable and steerable tool-use evidence, but that evidence does not guarantee strict execution. Probe-ranked examples often enter tool mode while failing parser-valid output requirements.

  • Diagnostic design: The diagnostic setup tests whether readable tool-use evidence before decoding guarantees parser-valid execution.It first establishes readability, then compares that evidence with strict execution outcomes.
  • Evidence readability: A linear probe separates Tool-Necessary from Non-Tool inputs with approximately 0.84 AUC, while shuffled labels remain near chance.Steering along the learned direction shifts trigger-token evidence more than a norm-matched random direction.
  • Evidence readability: The frozen backbone therefore contains a readable and causally relevant tool-use signal, but the signal’s sufficiency for executable success remains the stricter question.Readability and behavioral realization are evaluated separately.
  • Probe–trigger dissociation: High probe evidence predicts boundary entry much better than schema realization under strict execution.Table 2 frames the dissociation between probe scores, triggers, and strict success.
  • Probe–trigger dissociation: In the top 10% probe bucket, the baseline enters tool mode on all examples, but only 25.0% pass strict evaluation.Across top 25% and top 50% buckets, trigger recall remains 97.5% while strict success stays at 20.0%.
  • Implication: The dissociation motivates a controller that opens or suppresses the tool boundary and adds schema-local bias after entry.ASA combines shared boundary and domain-local directions with a signed gate.

5 Proposed Method

ASA is a backbone-training-free controller that constructs shared and domain-local steering directions from frozen representations, then applies a signed gate for inference-time tool-mode intervention. It supports both one-shot prefill and trajectory-level state-cascade correction under strict executable evaluation.

  • Controller construction: ASA constructs a gated controller from frozen-backbone representations using shared boundary and domain-local directions.The shared direction targets tool-mode entry, while local residual directions capture schema-specific variation without claiming perfect semantic factorization.
  • Inference instantiations: ASA applies the gated hidden-state update either once before decoding with ASA-Prefill or repeatedly across generation states with ASA-StateCascade.Prefill uses a single forward hook, while StateCascade recomputes features for multi-turn and nested trajectories without updating backbone parameters.
  • Controller construction: A calibration split estimates the shared boundary direction at layer L as the displacement from non-tool behavior into tool mode.The calibration data are separate from evaluation, and the direction is derived from final-token residual representations.
  • Signed abstention gate: At inference, a lightweight predictor identifies the tool-schema domain and its confidence before domain-specific evidence and boundary-readiness signals are computed.The predictor outputs the domain estimate, while a domain-specific probe estimates tool-use evidence and a calibration map produces readiness.
  • Signed abstention gate: The signed boundary action ab(x) ∈ {−1, 0, +1} opens the tool boundary, suppresses spurious crossing, or abstains.The action is determined by tool-use evidence and readiness thresholds; schema-local steering activates only when the boundary is reachable and the domain estimate is reliable.

6 Experiments

ASA is evaluated under strict parser-defined interfaces on NESTFUL and BFCL with Qwen3-8B, using disjoint calibration data and trajectory-level control for multi-step settings.

  • Protocol: Calibration examples, probes, thresholds, and hyperparameters are selected without access to held-out evaluation data.Experiments use greedy decoding throughout and retain ASA-Prefill as a diagnostic one-shot variant.
  • NESTFUL: ASA raises NESTFUL first-call accuracy from 24.46% to 41.94% and sequence accuracy from 16.94% to 25.00%.NESTFUL measures executable nested function calling, including first emitted calls and complete trajectories.
  • NESTFUL: ASA reduces NESTFUL missing-tool failures from 59.14% to 6.72%.The missing-tool rate measures failures to call a required tool.
  • BFCL: ASA improves BFCL multi-turn success from 32.50% to 38.75% and non-live single-turn AST accuracy from 89.40% to 95.60%.BFCL also evaluates live single-turn executable-subset AST accuracy and argument-level F1.
  • BFCL: ASA improves BFCL live single-turn AST accuracy from 72.50% to 77.00%.Multi-turn columns use official Prompt-mode success, while single-turn columns use executable-subset AST accuracy.

7 Analysis

Error-level analysis separates boundary entry from post-entry validity and shows that ASA primarily corrects missing entry and trajectory failures, while selective gating protects executable precision.

  • Failure analysis: Strict failure occurs either when boundary entry is missing or when post-entry validity fails for tools, arguments, formats, or trajectories.For Non-Tool inputs, over-triggering corresponds to entering tool mode when no tool is required.
  • Failure analysis: ASA mainly corrects missing boundary entry and trajectory-continuation errors, while schema realization and nested argument binding remain dominant residual failures.The analysis decomposes strict execution into parser-visible boundary entry and post-entry validity.
  • Failure analysis: ASA improves call recall while reducing false calls, tool-name errors, and path-level mismatches across the reported diagnostics.The paired transition breakdown reports 11 corrected baseline failures and one degraded baseline-correct case.
  • Ablations: ASA-StateCascade abstains on most BFCL generation states and intervenes selectively, concentrating interventions on high-evidence states.The controller uses state-dependent actions rather than unconditional intervention at every generation step.
  • Ablations: Removing the gate sharply increases false-positive trigger rate and lowers success precision.Success precision is the fraction of triggered outputs that pass strict execution validation.
  • Ablations: Random directions do not reproduce the gain, while global-only steering is weaker than full ASA.A cross-backbone sanity check reports a similar distinction between boundary-entry prefilling and trajectory-level ASA-StateCascade control.

8 Conclusion

The paper separates boundary entry from schema realization and applies gated shared–local control over a frozen backbone. Across NESTFUL and BFCL, ASA reduces missing-tool failures while improving executable accuracy, though schema realization and nested binding remain challenging.

  • Conclusion: ASA studies schema-constrained tool calling by separating boundary entry from post-entry schema realization.Hidden-state tool-use evidence does not by itself ensure execution success.
  • Conclusion: ASA uses a gated shared–local controller over a frozen backbone, with shared steering for boundary behavior and local residual steering for schema bias.The local residual is applied when domain estimates are reliable.
  • Conclusion: Across NESTFUL and BFCL, ASA reduces missing-tool failures and improves first-call and sequence accuracy.The conclusion summarizes the reported executable tool-calling gains across both benchmarks.
  • Conclusion: Nested binding and fine-grained schema realization remain challenging after ASA’s improvements.These are identified as remaining error sources rather than resolved aspects of execution.

Limitations

The study is limited to offline schema-constrained tool calling under fixed parser-defined evaluators, and its experiments do not fully isolate each controller component. Post-boundary schema realization, argument binding, and decoding-budget effects remain unresolved.

  • The evaluation focuses on offline schema-constrained tool calling under fixed parser-defined evaluators.
  • NESTFUL sequence-accuracy gains combine boundary entry, state-dependent gating, and schema-sensitive residual control rather than isolating each channel.
  • Post-boundary schema realization remains incomplete, with nested argument precision and recall still lagging.
  • Variable binding and argument propagation require finer-grained control than the current linear residual directions provide.
  • Matched decoding-budget comparisons and finer-grained executable benchmarks are needed to strengthen causal interpretation of controller components.

D Representation and Causal Diagnostics

Representation diagnostics show that tool-use directions are only partially aligned across domains, while the selected-layer geometry and boundary-token analyses support combining shared and domain-local steering.

  • Low-to-moderate off-diagonal cosine similarities show that domain tool-use directions are only partially aligned.
  • Code–Translation directions are moderately aligned, whereas Search–Translation directions are nearly orthogonal.
  • The shared boundary direction captures generic tool-mode movement, while domain-local residuals preserve schema-specific variation.
  • Figure 4 selects intervention depth through a probe sweep, and Figure 5 connects the selected direction to geometry and boundary-token behavior.
  • The diagnostics support ASA’s shared boundary component plus domain-local residual control because tool-use evidence is readable but not purely global.

E NESTFUL Implementation Details

The implementation evaluates ASA-StateCascade with deterministic, state-level interventions across NESTFUL and BFCL, using frozen Qwen3-8B backbones and disjoint calibration data. The controller combines gated boundary steering, domain-sensitive residuals, and trajectory-aware actions.

  • NESTFUL: NESTFUL uses Qwen3-8B in bfloat16 on one A100-40GB GPU, with hidden size 4096 and 36 transformer layers.
  • NESTFUL: ASA-StateCascade uses greedy decoding, max_new_tokens=2048, tokenizer max_length=16384, and probe layer 24.
  • NESTFUL: Active NESTFUL steering layers are 8, 12, 16, 20, 24, and 28, while layers 30 and 32 are excluded for dimensional incompatibility.
  • NESTFUL: The NESTFUL trajectory-level controller combines boundary steering, schema-sensitive residual control, and state-dependent gating without separately identifying each contribution.
  • NESTFUL: Most interventions correct boundary entry, while a substantial fraction prevents premature or harmful stopping.
  • BFCL: BFCL uses a fixed local Prompt-mode pipeline with greedy decoding, and calibration examples are disjoint from evaluation data.
  • BFCL: The BFCL evaluator requires exact function names, argument values, and rejection behavior for official success, with turn-level and path diagnostics reported separately.
  • BFCL: ASA improves every reported BFCL category, with the largest category-level gain on missing-parameter examples.

K.5 Detailed Rescue Analysis

Detailed rescue analyses indicate that ASA is strongest on near-boundary strict failures, while prompt-only control exposes a recall–false-positive trade-off and domain conditioning supports schema consistency. Cross-backbone checks show that trajectory-level control improves both first-call and sequence accuracy where one-shot prefill does not.

  • Rescue analysis: ASA has its largest advantage on the highest-ranked strict failures and remains above random steering as the subset expands.
  • Rescue analysis: The rescue pattern supports greater effectiveness on near-boundary failures, while deeper schema realization and argument binding remain harder.
  • Prompt comparison: Prompt-only control illustrates a recall–FPR trade-off: stronger demonstrations increase tool-trigger recall but also false triggers.
  • Cross-backbone check: On LLaMA-3.1, Probe-Prefill reduces missing-tool rate from 0.0323 to 0.0215 but does not improve sequence accuracy.
  • Cross-backbone check: On LLaMA-3.1, ASA-StateCascade improves first-call accuracy from 0.3414 to 0.3710.
  • Ablations: The ablations show that full ASA improves trigger trade-offs and success precision, while removing the gate increases FPR and random directions fail to reproduce gains.
  • Ablations: Domain mismatch primarily affects tool-name validity, indicating that direction selection matters for schema consistency.
Loading 2602.04935v3…