ASA: Backbone-Training-Free Representation Engineering for Tool-Calling Agents

Motivation

• Tool calls can fail through missed invocation, invalid tools, malformed schemas, or omitted nested arguments, especially across changing domains.Domain-specific interfaces vary in tool names, schemas, argument meanings, and calling conventions.Source: §1 · p. 1
• Prompting and parameter-efficient adaptation improve tool use but remain sensitive to interface changes or require retraining and regression testing.Prompt-only methods are affected by wording and context length, while tuning methods incur maintenance costs.Source: §1 · p. 1

Intent–Execution Gap

• Readable tool-use evidence mainly predicts crossing the parser-recognizable boundary into tool mode, not complete executable success.Execution still requires valid tools, arguments, formats, and trajectory-level constraints.Source: §1 · pp. 1–2

Approach

• ASA is a backbone-training-free inference controller that combines shared boundary steering, domain-local residual steering, and a probe-guided signed gate.The gate can open the tool boundary, suppress spurious crossing, or abstain under uncertainty.Source: §1 · p. 2

Results

• ASA reduces missing-tool failures and improves executable accuracy on NESTFUL and BFCL, while fine-grained schema realization and nested argument binding remain residual bottlenecks.The error analysis identifies boundary-entry and trajectory-continuation failures as the main corrected categories.Source: §1 · p. 2

Tool-use models

• ReAct, Toolformer, ToolLLM, Gorilla, and later function-calling models expand API use through elicitation, self-supervision, scaling, or specialization.These approaches establish tool use across increasingly broad API collections and model settings.Source: §2.1 · p. 2

Adaptation and decoding

• Prompting, constrained decoding, supervised tuning, reinforcement learning, and parameter-efficient adapters improve invocation or format compliance but add adaptation or robustness considerations.LoRA and QLoRA reduce adaptation cost without eliminating the need to fit additional parameters.Source: §2.1 · p. 2

Representation control

• Activation steering shifts intermediate representations without changing prompts or backbone weights and has been extended to conditional behavior and agent policies.ASA applies this inference-time control paradigm to parser-defined tool execution.Source: §2.2 · p. 2

Diagnostics

• Pre-generation probes can decode tool decisions, and tool identity can be linearly read and steered in fixed-menu settings.These findings motivate Probe&Prefill and Tool-identity steering as controls for ASA.Source: §2.2 · p. 2

Positioning

• ASA asks whether readable tool-use evidence guarantees boundary entry and correct execution under strict evaluation, and the paper shows that it does not.The target is stricter than merely eliciting tool use.Source: §2.1 · p. 2, §2.2 · p. 3

Execution model

• An input instruction x produces an autoregressive output y, with an oracle label indicating whether a tool call is required.The label distinguishes Tool-Necessary inputs from cases where a direct answer is sufficient.Source: §3.1 · p. 3
• A deterministic parser maps generated text to a binary trigger event indicating whether the model enters tool mode.Boundary entry is defined by the parser-visible trigger rather than by latent intent alone.Source: §3.1 · p. 3
• Strict schema realization requires boundary entry plus valid formatting, tool names, arguments, and applicable task-level or multi-turn correctness.The objective improves strict success on Tool-Necessary inputs while preserving no-trigger behavior on Non-Tool inputs without modifying model weights.Source: §3.1 · p. 3

Research question

• The central question is whether pre-generation tool-use evidence readable from hidden states is sufficient for parser-valid execution.This question motivates separating trigger behavior from post-entry validity.Source: §3.1 · p. 3

Representations

• Representations are taken from the final non-padding prompt token at a selected Pre-LN residual-stream layer and standardized using training-split statistics.The standardized representation uses the training mean and standard deviation.Source: §3.2 · p. 3

Domains and probes

• A domain denotes a tool-schema family sharing tool names, argument semantics, and executable output conventions.The domain is not defined as a topical category.Source: §3.2 · p. 3
• Probe scores are trained and selected on non-test splits and rank Tool-Necessary examples into diagnostic top-ρ buckets.These buckets compare readable evidence, parser-visible entry, and strict executable success.Source: §3.2 · p. 3

Diagnostic design

• The diagnostic setup tests whether readable tool-use evidence before decoding guarantees parser-valid execution.It first establishes readability, then compares that evidence with strict execution outcomes.Source: §4 · p. 3, §4.1 · p. 3

Evidence readability

• A linear probe separates Tool-Necessary from Non-Tool inputs with approximately 0.84 AUC, while shuffled labels remain near chance.Steering along the learned direction shifts trigger-token evidence more than a norm-matched random direction.Source: §4.1 · p. 3
• The frozen backbone therefore contains a readable and causally relevant tool-use signal, but the signal’s sufficiency for executable success remains the stricter question.Readability and behavioral realization are evaluated separately.Source: §4.1 · p. 3

Probe–trigger dissociation

• High probe evidence predicts boundary entry much better than schema realization under strict execution.Table 2 frames the dissociation between probe scores, triggers, and strict success.Source: Table 2 · p. 4
• In the top 10% probe bucket, the baseline enters tool mode on all examples, but only 25.0% pass strict evaluation.Across top 25% and top 50% buckets, trigger recall remains 97.5% while strict success stays at 20.0%.Source: §4.2 · p. 4

Implication

• The dissociation motivates a controller that opens or suppresses the tool boundary and adds schema-local bias after entry.ASA combines shared boundary and domain-local directions with a signed gate.Source: §4.3 · p. 4

Controller construction

• ASA constructs a gated controller from frozen-backbone representations using shared boundary and domain-local directions.The shared direction targets tool-mode entry, while local residual directions capture schema-specific variation without claiming perfect semantic factorization.Source: §5 · p. 4, §5.1 · p. 4

Inference instantiations

• ASA applies the gated hidden-state update either once before decoding with ASA-Prefill or repeatedly across generation states with ASA-StateCascade.Prefill uses a single forward hook, while StateCascade recomputes features for multi-turn and nested trajectories without updating backbone parameters.Source: §5 · p. 4, §5.2 · p. 5, §5.3 · p. 6

Controller construction

• A calibration split estimates the shared boundary direction at layer L as the displacement from non-tool behavior into tool mode.The calibration data are separate from evaluation, and the direction is derived from final-token residual representations.Source: §5.1 · p. 4

Signed abstention gate

• At inference, a lightweight predictor identifies the tool-schema domain and its confidence before domain-specific evidence and boundary-readiness signals are computed.The predictor outputs the domain estimate, while a domain-specific probe estimates tool-use evidence and a calibration map produces readiness.Source: §5.2 · pp. 4–5
• The signed boundary action ab(x) ∈ {−1, 0, +1} opens the tool boundary, suppresses spurious crossing, or abstains.The action is determined by tool-use evidence and readiness thresholds; schema-local steering activates only when the boundary is reachable and the domain estimate is reliable.Source: §5.2 · p. 5

Protocol

• Calibration examples, probes, thresholds, and hyperparameters are selected without access to held-out evaluation data.Experiments use greedy decoding throughout and retain ASA-Prefill as a diagnostic one-shot variant.Source: §6.1 · p. 6, §6 · p. 6

NESTFUL

• ASA raises NESTFUL first-call accuracy from 24.46% to 41.94% and sequence accuracy from 16.94% to 25.00%.NESTFUL measures executable nested function calling, including first emitted calls and complete trajectories.Source: §6.2 · p. 6, §6.1 · p. 6
• ASA reduces NESTFUL missing-tool failures from 59.14% to 6.72%.The missing-tool rate measures failures to call a required tool.Source: §6.2 · p. 6, §6.1 · p. 6

BFCL

• ASA improves BFCL multi-turn success from 32.50% to 38.75% and non-live single-turn AST accuracy from 89.40% to 95.60%.BFCL also evaluates live single-turn executable-subset AST accuracy and argument-level F1.Source: §6.2 · p. 6, §6.1 · p. 6, Table 4 · p. 7
• ASA improves BFCL live single-turn AST accuracy from 72.50% to 77.00%.Multi-turn columns use official Prompt-mode success, while single-turn columns use executable-subset AST accuracy.Source: §6.2 · p. 7, Table 4 · p. 7

Failure analysis

• Strict failure occurs either when boundary entry is missing or when post-entry validity fails for tools, arguments, formats, or trajectories.For Non-Tool inputs, over-triggering corresponds to entering tool mode when no tool is required.Source: §7.1 · p. 8
• ASA mainly corrects missing boundary entry and trajectory-continuation errors, while schema realization and nested argument binding remain dominant residual failures.The analysis decomposes strict execution into parser-visible boundary entry and post-entry validity.Source: §7.1 · p. 8, Table 5 · p. 8
• ASA improves call recall while reducing false calls, tool-name errors, and path-level mismatches across the reported diagnostics.The paired transition breakdown reports 11 corrected baseline failures and one degraded baseline-correct case.Source: §7.1 · p. 8

Ablations

• ASA-StateCascade abstains on most BFCL generation states and intervenes selectively, concentrating interventions on high-evidence states.The controller uses state-dependent actions rather than unconditional intervention at every generation step.Source: Figure 2 · p. 8
• Removing the gate sharply increases false-positive trigger rate and lowers success precision.Success precision is the fraction of triggered outputs that pass strict execution validation.Source: §7.2 · p. 8
• Random directions do not reproduce the gain, while global-only steering is weaker than full ASA.A cross-backbone sanity check reports a similar distinction between boundary-entry prefilling and trajectory-level ASA-StateCascade control.Source: §7.2 · p. 8

Conclusion

• ASA studies schema-constrained tool calling by separating boundary entry from post-entry schema realization.Hidden-state tool-use evidence does not by itself ensure execution success.Source: §8 · p. 8
• ASA uses a gated shared–local controller over a frozen backbone, with shared steering for boundary behavior and local residual steering for schema bias.The local residual is applied when domain estimates are reliable.Source: §8 · p. 9
• Across NESTFUL and BFCL, ASA reduces missing-tool failures and improves first-call and sequence accuracy.The conclusion summarizes the reported executable tool-calling gains across both benchmarks.Source: §8 · p. 9
• Nested binding and fine-grained schema realization remain challenging after ASA’s improvements.These are identified as remaining error sources rather than resolved aspects of execution.Source: §8 · p. 9

• The evaluation focuses on offline schema-constrained tool calling under fixed parser-defined evaluators.Source: Limitations · p. 9
• NESTFUL sequence-accuracy gains combine boundary entry, state-dependent gating, and schema-sensitive residual control rather than isolating each channel.Source: Limitations · p. 9
• Post-boundary schema realization remains incomplete, with nested argument precision and recall still lagging.Source: Limitations · p. 9
• Variable binding and argument propagation require finer-grained control than the current linear residual directions provide.Source: Limitations · p. 9
• Matched decoding-budget comparisons and finer-grained executable benchmarks are needed to strengthen causal interpretation of controller components.Source: Limitations · p. 9

• Low-to-moderate off-diagonal cosine similarities show that domain tool-use directions are only partially aligned.Source: D Representation and Causal Diagnostics · p. 14
• Code–Translation directions are moderately aligned, whereas Search–Translation directions are nearly orthogonal.Source: D Representation and Causal Diagnostics · p. 14
• The shared boundary direction captures generic tool-mode movement, while domain-local residuals preserve schema-specific variation.Source: D Representation and Causal Diagnostics · p. 14
• Figure 4 selects intervention depth through a probe sweep, and Figure 5 connects the selected direction to geometry and boundary-token behavior.Source: D Representation and Causal Diagnostics · p. 14, Figure 5 · p. 15
• The diagnostics support ASA’s shared boundary component plus domain-local residual control because tool-use evidence is readable but not purely global.Source: D Representation and Causal Diagnostics · p. 14

NESTFUL

• NESTFUL uses Qwen3-8B in bfloat16 on one A100-40GB GPU, with hidden size 4096 and 36 transformer layers.Source: E NESTFUL Implementation Details · p. 15
• ASA-StateCascade uses greedy decoding, max_new_tokens=2048, tokenizer max_length=16384, and probe layer 24.Source: E NESTFUL Implementation Details · p. 15
• Active NESTFUL steering layers are 8, 12, 16, 20, 24, and 28, while layers 30 and 32 are excluded for dimensional incompatibility.Source: E NESTFUL Implementation Details · p. 15
• The NESTFUL trajectory-level controller combines boundary steering, schema-sensitive residual control, and state-dependent gating without separately identifying each contribution.Source: E NESTFUL Implementation Details · p. 15
• Most interventions correct boundary entry, while a substantial fraction prevents premature or harmful stopping.Source: E NESTFUL Implementation Details · p. 15, Table 11 · p. 16

BFCL

• BFCL uses a fixed local Prompt-mode pipeline with greedy decoding, and calibration examples are disjoint from evaluation data.Source: F BFCL State-Cascade Implementation · p. 16, §I.1 · p. 17, §I.2 · p. 17
• The BFCL evaluator requires exact function names, argument values, and rejection behavior for official success, with turn-level and path diagnostics reported separately.Source: F BFCL State-Cascade Implementation · p. 16, Table 12 · p. 16
• ASA improves every reported BFCL category, with the largest category-level gain on missing-parameter examples.Source: Table 13 · p. 17, §K.3 · p. 20

Rescue analysis

• ASA has its largest advantage on the highest-ranked strict failures and remains above random steering as the subset expands.Source: §K.5 · p. 21
• The rescue pattern supports greater effectiveness on near-boundary failures, while deeper schema realization and argument binding remain harder.Source: §K.5 · p. 21

Prompt comparison

• Prompt-only control illustrates a recall–FPR trade-off: stronger demonstrations increase tool-trigger recall but also false triggers.Source: §K.7 · p. 21

Cross-backbone check

• On LLaMA-3.1, Probe-Prefill reduces missing-tool rate from 0.0323 to 0.0215 but does not improve sequence accuracy.Source: L Cross-Backbone and Variant Sanity… · p. 22, Table 26 · p. 22
• On LLaMA-3.1, ASA-StateCascade improves first-call accuracy from 0.3414 to 0.3710.Source: L Cross-Backbone and Variant Sanity… · p. 22

Ablations

• The ablations show that full ASA improves trigger trade-offs and success precision, while removing the gate increases FPR and random directions fail to reproduce gains.Source: O Additional Ablations · p. 24, Table 29 · p. 24, Table 30 · p. 24
• Domain mismatch primarily affects tool-name validity, indicating that direction selection matters for schema consistency.Source: O Additional Ablations · p. 24