PaperlayerView exact revision on arXiv

Source-linked AI summary

TACIT-Switch: Cost-Aware Model Escalation for LLM Agents from Censored Supervision

Ji'an Lei, Jian Huang

arXiv:2608.27911v1cs.LG

TL;DR

Routing agents must balance cheaper but less reliable backbones against stronger but costlier ones while deciding when intervention remains recoverable. TACIT-SWITCH learns a permanent handoff from paired Cheap–Strong outcomes and coarse teacher intervals on cumulative risk. It improves success at comparable cost in controlled simulation and ranks first among learned policies on the reported interactive benchmarks.

  • Problem

    Interactive agents need routing decisions that account for evolving trajectory evidence while balancing reliability against inference cost.

  • Method

    TACIT-SWITCH fits a mixture-cure threshold model to paired Cheap–Strong outcomes and interval-censored teacher supervision, then permanently hands off using a development-selected threshold without an online teacher.

  • Results

    TACIT-SWITCH achieves better held-out success–cost trade-offs than task-, step-, and fixed-prefix routing, ranking first among learned policies on ALFWorld and DABench.

  • Takeaways & Limitations

    Task features and cumulative trajectory risk provide complementary information for cost-aware permanent handoff decisions within the evaluated settings.

  • Takeaways & Limitations

    The method permits only one permanent handoff, depends on an application-specific cost function, and can be affected by annotation bias or distribution shift.

Abstract

from arXiv · show

Agents with smaller language-model backbones are less expensive but can drift into persistent failure modes, whereas those with larger backbones are generally more reliable but more costly. This reliability-cost trade-off motivates routing methods that decide when to invoke an agent with a larger backbone: before execution, after a fixed trajectory prefix, or locally at individual steps. Our method, TACIT-SWITCH, learns permanent handoff policies from accumulated trajectory evidence and Teacher-Annotated Censored Intervention Times (TACIT). It represents each annotation as an interval-censored observation on a cumulative-risk scale. The resulting mixture-cure threshold model estimates the probability that the paired Strong rollout succeeds and, conditional on success, the handoff threshold; no teacher is required at deployment. In a mechanism-based multi-step simulation, TACIT-SWITCH improves success by 7.4-11.1 percentage points over task-level, step-level, and fixed-prefix routing baselines at comparable cost. Within that controlled simulation, ablations show that task features and cumulative trajectory risk provide complementary information. With operating points selected on development data, TACIT-SWITCH achieves the highest held-out success among learned policies on both ALFWorld (48.5% with 4B Cheap; 45.5% with 9B Cheap) and DABench (73.1%).

1 INTRODUCTION

Interactive-agent routing is a recoverability-aware stopping-time problem because useful failure evidence emerges during execution. TACIT-SWITCH learns an adaptive permanent handoff from censored teacher supervision and improves success–cost trade-offs in controlled and interactive evaluations.

  • Motivation: Interactive routing must decide whether and when to hand control to a stronger model as trajectory evidence accumulates.Immediate handoff sacrifices the cheaper model’s cost advantage, while late handoff may leave little recovery opportunity.
  • Routing strategies: Existing methods route before execution, after a fixed prefix, or through reversible step-local interventions, whereas TACIT-SWITCH uses an adaptive one-way transfer.The compared strategies include task routing, SWE-Router, ReDAct, and TACIT-SWITCH.
  • Supervision: TACIT supervision uses complete Cheap–Strong rollouts and a coarse offline teacher handoff window instead of counterfactual labels at every checkpoint.Paired Strong failures provide no evidence that switching would rescue the task, while Cheap successes have no observed rescue time.
  • Method: TACIT-SWITCH models Strong-rollout success separately from the conditional cumulative-risk handoff threshold using a mixture-cure threshold model.Teacher windows are treated as interval-censored observations, and deployment switching uses a learned threshold without an online teacher.
  • Evaluation: The evaluation combines model-recovery, mechanism-based decision-utility, and interactive-agent experiments with ablations of task features and online risk.Interactive experiments compare held-out success and cost against task-, step-, and fixed-prefix routing.

2 PROBLEM FORMULATION AND TACIT SUPERVISION

The paper formulates permanent handoff as success maximization under an inference budget, using cumulative trajectory risk to time a one-way transfer. TACIT supervision derives threshold evidence from paired rollout outcomes and coarse teacher intervals.

  • 2.1 PERMANENT HANDOFF ON A CUMULATIVE-RISK SCALE: A permanent-handoff policy starts each episode with Cheap and may transfer once to Strong for all remaining actions.Episodes contain at most H interaction steps, and the policy records task success and deployment inference cost.
  • 2.1 PERMANENT HANDOFF ON A CUMULATIVE-RISK SCALE: The optimization target is task success under an inference budget, with cost defined by the deployment setting.TACIT-SWITCH estimates a handoff score first and selects its application-specific threshold α on development data.
  • 2.1 PERMANENT HANDOFF ON A CUMULATIVE-RISK SCALE: Observed trajectory prefixes are mapped to nonnegative diagnostic features, including repeated or invalid actions, uncertainty, and stalled progress.Nonnegative diagnostic weights combine these signals into a cumulative-risk score.
  • 2.1 PERMANENT HANDOFF ON A CUMULATIVE-RISK SCALE: Because diagnostic features and weights are nonnegative, cumulative risk is nondecreasing and induces a first crossing of the latent handoff threshold κi.The crossing is a decision-model construct and does not alter model abilities or environment dynamics.
  • 2.2 CENSORED TACIT SUPERVISION: Training pairs complete Cheap and Strong rollouts, using Ai and Bi to record their initial-state task outcomes.Bi denotes the realized outcome of one paired Strong rollout, not an intrinsic or state-independent capability label.
  • 2.2 CENSORED TACIT SUPERVISION: Table 1 maps paired rollout outcomes to handoff evidence, with only AB = 01 receiving a teacher interval.This mapping distinguishes outcome strata before threshold-location supervision is applied.
  • 2.2 CENSORED TACIT SUPERVISION: For AB = 01, the teacher marks a checkpoint or range where handoff appears useful and recovery remains feasible.A single checkpoint gives a one-step interval rather than an exact real-valued threshold.
  • 2.2 CENSORED TACIT SUPERVISION: Teacher ranges impose interval-censored threshold constraints, while AB = 11 is right-censored because Cheap completes without an observed need for rescue.The paired-Strong-success membership is known from Bi = 1 even when threshold location is unobserved.

3 TACIT-SWITCH: MODEL, ESTIMATION, AND DEPLOYMENT

TACIT-SWITCH models permanent handoffs by combining Strong-rollout success incidence with a cumulative-risk threshold, using interval-censored teacher windows for estimation. Deployment applies the fitted score at checkpoints and selects a fixed threshold on development data to balance success and cost.

  • 3.1 INCIDENCE–THRESHOLD MODEL: Mixture-cure models separate whether the paired Strong rollout succeeds from where the handoff threshold lies conditional on success.The threshold is modeled with a log-normal AFT component, while interval censoring represents coarse teacher windows.
  • 3.2 CENSORED LIKELIHOOD AND ESTIMATION: Teacher windows become cumulative-risk bounds, while right-censored episodes contribute the cumulative risk reached before termination.For a teacher window [a_i, b_i], the lower and upper bounds are R_i,w(a_i − 1) and R_i,w(b_i).
  • 3.2 CENSORED LIKELIHOOD AND ESTIMATION: The working likelihood accounts for trajectory-dependent teacher annotation and does not claim to be a generative model of trajectories or annotations.The coarsening mechanism may be non-ignorable because the teacher inspects the realized trajectory before constructing the interval.
  • 3.2 CENSORED LIKELIHOOD AND ESTIMATION: Under bounded interval corruption and regularity assumptions, the corrupted objective retains a unique stationary point near the clean population minimizer.The result holds with probability at least 1 − δ when the stated error term is smaller than µr/2.
  • 3.2 CENSORED LIKELIHOOD AND ESTIMATION: The handoff score combines Strong-success probability with the conditional probability that cumulative risk has crossed the learned threshold.The score is q_θ(x, u^(1:t)) = π_γ(x)F_β,s(R_w(t) | x).
  • 3.2 CENSORED LIKELIHOOD AND ESTIMATION: At deployment, the teacher is absent: the policy permanently switches at the first checkpoint whose score reaches α, or never switches if none does.Because cumulative risk is nondecreasing, lowering α cannot delay the crossing; α is selected on development data under the prescribed cost budget.

4 CONTROLLED STATISTICAL EXPERIMENTS

The controlled experiments assess statistical recovery, decision utility, information contributions, and sensitivity under separate simulation designs. TACIT-SWITCH recovers predictions as sample size grows and improves fixed-cost policy success, with both task features and online risk contributing.

  • Experimental design: Two complementary simulations test parameter and prediction recovery under correct specification, then switching-policy utility in a separate multi-step process.The second simulation is not assumed to follow the AB–AFT model and evaluates decision utility rather than parameter recovery.
  • Model recovery: Held-out q(t | x) RMSE decreases from 0.0203 ± 0.0006 at n = 500 to 0.0052 ± 0.0001 at n = 8,000.All 1,500 fits converge, and componentwise parameter RMSE approximately halves whenever sample size quadruples.
  • Robustness: Up to 20% corruption, adjacent-step teacher-interval shifts change the estimator little, whereas flipping paired Strong outcomes affects it more.These corruption tests appear in Appendix C.
  • Mechanism-based policy comparison: 73.52% success at nearly identical realized costs of 15.91–15.95 exceeds Task Router, Step Deferral, and the SWE-Router analogue.The corresponding baseline success rates are 62.40%, 62.98%, and 66.13%, respectively.
  • Information ablation: At nearly identical realized cost, Full reaches 73.51 ± 0.09% success versus 66.62 ± 0.10% for x-only and 71.35 ± 0.09% for risk-only.Online risk provides the larger marginal gain, while task features add information beyond online risk.
  • Scenario sensitivity: TACIT-SWITCH success increases from 71.56% to 74.87% across task-feature settings and from 67.38% to 77.96% across online-risk settings.Actor behavior and endpoint difficulty are held fixed while each information source varies separately.

5 INTERACTIVE-AGENT EXPERIMENTS

The experiments evaluate TACIT-SWITCH against task-, step-, and fixed-prefix routing under frozen development-selected operating points. TACIT-SWITCH achieves the highest held-out success among learned policies across both ALFWorld model pairs and DABench, with DABench also showing the lowest average cost among learned baselines.

  • Experimental setup: Paired Cheap–Strong rollouts define success outcomes, while AB = 01 records receive teacher intervals and AB = 11 records are right-censored.The paired Strong outcome is realized from the initial state, not an intrinsic capability label.
  • Experimental setup: All policy settings are selected once on development data under a fixed cost cap and frozen before test.The cost proxy uses 4B-equivalent units based on model parameter counts.
  • ALFWorld: TACIT-SWITCH has the highest held-out success among learned policies for both ALFWorld model pairs.With 9B Cheap, it exceeds Task Router and SWE-Router at lower cost; with 4B Cheap, it approaches pure-27B success at lower cost.
  • ALFWorld: A larger-scale ALFWorld check again places TACIT-SWITCH first among learned policies for both model pairs.The check uses 1,000 training, 300 development, and 274 test episodes but reuses the underlying task pool.
  • DABench: TACIT-SWITCH outperforms all three learned DABench routing baselines while having the lowest average cost among them.Its paired gain over SWE-Router is suggestive rather than conclusive because the bootstrap interval includes zero.

6 CONCLUSION AND LIMITATIONS

The paper concludes that TACIT-SWITCH improves success–cost trade-offs by learning permanent handoffs from paired outcomes, teacher intervals, task features, and cumulative trajectory risk. Its scope is limited to one permanent handoff, application-specific cost functions, and dependence on paired offline rollouts and teacher annotations.

  • Conclusion: TACIT-SWITCH combines task features with cumulative trajectory risk to learn permanent handoffs without teacher annotation at deployment.The paper reports better held-out success–cost trade-offs than task-, step-, and fixed-prefix routing in mechanism-based and interactive-agent experiments.
  • Conclusion: Controlled experiments indicate that task information and trajectory information are complementary.The model-based study also supports recovery under correct specification.
  • Limitations: TACIT-SWITCH models one permanent handoff and excludes reversible or repeated switching.This constrains the policy scope to a single irreversible transfer from Cheap to Strong.
  • Limitations: The operating point depends on an application-specified cost function, using parameter-weighted call count here as a deployment-cost proxy.Systematic teacher-annotation bias or distribution shift can also affect the learned policy.
  • Reproducibility: The accompanying artifact includes configurations, seeds, task manifests, fitted parameters, frozen scores, results, and regeneration scripts.Appendices document protocols, model fitting, cost accounting, and development-only selection.

A FINITE-SAMPLE THEORY

The finite-sample theory analyzes fitting under clean TACIT records and possible teacher-interval corruption using a locally identifiable, regularized likelihood. Its assumptions require smoothness, boundedness, positive scale, and likelihood masses bounded away from zero near the target.

  • Setup: A clean TACIT record contains task features, risk diagnostics, the paired-Strong outcome, and interval or right-censoring information.The fitted record may differ through possible teacher-interval corruption while retaining the other fields.
  • Objective: The objective combines the negative log-likelihood contribution from Equation (7) with a deterministic regularizer under parameters (w, γ, β, s).The parameterization is locally identifiable.
  • Parameterization: The simplex uses d − 1 independent coordinates instead of all d softmax logits to remove common-shift nonidentifiability.A smooth local coordinate may represent the positive scale.
  • Assumptions: The clean records are i.i.d., recordwise losses are twice continuously differentiable, and differentiation and expectation may be interchanged through order two.These conditions hold on a closed ball contained in the interior of the local coordinate chart.
  • Assumptions: The target is an interior stationary point of the population objective, which is twice continuously differentiable.The local theory additionally requires bounded covariates and diagnostics, scale bounded away from zero, and likelihood masses bounded away from zero.

A.2 PROOF OF THEOREM 1

The proof establishes local finite-sample stability by controlling score concentration, empirical-Hessian behavior, and teacher-interval corruption. Strong convexity excludes boundary minimizers, yields a unique interior minimizer, and supports parameter and decision-score bounds.

  • Score concentration: A 1/2-net of the Euclidean unit sphere reduces uniform directional score control to finitely many sub-Gaussian bounds.The net size is bounded by 5^p, and a union bound produces the stated high-probability event.
  • Corruption control: The deterministic regularizer cancels when the centered empirical objective is compared with its population counterpart.The proof uses stationarity of θ† and bounds the effects of replaced teacher intervals through recordwise gradients.
  • Hessian control: The empirical Hessian remains positive definite throughout the local ball after combining clean concentration with Hessian bounds for corrupted records.The resulting lower curvature is (3µ/4)I_p.
  • Local minimizer: Boundary minimizers are ruled out, so the minimizer is interior and stationary; strong convexity then makes it unique on the ball.This conclusion follows from the boundary argument and Equation (17).
  • Stability bound: Strong monotonicity of the gradient yields the parameter-error bound, which transfers through uniform Lipschitzness to every evaluation-task decision score.The argument applies to arbitrary tasks and decision steps within the local ball.

B.1 CONTROLLED STATISTICAL PROTOCOLS

The controlled protocol evaluates routing policies on coupled multi-step tasks, with development-based selection and held-out replication. Diagnostics and sensitivity analyses test the value of cumulative risk and task features.

  • The simulator gives each task ten opportunities, requires at least eight correct actions, and models on-track, recoverable, and off-track states.Cheap and Strong use fixed transition and action-success probabilities, with Strong more likely to recover degraded states.
  • 100 replicates use 8,000 training, 4,000 development, and 20,000 test tasks, with Cheap and Strong step costs of 1 and 3.All methods share a development cost cap of 16.
  • TACIT-SWITCH combines task features with cumulative Cheap risk and permanently hands off to Strong at the first score crossing.Task Router, Step Deferral, and a fixed-prefix SWE-Router analogue provide the comparison policies.
  • AUC .774 versus .605 and .691 shows cumulative risk predicts terminal Pure-Cheap failure better than task features alone or latest risk.The diagnostic comparison uses the same controlled task setting.
  • 80.1% of AB = 01 tasks contain a candidate handoff checkpoint, and immediate handoff after the marked Cheap step succeeds in 85.6% of those cases.These are held-out replicate averages for the controlled simulation.
  • Across sensitivity settings, TACIT-SWITCH has higher held-out success than the best development-selected baseline at comparable cost.The gains increase when online-risk information becomes more informative.

B.2 INTERACTIVE-AGENT PROTOCOLS

The interactive protocols compare routing policies under benchmark-specific execution semantics, using common cost accounting and development-only operating-point selection. Teacher annotation is offline and excluded from deployment cost.

  • Costs are computed as Cost_i = c_m n_m,i + 6.75n_27,i + 1.75h_i, with c_4B = 1 and c_9B = 2.25.The proxy uses common 4B-equivalent units and realized model-call counts.
  • ALFWorld: ALFWorld selects TACIT-SWITCH thresholds on SEEN40 and freezes them before UNSEEN134 evaluation under a 90% pure-27B cost cap.Development selects α = 0.2 for 4B and α = 0.3 for 9B.
  • DABench: DABench uses Train140, Dev50, and Test67, explicitly noting that this custom split is not an official benchmark partition.Test67 is held out from fitting and development selection.
  • TACIT-SWITCH permanently transfers control at its first cumulative-risk crossing, whereas Step Deferral delegates only the current action and then returns to Cheap.The fixed-prefix baseline predicts Pure-Cheap success from task features and K Cheap risk observations before continuing or restarting.
  • The paired bootstrap 95% success-difference intervals versus SWE-Router are [17.2, 35.1] points for ALFWorld 4B–27B, [2.2, 17.9] for ALFWorld 9B–27B, and [−3.0, 14.9] for DABench 9B–27B.Intervals use 10,000 with-replacement task-level resamples.
  • ALFWorld makes the handoff decision after observing the Cheap action and result, while DABench evaluates an unexecuted Cheap proposal before execution.DABench aligns a teacher mark after step t with the proposal at step t + 1.

B.3 LARGER-SAMPLE ALFWORLD SCALE-UP EVALUATION

The larger-sample ALFWorld evaluation freezes policies on Dev300 and evaluates them on Test274 using realized model-call costs. TACIT-SWITCH attains the highest learned-policy success point estimate for both Cheap-model sizes.

  • Test274 combines 140 valid-seen and 134 valid-unseen tasks, while fit and development sets contain 1,000 and 300 episodes from disjoint parent-task groups.The study is presented as a scale check rather than a separate benchmark.
  • All candidates are selected on Dev300 under a 180.77 4B-equivalent-unit cost cap and frozen before Test274 evaluation.Test costs use actual model-call counts under Equation (20).
  • TACIT-SWITCH has the highest success point estimate among learned policies for both the 4B–27B and 9B–27B pairs.It is also less costly than Task Router and SWE-Router.
  • 11.7 percentage points for 4B–27B and 17.9 percentage points for 9B–27B are TACIT-SWITCH’s success gains over Step Deferral.The corresponding additional costs are 8.8 and 6.1 units, respectively.
  • The ALFWorld TACIT-SWITCH model combines standardized task features with five directly used step-level risk diagnostics.The task features cover requested state, task form, target category, receptacle type, and visible-entity count.

C ROBUSTNESS TO NOISY TEACHER SUPERVISION

The robustness study evaluates teacher-interval and paired-Strong-outcome corruption under matched simulation conditions. Strong-outcome corruption causes substantially larger threshold-estimation error than shifting teacher intervals by one step.

  • 100 paired replicates at n = 4,000 evaluate teacher-interval noise, Strong-outcome noise, and matched joint noise without corrupting test measurements.Teacher-interval noise shifts one-step intervals to adjacent steps; outcome noise flips B_i for a fraction of tasks.
  • At 20% corruption, q(t | x) RMSE is 0.0077 ± 0.0005 for teacher-interval noise, 0.0208 ± 0.0007 for Strong-outcome noise, and 0.0209 ± 0.0007 for matched joint noise.The clean RMSE is 0.0075 ± 0.0005.
  • Strong-outcome corruption has a larger effect than shifting the teacher interval by one step under the matched-noise design.Figure 6 plots estimation or held-out prediction errors against direct corruption rate.
  • The inference algorithm updates cumulative risk at each Cheap checkpoint and permanently switches to Strong when q reaches the development-selected threshold α.After switching, it continues with Strong; benchmark-specific checkpoint semantics determine when evidence is obtained.
Loading 2608.27911v1…