PaperlayerView exact revision on arXiv

Source-linked AI summary

TaoLive Digital Avatar Agent Technical Report: Training Agents to Evolve with Their Harness

TaoLive AIGC LLM Team, Yuhan Sun, Wenhao Lin, Yongdong Luo, Yibo Hu, Meiguang Jin, Junfeng Ma, Weihang Pan, Jiaxin Zhao, Zulong Chen

arXiv:2608.15763v1cs.CL

TL;DR

Evolving live-commerce Harnesses create a moving execution environment that fixed-Harness training may not accommodate. Harness-Aware Training incorporates Harness variation into training, producing a compact agent that remains effective under evaluated changes while retaining general instruction following and latency feasibility.

  • Problem

    Evolving Harness states can make fixed-Harness-trained models stale, creating a training problem for agents whose execution environments change without weight updates.

  • Method

    Harness-Aware Training places Harness-state variation in the training distribution through HSA-SFT, General OPD, and HSA-RL.

  • Results

    94.8 Live-Stream QA AVG was achieved without the general-set drop observed after Fixed-Harness SFT, alongside 8.114 s P95 latency in controlled replay.

  • Takeaways & Limitations

    HAT produces a latency-feasible compact agent that preserves adaptability to evolving production Harnesses within the evaluated scope.

  • Takeaways & Limitations

    The selected HSA-RL checkpoint is an engineering operating-point choice, so its single-run contrasts are not method-level effects.

Abstract

from arXiv · show

AI-powered digital-avatar streamers in live e-commerce must answer product questions, engage viewers, and execute changing business strategies in real time. This requires low latency, factual and effective replies, and rapid adaptation to updated campaign, compliance, and style requirements. We develop an evolvable Harness that decouples Skills, Hooks, system prompts, and tools from model weights, allowing runtime behavior to change without retraining. However, Harness evolution creates a moving execution environment: compact models fine-tuned on one configuration may memorize names, schemas, and prompt templates rather than follow the Harness currently provided, while stronger zero-shot models are too slow for real-time use. We address this tension with Harness-Aware Training (HAT), which makes Harness states part of the training distribution. HAT applies task-preserving Harness-State Augmentation (HSA) to Skills, tool schemas, prompt structures, and interaction constraints, and comprises three stages: HSA-based supervised fine-tuning, general on-policy distillation to recover general capabilities, and HSA-based agentic reinforcement learning in a production-informed live-room simulator. Across four evaluation sets with more than 4,500 cases, our compact 35B model scores 94.8 on real-world Live-Stream QA, versus 80.3 for the base model and 93.0 for the strongest evaluated general LLM, while scoring 94.6 on Harness-Variant QA and retaining 83.5 on IFEval. By contrast, fixed-Harness SFT reduces IFEval by 7.7 points. In a controlled complete-agent replay on one NVIDIA H20 GPU with MTP enabled, the system achieves 3.407 s P50 and 8.114 s P95 latency. These results show that HAT produces a latency-feasible compact agent that remains effective under evaluated Harness changes without sacrificing general instruction following.

1 Introduction

Live-streaming e-commerce requires low-latency digital-avatar agents that answer product questions, engage viewers, and execute changing strategies. The paper introduces an evolvable Harness and Harness-Aware Training to let compact models adapt to changing execution environments while retaining general capabilities.

  • Live-streaming e-commerce demands real-time viewer interaction, product-question answering, and marketing-strategy execution under strict latency constraints.
  • The evolvable Harness decouples Skills, Hooks, system-prompt assembly, and commerce-tool definitions, enabling strategy and constraint updates without retraining.Harness Evolution keeps policy-model weights fixed while the execution environment changes.
  • Harness Evolution creates a moving execution environment in which rewritten Skills, added Hooks, reassembled prompts, and renamed tool schemas can exceed fixed-Harness training.
  • Harness-Aware Training makes Harness state part of the training distribution through HSA-SFT, General OPD, and HSA-RL, allowing compact models to track Harness changes while retaining general capabilities.
  • 94.8 average score: the selected 35B checkpoint surpasses the untuned base model’s 80.3 on real-world Live-Stream QA while avoiding Fixed-Harness SFT’s significant IFEval loss.
  • 3.407s P50 and 8.114s P95: the complete agent meets strict latency requirements on one NVIDIA-H20 GPU with MTP.The evaluation uses a Harness-based judge calibrated against human annotations.

2 Digital-Avatar Harness Agent Architecture

The digital-avatar Harness Agent separates fixed policy weights from an evolving runtime of skills, prompts, hooks, and tools, enabling business changes without retraining while creating a stale-model risk. Harness Evolution addresses this through human-confirmed runtime edits, but its gains are non-monotonic, motivating Harness-Aware Training for changing Harness states.

  • Architecture and evolution: The architecture separates a slowly updated policy model from a rapidly evolving Harness state, allowing business behavior changes without retraining but requiring effectiveness across evolving environments.A model that memorizes one Harness state becomes stale when the system evolves.
  • Runtime architecture: The runtime combines dynamically loaded Skills, assembled system prompts, validating Hooks, and external tool orchestration for live-commerce interaction.Inputs include viewer messages, product data, link IDs, and living-room status; outputs include spoken or private replies and structured tool-call trajectories.
  • Harness Evolution: Harness Evolution changes skills, prompts, hooks, or tools while holding model weights fixed, with AI proposing edits and a developer confirming, evaluating, and promoting or stopping them.The process is explicitly human-in-loop rather than fully autonomous.
  • Harness Evolution: 92.55 Accuracy and 92.75 Effectiveness are reached by Evolution 2, which is selected as the engineering early-stop checkpoint after later long-tail regressions.On a 482-case calibration set, modular harnessing raises Effectiveness from 84.58 to 87.16, while Evolution 1 raises Accuracy from 82.40 to 92.13 but lowers Effectiveness to 84.16.
  • Runtime execution: The live-commerce runtime supports single-comment and multi-comment modes, configurable agent rounds, tool calls, Skill loading, lifecycle checks, retries, and final reply dispatch.Hooks cover input validity, tool parameters, Skill requirements, response format, factuality risks, and retry control.

3 Harness-Aware Training (HAT)

Harness-Aware Training treats the evolving Harness as part of the training distribution, using augmentation to improve adaptation across Harness changes while retaining general instruction-following capabilities. It combines HSA-based supervised fine-tuning and general on-policy distillation with agentic reinforcement learning in a production-informed simulator.

  • Harness-Aware Training: Harness state h includes active Skills, the tool registry, the dynamic system prompt, and Hooks governing validation, retries, and format enforcement.The policy is modeled as πθ(a | x, h), where h is the full inference-time configuration.
  • Training Objective: HAT targets both effectiveness under deployment Harness changes and retention of general-purpose capabilities after domain-specific fine-tuning.Deployment Harness states are drawn from a change envelope broader than any single training snapshot.
  • Harness-State Augmentation: Harness-State Augmentation perturbs Skills, skill content, tool schemas, prompt structures, and interaction constraints to reduce reliance on fixed names and templates.The method expands, masks, renames, rewrites, paraphrases, and reorders Harness elements while preserving task behavior.
  • Agentic RL Training: Agentic RL trains the model through bounded multi-round interactions with Skills, tools, Hooks, and controlled failures in a production-informed live-streaming simulator.GRPO rollouts are rewarded for accuracy, effectiveness, tool rationality, and Skill selection, with CoT thresholds set to 100 and 200 tokens.

4 Evaluation Design

The evaluation covers real live-stream quality, Harness variation, robustness, general instruction following, and deployment performance. It combines rubric-based offline metrics with complete-agent replay under a fixed production-informed configuration.

  • Offline evaluation: Live-Stream QA measures industrial reply quality, while Harness-Variant QA tests generalization and robustness to changing Harness contexts using real live-room interactions.Both sets are built from real live-room interactions and target the main industry scenarios in the application domain.
  • Offline evaluation: Accuracy penalizes factual errors and unsupported claims, while Effectiveness scores whether responses address viewer intent and provide useful information.Accuracy is binary; Effectiveness uses scores of 0, 0.5, or 1, with both evaluated by a Harness-based Agent-as-a-Judge.
  • Offline evaluation: Tool Robustness uses ADR, RC, and TC to assess query coverage, use of required tool-returned information, and expected tool invocation.These are task-specific rubric metrics.
  • Offline evaluation: Prompt Robustness uses ADR, LLM, and TCO to evaluate query coverage, instruction-constraint satisfaction, and required tool-call ordering.These metrics are task-specific rubric measures for prompt robustness.
  • Deployment replay: Deployment replay reports wall-clock latency P50 and P95, TTFT, decoding throughput, execution success, and 15-second attainment.Wall-clock latency spans input receipt to the final Agent answer; execution success denotes request completion rather than answer quality.
  • Deployment replay: 100 requests are scored after 10 warm-up cases, with complete-Agent execution using one Harness commit, real MCP services, at most four loops, and a 5-second tool timeout.Generation uses temperature 1.0 and top-p 0.95; Qwen3.6-35B-A3B and the task-trained checkpoint run in BF16 on one H20 with TP=PP=1.

5 Experiments

Experiments show that HAT improves live-stream quality and Harness robustness while preserving general instruction following, and that its staged training and augmentation choices affect the final operating point. Deployment replay and held-out Harness-edit evaluations further assess latency, adaptation, and comparative behavior.

  • Main results: 94.8 on T1: HAT exceeds the strongest Top-model score of 93.0 on the same business dataset while avoiding Fixed-Harness SFT’s robustness and generalization drops.Fixed-Harness SFT lowers Prompt Robustness by 4.6 and IFEval by 7.7/5.3.
  • Training ablations: HSA-SFT plus General OPD provides a better starting point for Agentic RL than Fixed-Harness SFT plus General OPD, while HSA-RL improves over Agentic RL in the original Harness environment.These findings associate stronger Harness generalization with a better RL operating point and expose the RL policy to changing Harness contexts.
  • Training ablations: +9.4 IFE-P for HSA-SFT versus +1.0 for HSA-RL, indicating that the main isolated HSA gain comes from the SFT stage.The differences are descriptive and exclude retraining variance.
  • Deployment replay: 8.114 s P95 latency at concurrency 1 and 9.047 s at concurrency 2 satisfy the 15-second bound with the selected MTP configuration.On the same checkpoint and H20, MTP increases decoding throughput from 160.30 to 271.40 tokens/s at concurrency 1 and from 129.22 to 196.15 at concurrency 2.
  • Harness edits and preference evaluation: 97.2% non-tie win rate: Harness is preferred in 35 of 36 clear-preference examples, with p < 0.001 against equal preference.The sole ReAct-preferred example is treated as an isolated fallback-policy failure.
  • Harness edits and preference evaluation: 80.0% of Harness-preferred cases are attributed to input understanding, output reliability, and scenario-appropriate behavior.The categories include separating live-room events from viewer questions, avoiding unsupported claims, handling low-information comments, and using tool evidence appropriately.

6 Deployment and Production Lessons

Deployment separates independently versioned policy and Harness paths, with release gates, rollback controls, and defense-in-depth safeguards supporting weekly evolution. Production serving meets the stated latency target in controlled replay, while preliminary online A/B results show uplifts that remain ongoing and untested for significance.

  • Versioning and release: Policy checkpoints and Harness components evolve independently, while release manifests bind versions, schemas, fallback thresholds, and evaluation sets.Every proposed change runs the business regression suite and general-capability gate before traffic promotion.
  • Versioning and release: 15-second latency, business regression, and general-capability gates govern promotion, with independent switches enabling attribution and reversal of either layer.The production policy must retain open-ended instruction following and accommodate weekly changes to Skills, tools, and prompts.
  • Runtime safeguards: 3 s first-call and 2 s later-call timeouts, a 10 s trajectory budget, bounded retries, and four-round termination provide defense in depth before compact fallback or conservative recovery.Hooks reject malformed calls, type service errors, and enforce skill-loading and output-format constraints.
  • Serving configuration: 1–2 concurrent requests are the decision-relevant range: MTP delivers 1.69×/1.52× higher decoding throughput while maintaining 8.114/9.047 s P95 and 100% 15-second attainment.At concurrency 4–8, the decoding benefit falls to 1.27×/1.19× and MTP no longer improves P95, motivating additional replicas.
  • Online production evaluation: 5.54% confirmed-receipt GMV per bucketed user and 0.80% completed orders per bucketed user were observed versus ReAct in the preliminary seven-day production snapshot.The online A/B test remains ongoing; estimates are unadjusted, with no confidence intervals or significance tests available.

7 Related Work

Prior work spans agent architectures, robustness to prompt and tool-interface changes, agent post-training, and adaptation under distribution shift. These studies motivate evaluating agents across changing Harness states and combining supervised, reinforcement, and deployment-time adaptation methods.

  • Agent architectures and Harness Evolution: Agent architectures combine reasoning, tool invocation, self-correction, planning, and prompt compilation, establishing foundations for evolvable agent Harnesses.ReAct interleaves reasoning and action; Toolformer teaches tool invocation; Reflexion adds within-episode self-correction; TPTU-v2 improves planning and tool use; DSPy compiles declarative modules into optimized prompts.
  • Prompt, schema, and agent-evaluation robustness: Prompt-template equivalence and natural-language tool-description edits can alter model behavior and tool selection, while SMART emphasizes learning when not to invoke tools.These findings motivate varying prompt and tool interfaces instead of evaluating only one frozen surface form.
  • Agent post-training: Agent post-training uses supervised trajectories, rollout filtering, and environment-interaction reinforcement learning, including structured rewards for tool selection and application.The cited approaches cover SFT, rejection sampling, self-taught variants, RL, ToolRL, and GRPO.
  • Training under distribution shift: Distribution-shift methods include domain adaptation, continual learning, test-time adaptation, and agent-specific deployment-time learning of environment formats and dynamics.Deployment-time adaptation is described as complementary when changing weights or probing the environment online is constrained.

8 Conclusion … B.1 Evolution Loop

The paper presents Harness-Aware Training as a methodology for training compact live-streaming digital-avatar agents to remain effective as their runtime Harness evolves. It combines Harness-state variation in training with a versioned evolution loop and reports strong quality and latency results.

  • 8 Conclusion: HAT treats Harness-state variation as a training-distribution design problem within an explicitly defined change envelope.The methodology addresses versioned Harness change and policy stability in live-streaming digital-avatar agents.
  • 8 Conclusion: HAT combines Harness-State Augmentation with HSA-SFT, General OPD, and HSA-RL in a three-stage training pipeline.Harness Evolution versions runtime changes separately from policy updates through modular Skills, Hooks, prompts, and tools.
  • 8 Conclusion: 94.8 Live-Stream QA AVG is achieved by the HAT-trained compact model without the general-set drop observed after Fixed-Harness SFT.The reported result supports quality under the evaluated training setup while avoiding the stated general-set degradation.
  • 8 Conclusion: 8.114 s P95 is reached at concurrency 1 and 9.047 s at concurrency 2 on one H20 with MTP enabled.The controlled replay also reports 100% 15-second attainment.
  • 8 Conclusion: 1.69×/1.52× higher client-observed decoding throughput is reported with MTP than without MTP on the same checkpoint.The comparison is reported for the controlled replay setup using one H20 with MTP enabled.
  • A Runtime Interface and Skill Inventories: Runtime interfaces and Skills are documented in working inventories covering built-in and MCP-discovered interfaces plus routing responsibilities.These inventories are presented as Table 12 and Table 13.
  • B.1 Evolution Loop: Each Harness Evolution stage holds the model fixed while changing only editable runtime modules, using a five-step evolution loop.The loop is defined in the Harness Evolution protocol and diagnostics section.
  • B.1 Evolution Loop: The loop proceeds through AI diagnosis, human confirmation, AI-assisted editing with versioned snapshots, and human-triggered evaluation.The supplied passage lists these operational stages and their responsibilities.

B.2 What Changed across Evolution Stages

Evolution produced its largest gains in specific conversation categories, but later long-tail rule changes introduced cross-component interactions and metric regressions. The report therefore supports configuration diversity in training and regression-aware early stopping during production evolution.

  • Evolution gains: Accuracy gains peaked at +60.7 points for negative-feedback chat, while Effectiveness gains peaked at +27.3 for emotion expression.Other major gains included +53.3 Accuracy for after-sales and returns and +18.8 Effectiveness for membership or gift benefits.
  • Evolution gains: After Evolution 2, 36 Accuracy failures remained across 16 categories, alongside 35 Effectiveness failures without a dominant systematic cluster.The report treats these category values as engineering diagnostics rather than pre-specified inferential analyses.
  • Fixed-policy Harness Evolution: 92.55 Accuracy and 92.75 Effectiveness were selected for Evolution 3, restoring Effectiveness while retaining Accuracy.Evolution 3 added attribution, factuality, parameter-completeness, tool-use, and system-message rules, and expanded refusal exclusions.
  • Fixed-policy Harness Evolution: 91.51 Accuracy and 90.89 Effectiveness followed Evolution 4, as long-tail rules interacted and regressed both metrics.Evolution 4 relaxed length, tool-trigger, parameter, transaction-intent, and attribution restrictions.

C HSA-RL Training Details

HSA-RL combines Harness-State Augmentation with multidimensional policy optimization and auxiliary constraints. Training diagnostics show shorter reasoning traces, while partial rollback does not recover Effectiveness.

  • Rollback Result: 91.51 to 89.96 shows that partial rollback does not recover Effectiveness.The passage reports this comparison directly but does not specify the evaluation conditions.
  • HSA-RL Procedure: HSA-RL combines HSA with GDPO multidimensional advantage estimation, auxiliary CoT-length optimization, GSPO sequence-level importance sampling, and zero-advantage sample filtering.The procedure uses tool and reply dimensions, with λ = 0.1 for post-normalization CoT weighting and ε as the PPO clip range.
  • Training Diagnostics: 290.6 to 171.2 extracted tokens for tool-call CoT and 93.6 to 37.0 for final-reply CoT across the first-to-last 100-step windows.The strongest reduction occurs early, followed by a noisy plateau, indicating shorter reasoning traces in this run.

D Judge as a Harness Evolution Case · E Evaluation and Reproducibility Details · E.1 Judge Roles and Independence

The paper evolves its final evaluation judge against human labels and separates judging roles to preserve calibration and independence. It also specifies reproducibility details for HSA-RL training and evaluation.

  • D Judge as a Harness Evolution Case: 90.46% Accuracy agreement was reached at Evolution 6, up from 81.54% at Evolution 1, across 482 human-labeled interactions.Effectiveness agreement rose from a 63.70% uncalibrated baseline to an 83.40% peak.
  • D Judge as a Harness Evolution Case: The Final Evaluation Judge is a Harness-based panel combining DeepSeek-V4-pro, GLM-5.2, and Qwen3.7-max by majority vote.Its Skills, prompts, and tool-verification procedures evolve through disagreement analysis against human labels.
  • E.1 Judge Roles and Independence: Five roles remain distinct: SFT teacher, rejection-sampling Judge, Reward Judge, Final Evaluation Judge, and human calibration labels.DeepSeek-V4-pro generates supervised candidates; DeepSeek-V4-flash scores HSA-RL trajectories; and the final panel produces offline majority-vote scores.
  • E.1 Judge Roles and Independence: Zero model overlap exists between the Reward Judge and Final Evaluation Judge, because DeepSeek-V4-flash is excluded from the final three-model panel.Although DeepSeek-V4-pro is both SFT teacher and final voter, a two-of-three majority prevents its vote from deciding an item alone.
  • E Evaluation and Reproducibility Details: CoT diagnostics distinguish tool-call and final-reply lengths and apply no-penalty, linear, and saturated regions to the logged score.Figure 6 is explicitly a training diagnostic, not a causal ablation of latency or quality.
  • E.1 Judge Roles and Independence: All three final voters can call evidence tools for product-fact checks, and human alignment is measured before model comparison.Figure 7 evaluates the scoring instrument on a 482-example human-labeled calibration cohort rather than evaluating the policy.
  • E Evaluation and Reproducibility Details: HSA-RL samples query–Harness-state pairs, rolls out grouped trajectories, and separates tool and final-reply segments for scoring.The algorithm tracks tool and reply losses, extracts CoT lengths, computes grouped advantages, and updates the policy by minimizing their mean.

E.2 Offline Scoring and Implementation … F Deployment Inference and Operating Details

The paper defines offline scoring and robustness evaluation procedures, including HSA-RL implementation details, synthetic T3 subsets, and an independent paired-bootstrap stability check. T3 evaluates tool-selection and instruction-following robustness using rubric-based metrics derived from real live-streaming e-commerce scenarios.

  • E.2 Offline Scoring and Implementation: Accuracy is binary, Effectiveness uses 0, 0.5, and 1, and AVG is the per-sample mean computed before rounding.IFEval uses official prompt- and instruction-level evaluators.
  • E.2 Offline Scoring and Implementation: HSA-RL uses GRPO with Accuracy, Effectiveness, Tool Rationality, and Skill Selection rewards under five Harness-state perturbation types.The perturbations cover Skill, tool, prompt, Hook, and message-processing changes; the policy is Qwen3.6-35B-A3B.
  • E.3 T3 Robustness Subset Construction: T3 contains 1,532 Tool Robustness items and 491 Prompt Robustness items generated from live-streaming e-commerce scenario seeds.Each seed records product context, viewer intent, possible tool needs, and reply constraints for a scenario family.
  • E.3 T3 Robustness Subset Construction: The shared T3 pipeline expands business-data seeds into viewer comments, contexts, and expected behaviors before converting them into evaluated items.Seed inputs include product context, FAQ-style knowledge, dialogue history, and historical tool-use traces.
  • E.3 T3 Robustness Subset Construction: Tool Robustness tests tool selection against distractor tools, while Prompt Robustness additionally tests satisfaction of valid prompt constraints.The corresponding metrics are TC, RC, and ADR for Tool Robustness, and ADR, LLM, and TCO for Prompt Robustness.
  • E.3 T3 Robustness Subset Construction: T3 computes AVG as the fraction of passed rubrics per item before aggregation, with final items potentially containing multiple rubrics.This design evaluates robustness through item-level rubric outcomes rather than a single binary item label.
  • E.4 Independent Bootstrap Stability Check: Table 15 uses 10,000 paired bootstrap resamples over evaluation items to compute independent 95% percentile intervals.The check uses cached model outputs from a separate fixed evaluation run and is intended to assess stability of qualitative contrasts despite generation stochasticity.

F.1 Deployment Inference Benchmark … H.3 Deployment Measurement Contract

The deployment evaluation measures complete-Agent performance under a controlled workload and serving protocol, including standalone MTP adaptation, concurrency-dependent speedups, quality checks, scheduler diagnostics, and reproducibility records. MTP improves low-concurrency throughput without demonstrated aggregate quality degradation, but its benefits weaken under stress and the measurements remain workload- and instrumentation-dependent.

  • Protocol and Workload Shape: 100 requests were retained after excluding 10 warm-up cases from the 110-case deployment set.The client used an office-network host Harness, real RAG and MCP services, a four-round Agent limit, and a 5-second tool timeout; local runs used one H20, BF16, TP=PP=1, and no quantization.
  • Protocol and Workload Shape: Wall latency ends at the Agent’s final response, while TTFT includes networking, queueing, and server work rather than pure kernel prefill.Per-call Decode TPS is the arithmetic mean of completion tokens divided by call latency minus TTFT; client-derived Prefill TPS, TPOT, and inter-token latency are not reported.
  • MTP Draft-Head Adaptation: A task-trained policy can adapt a transplanted base-model NextN head through standalone multi-token cross-entropy training on policy-rollout labels.The MTP update uses supervised labels from on-policy rollouts, with no reward or advantage entering the MTP loss; base MTP initialization avoids the very high initial loss observed in a pilot.
  • MTP across Concurrency: 1.69×/1.52× speedups at C=1–2 retain 100% 15-second attainment, whereas 1.27×/1.19× at C=4–8 provide no P95 improvement.The authors therefore treat MTP as a low-concurrency latency optimization rather than a universal capacity multiplier.
  • MTP across Concurrency: At C=4, Ours reaches Decode 121.61–130.18 with MTP On versus 97.40–102.55 Off, while attainment remains 96–100% versus 99–100%.These are run ranges; Wall P95 and TTFT P95 also vary across runs, so they should not be interpreted as pooled percentiles.
  • H.1 MTP Serving Configuration: MTP-On point estimates are not lower on either quality dimension, but the single run is not an equivalence test because paired outputs and repeated decoding runs are unavailable.The MTP-On serving configuration is recorded separately for the quality check and deployment measurements.
  • H.2 SGLang MTP Diagnostics: HAT’s trace shows mean accept length 3.12 and mean acceptance rate 0.780, up from Base values of 2.94 and 0.735, respectively.These SGLang traces are mechanism diagnostics rather than request-level latency; acceptance measures speculative-token verification, not semantic answer quality, and HAT’s running-request P95 is 7.
Loading 2608.15763v1…