PaperlayerView exact revision on arXiv

Source-linked AI summary

Echo: Coupled Action and Task Status Signal Denoising for Vision Language Action Models

Yufeng Liu, Bohao Fan, Juntu Zhao, Xinyi Wang, Hang Yu, Wenzhuan Luo, Yingzhi Wang, Qinghai Li, Yingdong Hu, Junliang Guo, Richard Wang, Dequan Wang, Yang Gao, Cheng Ma

arXiv:2610.31569v1cs.RO

TL;DR

Reliable robot execution requires recognizing task status, yet existing methods may need fine-grained supervision or complex integration and often use signals only for monitoring. Echo jointly denoises status signals and actions within one VLA, learning from successful, failed, and recovery data and using cleaner signal estimates to guide action generation. Across five real-world tasks, it accurately predicts status and improves task completion.

  • Problem

    Existing task-status methods can require frame-level supervision, external monitoring, or complex designs, while their signals are typically used only for monitoring rather than action generation.

    Source: §1 · pp. 1–2

  • Method

    Echo adds stop and failure signals to action chunks, learns them from successful, failed, and recovery data without frame-level progress annotations, and uses controlled interactions and faster signal denoising to guide actions.

    Source: §1 · p. 2

  • Results

    Across five real-world tasks, Echo accurately predicts task status and improves task completion over action-only baselines, with larger out-of-distribution gains.

    Source: §1 · p. 2, §4.2 · pp. 7–8, §5 · p. 10

  • Takeaways & Limitations

    The experiments support using task-status signals for both monitoring and action generation in real-world manipulation.

    Source: §5 · p. 10

Abstract

from arXiv · show

Recognizing task completion and failure enables robots to stop upon success and initiate recovery or replanning after failure. Existing methods augment Vision Language Action (VLA) policies with task status signal prediction to support these execution decisions. However, learning and integrating these signals can require fine-grained supervision, external monitoring models, or complex architectural designs. Moreover, the potential of task status signals to directly guide action generation remains underexplored. We present Echo, a framework that combines task status signal learning without frame-level progress annotations with a compact design for status prediction and action generation within a single VLA. Specifically, Echo represents task status with stop and failure signals, added as two dimensions of each action chunk and denoised together with robot actions. Their combination distinguishes ongoing execution, successful completion, and failure. To learn these signals without fine-grained progress annotations, Echo constructs successful, failed, and recovery trajectories with complementary supervision properties. To enable status signal-guided action generation, Echo controls interactions between the signals and actions and denoises the signals faster than the actions. These allow later action denoising steps to condition on cleaner status estimates while preserving accurate signal prediction. Extensive real-world manipulation experiments show that Echo accurately predicts task status and consistently improves policy performance.

1 INTRODUCTION

Reliable robot execution requires recognizing whether a task is ongoing, successful, or failed, but existing status-signal approaches can require fine-grained supervision or complex integration and often do not use signals to guide actions. Echo jointly predicts stop and failure signals with actions, learns from complementary trajectory types, and uses status estimates during action generation.

1 INTRODUCTION

  • Reliable execution requires distinguishing ongoing tasks, success, and failure to decide whether to continue, stop, or intervene.Existing methods predict progress, completion, or failure within VLA policies or through external monitors.Source: §1 · p. 1
  • Existing approaches can require frame-level supervision, external monitors, or complex architectures, and typically use predicted signals only for monitoring.The potential for signals to improve action generation remains underexplored.Source: §1 · pp. 1–2
  • Echo jointly denoises stop and failure signals with robot actions in one compact VLA, using the signals to improve action generation without frame-level progress annotations.The signals are added as two dimensions of each action chunk, with little additional parameter overhead.Source: §1 · p. 2
  • Echo learns status signals from successful demonstrations, failed rollouts, pre-intervention failure segments, and recovery demonstrations using subtask prompts and success or failure labels.Failed actions are excluded from imitation; recovery demonstrations pair failure supervision with corrective actions at corresponding recoverable failure states.Source: §1 · p. 2
  • Echo controls interactions among stop signals, failure signals, and actions, and denoises signal tokens faster so later action steps use cleaner status estimates.For recoverable failures, those estimates and current observations can guide actions toward corrective behavior learned from recovery demonstrations.Source: §1 · p. 2
  • Experiments on five real-world manipulation tasks show accurate status prediction and improved performance over action-only baselines, with larger out-of-distribution gains.The paper also examines component ablations, signal interventions, and failure-data scaling.Source: §1 · p. 2

2 RELATED WORKS

Prior work predicts task status through reward models, completion and failure detectors, or lightweight probes, while other approaches use execution feedback to adjust policies or trigger recovery. Jointly maintaining reliable status prediction and directly guiding actions with low overhead remains an open challenge.

2 RELATED WORKS

  • Task-status work includes reward models, completion prediction, failure detection, and lightweight probes, but standalone evaluators add evaluation alongside the control policy.Accurate signal extraction alone does not ensure feedback to action generation.Source: §2 · p. 2
  • Execution feedback has been used for online reinforcement learning, trajectory-preference optimization, intervention, backtracking, and corrective control.VLA-Corrector uses gradient-guided replanning, while ProgressVLA propagates progress gradients through a world model to guide action denoising.Source: §2 · p. 2
  • Jointly achieving reliable status prediction and signal-guided action adjustment with low overhead remains an open challenge.Existing approaches may add policy optimization, monitoring and recovery components, or inference-time gradient computation.Source: §2 · p. 3

3 METHODOLOGY

Echo extends action-chunk prediction with stop and failure signals, learns their targets from segment boundaries and recovery progress, and uses status estimates to guide action generation. It combines controlled signal–action interactions with asynchronous denoising so actions can use cleaner status estimates.

3.1 PRELIMINARIES

  • Action chunks predict H-step actions relative to the chunk’s starting robot state; padding marks positions beyond the current segment and supplies boundary-based status labels without frame-level progress annotations.The flow-based policy conditions on observations, robot state, and instruction, and integrates predicted velocity from noise toward clean data.Source: §3.1 · pp. 3–4

3.2.1 TASK-STATUS REPRESENTATION

  • Stop and failure signals share the action horizon and distinguish ongoing execution, successful completion, and terminal failure through their combined estimates.Low stop indicates ongoing execution; high stop with low failure indicates success, while high values for both indicate terminal failure.Source: §3.2.1 · p. 4

3.2.2 DATA CONSTRUCTION AND SUPERVISION

  • Successful demonstrations supervise actions and zero failure targets; failed data supervise status rather than actions, while recovery data pair failure-state supervision with corrective actions.Padding and segment type provide boundary-based targets; recovery stop targets are supervised only at non-padding positions, and recovery failure targets decrease with progress.Source: §3.2.2 · p. 5

3.2.3 STATUS-GUIDED ACTION GENERATION

  • Action tokens attend to both signal groups, while signal tokens attend to actions; direct stop–failure attention is blocked to preserve distinct semantics.Asynchronous denoising makes signals clean before action denoising finishes, allowing later action updates to use cleaner status estimates; synchronous estimates show signals converge earlier than actions.Source: §3.2.3 · p. 6, Figure 3 · p. 6

4 EXPERIMENTS

Experiments evaluate Echo against action-only and existing-method baselines on real-world manipulation tasks, then analyze distribution shift, component choices, signal interventions, failure-data quantity, and simulation performance. The protocol uses π0.5 and, unless specified otherwise, 30 trials per model and task.

4.2 MAIN RESULTS

  • Echo predicts task status accurately without frame-level progress annotations and outperforms the action-only π0.5 baseline on every real-world task.Both models use the same action-supervision data; Table 2 reports task completion score and frame- and trajectory-level Macro-F1.Source: §4.2 · pp. 7–8, Table 2 · p. 7

4.3 COMPARISON WITH EXISTING METHODS

  • On Insert Flower, Echo achieves the strongest status-prediction and task-completion performance on the applicable metrics among CycleVLA, VLA-Corrector, and Robometer.Robometer is evaluated using binary success/non-success Macro-F1 because it lacks termination prediction.Source: Table 3 · p. 7, §4.3 · p. 8

4.4.1 IN-DOMAIN AND OUT-OF-DISTRIBUTION ANALYSIS

  • Echo’s gains are larger under OOD conditions, while ID differences are modest and task-dependent; status prediction remains broadly stable under distribution shift.Trajectory-level performance declines slightly under shift, while frame-level performance changes modestly.Source: §4.4.1 · p. 8, Table 4 · p. 8

4.4.2 COMPONENT ABLATIONS

  • Blocking action attention to status signals preserves signal quality but reduces task completion near the action-only baseline; direct SS–FS attention and shared MLPs also degrade prediction or execution.Direct SS–FS attention reduces frame-level accuracy and task completion while preserving strong trajectory-level performance; shared MLPs cause larger degradation in both prediction and execution.Source: §4.4.2 · pp. 8–9, Table 5 · p. 8
  • Synchronous denoising modestly decreases frame-level accuracy but degrades task completion and timely outcome recognition.The ablation supports making cleaner status estimates available earlier for subsequent action updates.Source: §4.4.2 · p. 9

4.4.3 DO TASK-STATUS SIGNALS INFLUENCE ACTION GENERATION?

  • With SS fixed at zero, imposing FS = 1 elicits repeated upward recovery motions, whereas FS = 0 produces task-directed motions on Insert Flower.This intervention shows that prescribed status inputs can directly steer action generation toward corrective behavior.Source: §4.4.3 · p. 9, Figure 5

4.4.4 EFFECT OF FAILURE DATA QUANTITY

  • As terminal-failure data grows, frame-level status prediction improves while task-completion scores remain broadly stable.The terminal-failure actions are excluded from imitation; pre-intervention failures and recovery demonstrations provide recoverable-error supervision and corrective actions, respectively.Source: §4.4.4 · p. 9, Figure 6 · p. 9

4.5 SIMULATION EVALUATION

  • Across 4,500 LIBERO-Recovery trials, Echo reaches 34.76% task success versus 32.67% for action-only π0.5, and achieves 89.97% frame-level status accuracy.The evaluation covers 150 fixed scenes across five LIBERO suites.Source: §4.5 · p. 10

5 CONCLUSIONS

The conclusion presents Echo as a single-VLA framework for jointly denoising actions and task status signals. Its real-world experiments support using status signals for both monitoring and action generation.

5 CONCLUSIONS

  • Echo jointly denoises actions and task status signals within a compact, single-VLA framework.The conclusion describes controlled signal–action interactions and asynchronous denoising as part of the approach.Source: §5 · p. 10
  • Echo learns from successful, failed, and recovery data without frame-level progress annotations, using status signals to guide action generation.The conclusion identifies controlled interactions and asynchronous denoising as the means used to guide action generation.Source: §5 · p. 10
  • Experiments on five real-world tasks demonstrate accurate status prediction and improved task completion, supporting status signals for monitoring and action generation.Source: §5 · p. 10
  • The authors report using generative AI tools for clarity, grammar, and readability edits, which they reviewed and take responsibility for.Source: AI USE STATEMENT · p. 10

A.1 ROBOT PLATFORM AND TASK DETAILS

The appendix describes the robot platform, common episode conditions, five evaluated tasks, and their evaluation positions. Tasks share an initial robot state and a five-minute episode limit.

A.1 ROBOT PLATFORM AND TASK DETAILS

  • The wheeled humanoid has two seven-degree-of-freedom arms, grippers, a six-degree-of-freedom waist–leg mechanism, and three RGB cameras; its mobile base remains stationary during tasks.The policy controls 20 degrees of freedom through the arms, grippers, and waist–leg mechanism.Source: §A.1 · p. 13
  • Episodes last at most five minutes and count as failures if incomplete at the time limit or if the robot reaches an unsafe posture before completion.Tasks start from the same initial robot state, with objects placed at specified evaluation positions.Source: §A.1 · p. 13, Table 6 · p. 13
  • The five tasks are Insert Flower, Capsule Toy, Insert Pen, Make Tanghulu, and Open Drawer, covering grasping, precise insertion, and articulated-object interaction.Success is task-specific: examples include successful insertion, opening the drawer, or dispensing a capsule by rotating its knob.Source: §A.1 · p. 13

A.2.1 TASK COMPLETION SCORE

Task Completion Score assigns points to successful and failed episodes, with task-specific deductions and a partial-credit exception for Make Tanghulu.

A.2.1 TASK COMPLETION SCORE

  • Insert Flower and Insert Pen lose one point per contact event, capped at three points.Source: §A.2.1 · p. 13
  • Capsule Toy and Open Drawer lose one point per empty grasp attempt, capped at five points.Source: §A.2.1 · p. 13
  • Successful episodes receive 10 points before task-specific deductions; failed episodes receive 0 points except for a specified Make Tanghulu outcome.When the hawthorn falls off after insertion and release, that failed episode receives 5 points as partial credit.Source: §A.2.1 · p. 13, Table 7 · p. 14

A.2.2 FRAME-LEVEL AND TRAJECTORY-LEVEL METRICS

Frame-level and trajectory-level status metrics are computed per episode and then aggregated across episodes. Trajectory evaluation checks predictions within a fixed window beginning at each annotated outcome boundary.

A.2.2 FRAME-LEVEL AND TRAJECTORY-LEVEL METRICS

  • Frame-level Macro-F1 is computed per episode over execution-state classes present in its ground-truth annotations.Source: §A.2.2 · p. 14
  • Trajectory-level evaluation checks the first W predictions from each annotated success or failure boundary, prioritizing failure, then success, and otherwise ongoing.Intervals shorter than W use all available frames.Source: §A.2.2 · p. 14
  • Trajectory outcome F1 is computed per episode over ground-truth outcome classes; an ongoing prediction counts as a missed outcome.The main-text results use W = 15 frames, equivalent to 3 seconds at 5 queries per second.Source: §A.2.2 · p. 14
  • Metrics are aggregated by reporting their mean and standard error across episodes, with 30 episodes per task.The same episode-first aggregation is used for task score and both F1 metrics.Source: §A.2.2 · p. 14

A.3 DATA SAMPLING RATIO

Training uses the same sampling ratio across tasks for successful demonstrations, failures, pre-intervention failures, and recovery data.

A.3 DATA SAMPLING RATIO

  • Each task uses a 20 : 1 : 2 : 20 sampling ratio for Success, Failure, Pre-intervention failure, and Recovery data, respectively.Source: §A.3 · p. 14

A.4 IMPLEMENTATION, HYPERPARAMETERS, AND INFERENCE COST

The appendix details Echo’s training and deployment settings, implementation choices, baseline evaluations, sensitivity analyses, and simulation study. It reports low added inference latency, task-status prediction findings, and improved simulation success over an action-only baseline.

A.4.2 INFERENCE OVERHEAD

  • Echo adds 5 ms of inference latency on an NVIDIA RTX 4090, increasing from 115 ms to 120 ms despite expanding the generated sequence from 60 to 180 tokens.Source: §A.4.2 · p. 15

A.5 IMPLEMENTATION OF THE MOTIVATING EXPERIMENT

  • The motivating experiment illustrates that status estimates can stabilize earlier than action curves, motivating cleaner signal estimates for later action updates without implying every signal is correct.The synchronous reference jointly denoises actions, SS, and FS; Echo’s asynchronous schedule makes signals clean sooner, while direct SS–FS attention remains blocked.Source: §A.5 · pp. 15–16

A.6 BASELINE IMPLEMENTATIONS AND PERFORMANCE ANALYSIS

  • On Insert Flower, CycleVLA and Robometer sometimes lack discriminative capability for real-world failures, while the action-only baseline predicts no status metrics.CycleVLA lacks real-world policy failure data; VLA-Corrector uses latent-space monitoring and corrective replanning but does not establish terminal failure.Source: §A.6 · pp. 16–17

A.7 DETAILED TASK-STATUS PREDICTION RESULTS

  • Ongoing-state recall exceeds 99% on every task, while frame-level recognition of success and failure is lower.Table 9 provides detailed frame-level and trajectory-level metrics, including per-state recall and outcome counts.Source: §A.7 · p. 17, Table 9 · p. 18

A.8 HYPERPARAMETER SENSITIVITY

  • Frame-level F1 is generally more sensitive to the stop-signal threshold than the failure-signal threshold, and the best stop threshold varies by task.Trajectory-level results depend jointly on threshold and window: longer windows can improve recognition, but do not rescue every threshold choice.Source: §A.8 · pp. 18–19

A.9 COMPATIBILITY WITH EXTERNAL MONITORING AND PLANNING

  • Echo’s internal status estimates and corrective action generation are architecturally compatible with external monitoring and planning, but any combined system requires separate evaluation.Source: §A.9 · p. 19

A.10 SIMULATION EXPERIMENTS

  • The simulation evaluates ongoing and success recognition, not terminal-failure timing; timeout is unsuccessful execution but lacks a terminal-failure onset annotation.The study uses 150 fixed LIBERO-Recovery scenes and 4,500 trials per model; Echo continues through the full rollout so signals can be inspected after success.Source: §A.10 · pp. 19–20
  • Echo succeeds on 1,564 of 4,500 simulation trials (34.76%), versus 1,470 (32.67%) for the action-only baseline, a 2.09-percentage-point increase.Echo’s frame-level signal accuracy is 89.97% (1,725,710/1,918,181 evaluated frames); its success rate is higher on all three evaluation seeds.Source: Table 10 · p. 19, §A.10 · p. 21

Related papers

Loading 2610.31569v1…