PaperlayerView exact revision on arXiv

Source-linked AI summary

Evidence-Guided Counterfactual Learning for Long-Tailed Robotic Policies

Weiwei Chen, Chenghao Xu, Meiling Yu, Xu Yang, Cheng Deng

arXiv:2610.12861v1cs.RO

TL;DR

Robot demonstration data can be imbalanced across tasks, and tail tasks with insufficient visual grounding may fall back on head-task action patterns. ECL identifies decision-relevant visual evidence and uses frequency-adjusted counterfactual supervision; across two VLA policies, it improves macro-averaged success on LIBERO-Core tasks.

  • Problem

    Robot demonstration datasets can be highly imbalanced across tasks, and weak tail-task visual grounding is associated with greater susceptibility to head-task action bias.

    Source: §1 · pp. 1–2

  • Method

    ECL identifies visual evidence relevant to instructions and actions, constructs counterfactual observations by removing selected evidence, and adapts supervision to task demonstration frequency.

    Source: §1 · p. 2, §3.2.2 · p. 6

  • Results

    Across ten LIBERO-Core tasks, ECL raises macro-averaged success from 40.8% to 53.0% for MiniVLA and from 40.3% to 47.2% for π0.5.

    Source: §4.1 · p. 7

  • Takeaways & Limitations

    ECL improves long-tailed policy learning without additional demonstrations or relation-level annotations, while encouraging task-specific visual grounding.

    Source: §5 · p. 9

    Counterfactual construction and extra forward passes add training overhead, creating an opportunity cost under fixed wall-clock budgets and potentially suboptimal compute allocation.

    Source: §5 · pp. 9–10

Abstract

from arXiv · show

In the practical deployment of vision--language--action policies, handling long-tailed task distributions is a common challenge due to the highly imbalanced composition of robot demonstration datasets. To investigate this issue, we study the head-task action bias in shared robot policies, particularly focusing on its relationship with task-specific visual evidence. Our experimental analysis shows that, when the visual evidence required by a tail task is insufficient, the policy increasingly falls back on action patterns reinforced by data-rich head tasks, resulting in degraded performance on under-represented tasks. Motivated by this observation, we propose Evidence-Guided Counterfactual Learning, a training framework for long-tailed robot policies. Our method identifies decision-relevant visual evidence using instruction-related and action-related signals, and constructs counterfactual observations by removing the selected evidence. The resulting counterfactual predictions are used to strengthen the association among task instructions, visual evidence, and actions, with the supervision adaptively adjusted according to the number of demonstrations available for each task. The proposed framework requires neither additional training data nor region-level annotations and incurs no additional computation during deployment. Extensive experiments across simulated benchmarks, different VLA architectures, and real-world robot platforms demonstrate that our method robustly and consistently improves long-tailed robot policy learning, particularly for tail tasks, while maintaining reliable performance on head tasks.

1 INTRODUCTION

Robot task demonstrations can be highly imbalanced, and tail-task performance drops under long-tailed training as weak visual grounding leaves shared policies susceptible to head-task action bias. ECL uses counterfactual training to strengthen task-specific evidence and adapt supervision to task frequency.

1 INTRODUCTION

  • Many long-tailed learning methods target classification, while APA uses extra tools for augmentation and does not directly constrain head-task action bias.These differences motivate an approach tailored to VLA task formulation and architecture.Source: §1 · p. 1
  • Tail-task performance drops substantially under LT; some head tasks perform worse under FULL, where overall improvements are not uniform.Figure 1(a) compares success rates for policies trained on FULL and LT.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • When tail tasks lack reliable visual grounding, shared policies tend to fall back on frequent head-task action patterns.Head tasks attend reasonably to target-object evidence, though their attention can be more dispersed due to cross-task interference.Source: §1 · p. 2
  • ECL identifies decision-relevant visual evidence and removes it to construct counterfactual observations, strengthening associations among instructions, visual relations, and actions.It uses original data without additional annotations or training data and adapts supervision by demonstration count.Source: §1 · p. 2
  • The paper reports state-of-the-art performance after evaluating ECL on two VLA policies in LIBERO-Core-LT and two real-world robot platforms.The contribution summary says macro-averaged gains and task-specific outcomes are reported.Source: §1 · p. 2

2 RELATED WORK

Prior work addresses long-tailed imbalance through methods developed mainly for classification, while robotic foundation models span diverse architectures and training approaches. Imbalance across robot tasks, states, and behaviors remains a concern.

2 RELATED WORK

  • Long-tailed recognition methods include reweighting, decoupled learning, logit adjustment, feature augmentation, and contrastive approaches, but primarily target discrete class imbalance rather than sequential robotic state and behavior distributions.Examples include LDAM, Balanced Softmax, DisAlign, PaCo, and BCL.Source: §2 · p. 2
  • Robotic foundation models span autoregressive, diffusion-based, and continuous generative policies, yet training data can remain imbalanced across tasks, states, and behaviors.The section surveys models including RT-1, RT-2, OpenVLA, Octo, π0, and SmolVLA.Source: §2 · p. 3

3 DIAGNOSING AND MITIGATING HEAD TASK ACTION BIAS

In a two-task diagnostic, masking visual evidence and increasing task imbalance both accompany stronger head-task action bias under tail-task instructions. ECL counters this with joint language–action evidence selection and counterfactual supervision weighted by task frequency.

3.1 TOY EXPERIMENT ON HEAD TASK ACTION BIAS

  • Under tail-task instructions, masking shifts actions toward the head-task direction, while Head-Prior Score and Head-Action Rate rise with imbalance.Swapping the head–tail assignment reverses the directional preference; the tested ratios combine increasing imbalance with reduced tail-data coverage.Source: §3.1 · pp. 3–4, Figure 2 · p. 3

3.2 FREQUENCY-AWARE COUNTERFACTUAL SUPERVISION

  • ECL uses a counterfactual branch to strengthen associations among task instructions, visual relations, and actions without requiring additional training data or annotations.At deployment, the factual policy alone generates actions.Source: §3.2 · p. 4, Figure 3 · p. 4

3.2.1 RELATION-AWARE COUNTERFACTUAL BRANCH

  • ECL multiplies instruction- and action-related attention scores to prioritize visual tokens relevant to both task semantics and action prediction.It selects the top-K tokens, using a default selection ratio of 10%; the joint score is not treated as the causal contribution of individual tokens.Source: §3.2.1 · p. 5
  • The masked branch provides a reference under incomplete visual evidence, and the contrastive output combines it with the intact-observation prediction for action supervision.The masked reference still reflects unmasked information and is not itself equivalent to head-task action bias.Source: §3.2.1 · p. 5

3.2.2 FREQUENCY-AWARE COUNTERFACTUAL SUPERVISION

  • Factual, masked, and counterfactual-effect predictions receive separate action losses, respectively retaining deployment supervision, providing a masked reference, and strengthening evidence–action associations.The masked branch may still use task information that remains available.Source: §3.2.2 · p. 5
  • Counterfactual supervision increases with task rarity, using demonstration counts as a proxy for the supervision available to each task.The schedule assigns stronger supervision to less frequent tasks while limiting additional constraints on frequent tasks.Source: §3.2.2 · p. 6

4 EXPERIMENTS

The experiments evaluate ECL’s performance in simulation and the real world, test its design choices, and assess robustness to context-level long-tailed distributions. Across these settings, ECL improves success over the relevant baselines.

4 EXPERIMENTS

  • The evaluation tests long-tail performance, design sensitivity, and robustness using LIBERO-Core tasks and MiniVLA and π0.5 policies.Simulation uses ten manipulation tasks and a 7-DoF Franka Panda; main results use factual-only inference.Source: §4 · p. 6, §4.1 · pp. 6–7

4.1 SIMULATION EXPERIMENTS

  • ECL raises macro-averaged success from 40.8% to 53.0% for MiniVLA and from 40.3% to 47.2% for π0.5 in simulation.Against APA, macro-averaged success rises from 44.8% to 53.0% for MiniVLA and from 41.7% to 47.2% for π0.5; tail-task averages also improve for both policies.Source: §4.1 · p. 7

4.2 REAL-WORLD EXPERIMENTS

  • Across four real-world platform-policy combinations, ECL raises macro-averaged success and improves tail-task success over APA on both robots.Tail-task success improves over APA by an average of 9.0% on AGILEX PIPER and 4.3% on xArm6.Source: Table 2 · p. 7, §4.2 · p. 8

4.3 ABLATION STUDIES

  • Across ablations, λ = 0.3 maximizes success, Object–Action Relation reaches 53.0%, a 0.10 masking ratio is best, and masked-branch supervision matters.Without masked-branch supervision, performance stays close to the standard long-tailed baseline; the masked prediction can become nearly identical to the factual prediction, making counterfactual supervision redundant.Source: §4.3 · pp. 8–9, Figure 4 · p. 8

4.4 ROBUSTNESS TO CONTEXT-LEVEL LONG-TAILED DISTRIBUTIONS

  • ECL outperforms the LTC baseline in simulation and real-world context shifts, raising average success from 34.7% to 37.6% and from 28.5% to 34.3%, respectively.Success rates generally decline toward tail contexts.Source: §4.4 · p. 9, Figure 5 · p. 9

5 CONCLUSION, LIMITATIONS AND FUTURE WORK

ECL targets weak task-specific visual grounding as a factor that leaves rare tasks susceptible to head-task action bias. Its extra training computation creates a trade-off between tail-task gains and further head-task optimization.

5 CONCLUSION, LIMITATIONS AND FUTURE WORK

  • ECL uses instruction- and action-relevant visual evidence to construct masked counterfactual observations and adapts supervision strength to task frequency.The approach requires neither additional demonstrations nor relation-level annotations.Source: §5 · p. 9
  • Counterfactual sample construction and extra forward passes add training overhead.Under a fixed wall-clock budget, that computation could otherwise support further head-task optimization.Source: §5 · pp. 9–10
  • Future work will explore more efficient counterfactual supervision and adaptive computation allocation to improve the trade-off across tasks.Source: §5 · p. 10

6 AI USE STATEMENT

The authors report using an AI-based language tool to refine the manuscript’s prose and state that they reviewed its suggestions and take accountability for the final version.

6 AI USE STATEMENT

  • The authors used an AI-based language tool for prose refinement and reviewed its suggestions, retaining accountability for the submitted content and research artifacts.Source: §6 · p. 10

A.1 SIMULATION TASK-LEVEL LONG-TAILED DATASET

The appendix describes the LIBERO-Core long-tailed dataset and how task frequencies define head and tail groups.

A.1 SIMULATION TASK-LEVEL LONG-TAILED DATASET

  • LIBERO-Core-LT uses a Pareto-based task-frequency distribution over ten tasks, with Tasks 0–2 designated head and Tasks 3–9 tail.The task counts set the frequency-aware correction strength; Table 3 lists task instructions and demonstration counts.Source: §A.1 · p. 13, Table 3 · p. 13

A.2 REAL-WORLD TASK-LEVEL LONG-TAILED DATASET

The real-world benchmark defines six manipulation tasks across AGILEX PIPER and xArm6, with balanced FULL and long-tailed LT demonstration allocations.

A.2 REAL-WORLD TASK-LEVEL LONG-TAILED DATASET

  • LT uses 50, 46, 29, 13, 10, and 9 demonstrations across tasks, versus 45–50 per task in FULL.The resulting allocations are 157 demonstrations for LT and 285 for FULL, with an LT maximum-to-minimum frequency ratio of approximately 5.56:1.Source: §A.2 · p. 13
  • The benchmark covers six grasp-and-place tasks on AGILEX PIPER and xArm6; PIPER has wrist- and global-camera views, while xArm6 has global-camera views.Tasks involve placing objects onto supports or into receptacles.Source: §A.2 · p. 13, Table 4 · p. 13
  • Figure 6 illustrates the six manipulation tasks; Figure 6 examples include PIPER global- and wrist-camera views and xArm6 global-camera views.Task columns correspond to those listed in Table 4.Source: Figure 6 · p. 14

A.3 LONG-TAILED CONTEXT (LTC) DATASET

The Long-Tailed Context dataset holds the instruction and manipulation goal fixed while varying scene configuration, with rarer configurations receiving fewer demonstrations.

A.3 LONG-TAILED CONTEXT (LTC) DATASET

  • Scene IDs 0–5 denote increasingly rare context configurations, while all scenes share the same language instruction.The dataset is collected on AGILEX PIPER with synchronized wrist- and global-camera views.Source: §A.3 · p. 14, Table 5 · p. 14
  • FULL allocates 50 demonstrations per scene, whereas LT progressively reduces demonstrations for rarer scene configurations.Figure 7 shows the six scene configurations and both camera views.Source: Table 5 · p. 14, §A.3 · p. 14

B EXPERIMENTAL ROLLOUT RESULTS

The appendix complements aggregate success rates with representative closed-loop trajectories across simulation and real-world evaluations, and includes visual analyses of masking strategies and tail-task evidence grounding.

B EXPERIMENTAL ROLLOUT RESULTS

  • The appendix presents representative rollouts for two tasks per policy backbone, alongside real-world rollouts on AGILEX PIPER and xArm6.Simulation tables show frames sampled at fixed evaluation steps; real-world tables show selected camera frames.Source: B EXPERIMENTAL ROLLOUT RESULTS · p. 15

B.1 SIMULATION ROLLOUTS

  • Each simulation task is evaluated for 675 episodes across three seeds, using the same evaluation budget and stopping rule for MiniVLA and π0.5.The appendix includes rollout tables for MiniVLA Tasks 3 and 6 and π0.5 Tasks 8 and 9.Source: §B.1 · p. 15, Table 6 · p. 15, Table 7 · p. 15

B.2 REAL-WORLD ROLLOUTS

  • Real-world policy–robot pairs are tested for 50 trials per task with randomized initial object poses, and each rollout is represented by eight uniformly sampled frames.The rollout comparison covers APA and ECL.Source: §B.2 · p. 17, Table 11 · p. 17

B.2.1 AGILEX PIPER

B.2.2 XARM6

  • On xArm6, MiniVLA and π0.5 rollout comparisons use global-camera views; Figure 8 reports per-task success rates for APA and ECL.The rollout examples illustrate controller behavior; quantitative claims are based on the complete success-rate evaluation.Source: Table 14 · p. 18, Table 15 · p. 18, Figure 8 · p. 18

C COMPARISON OF MASK SELECTION STRATEGIES

  • Figure 9 holds the observation and masking budget fixed at 26 of 256 patches to compare object-only, action-only, random, and object–action relation selection.The relational illustration combines object coverage and attention with an additional goal-region contribution; random patches are dispersed, while relational patches include areas around the bottle and rack.Source: C COMPARISON OF MASK SELECTION… · p. 19, Figure 9 · p. 19

D EFFECTIVENESS OF VISUAL EVIDENCE IN ECL

  • For two LT tail tasks, ECL focuses on instructed objects, interaction regions, and target locations, whereas the LT baseline often attends to background or irrelevant regions.The analysis links improved task-specific visual evidence from scarce demonstrations to action prediction; ECL uses instruction- and action-related signals to guide counterfactual intervention and strengthen the instruction–evidence–action association.Source: D EFFECTIVENESS OF VISUAL EVIDENCE IN… · p. 19, Figure 10 · p. 20

E TRAINING COST DETAILS

The cost analysis compares recorded training usage and explains how to interpret those timings, including the added cost observed when ECL’s counterfactual branch is active.

E TRAINING COST DETAILS

  • Recorded timing boundaries cannot be verified, so logged time is not treated as end-to-end training time and excludes unrecorded preparation, checkpointing, evaluation, and restart intervals.The reported log statistic is the arithmetic mean of timing records in hours.Source: E TRAINING COST DETAILS · p. 20
  • ECL’s mean recorded step time rises from 0.519 s during the 1,923-step warm-up to 1.100 s across the subsequent 32,698 steps with the counterfactual branch active.The phase-mean ratio is approximately 2.12×; this is a within-run comparison, not a matched measurement against LT.Source: E TRAINING COST DETAILS · p. 20
  • 19,234 samples and 34,622 logged steps are shared by LT and ECL, but ECL records 10.27 hours versus LT’s 5.89.ECL’s mean step time is 1.068 s, compared with 0.608 s for LT; the additional computation is confined to training.Source: E TRAINING COST DETAILS · p. 21

Related papers

Loading 2610.12861v1…