PaperlayerView exact revision on arXiv

Source-linked AI summary

FabriMAE I Trust Myself? Self-Evaluating VLA Action Generation with Markov Attention Entropy

Aniri, Chen Yilin, Jinhe Bi, Junfei Guo, Donglai Ran, Xu Bian, Zengjie Jin, Yujun Wang, Yijun Tian, Volker Tresp, Fei Shen, Tat-Seng Chua, Yunpu Ma

arXiv:2608.16697v1cs.AI

TL;DR

VLAs lack a unified, externally supervision-free way to assess action-generation reliability across heterogeneous architectures. The paper formulates action generation as a Conditional Generative Markov Chain and proposes MAE, which converts internal attention entropy into architecture-aware reliability scores. Across heterogeneous VLAs and LIBERO-Reflect, MAE improves reliability ranking over baselines, while FabriMAE supports verifier-free test-time action selection for PI-family models.

  • Problem

    A unified way to transform VLA internal signals into architecture-agnostic action-reliability metrics without external supervision remains missing.

  • Method

    MAE models heterogeneous VLA generation as a Conditional Generative Markov Chain and converts attention entropy during latent-state transitions into architecture-aware reliability scores.

  • Results

    Across LIBERO-Reflect’s four capability axes and three open-source VLAs, MAE improves reliability ranking over baselines across AUROC, AUPR, and FPR@95 without external evaluators or repeated rollouts.

  • Takeaways & Limitations

    Internal attention dynamics can support self-evaluation and verifier-free test-time action selection for PI-family Latent-Refinement VLAs.

  • Takeaways & Limitations

    Baseline comparisons are not uniform across action-generation types because continuous-action and flow-action policies cannot expose the token variables used by some estimators.

Abstract

from arXiv · show

Vision-Language-Action models (VLAs) integrate visual perception, language instruction, and action generation into end-to-end policies across heterogeneous architectures. However, enabling VLAs to self-evaluate their action generation reliability without external supervision remains a major challenge. Existing methods either rely on expert annotations or estimate uncertainty only from output statistics, largely ignoring internal signals. In this work, we observe that internal visual modality entropy exhibits consistent distinctions between successful and failed tasks across heterogeneous VLAs. Although VLAs' architectures differ in their action generation, we show that they share a common latent action generation abstraction evolving under visual perception, language instruction, and state input, which we formulate as a Conditional Generative Markov Chain. Based on this formulation, we propose MAE (Markov Attention Entropy), a self-evaluation framework that directly converts internal attention signals into architecture-aware reliability scores, and introduce LIBERO-Reflect, a 4,000-episode benchmark combining 2,000 standard episodes and 2,000 challenging episodes across four subsets. Extensive experiments across heterogeneous VLA architectures and diverse scenarios show that MAE consistently outperforms state-of-the-art baselines on AUPR, AUROC, and FPR@95. We further instantiate FabriMAE for verifier-free test-time action selection, showing that MAE-guided multiple sampling improves PI-family robustness on LIBERO-Plus with small observed runtime overhead.

1 Introduction

The paper frames heterogeneous VLA action generation as a Conditional Generative Markov Chain and proposes MAE, a white-box framework that converts internal visual-attention entropy into architecture-aware reliability scores. It evaluates MAE on LIBERO-Reflect and extends it to verifier-free test-time action selection with FabriMAE.

  • Internal signal: Visual attention entropy separates successful and failed executions more clearly than text attention entropy because it reflects how the transition process accesses visual evidence.Successful queries concentrate on task-relevant objects, whereas failed queries disperse across distractor regions.
  • Evaluation benchmark: 4,000 episodes form LIBERO-Reflect, combining 2,000 standard and 2,000 challenging episodes across four subsets and four capability axes.The benchmark evaluates whether self-evaluation scores rank successful episodes above failed episodes across AUROC, AUPR, and FPR@95.
  • Unified formulation: Heterogeneous VLAs are unified by modeling latent action states as a Conditional Generative Markov Chain conditioned on visual observations, language instructions, and state input.The transition kernel captures architecture-specific autoregressive, continuous-readout, or flow-style refinement updates.
  • MAE framework: MAE converts internal attention dynamics into reliability scores by measuring visual-attention entropy with architecture-aware orientations, MAE-D and MAE-C.The framework targets white-box self-evaluation without relying on external supervision.
  • Experimental findings: MAE improves reliability ranking across three open-source VLAs over black-box and white-box baselines without external evaluators, repeated rollouts, or auxiliary model passes.The evaluated models are OpenVLA, OpenVLA-OFT, and QwenPI-Flow, spanning latent-readout and latent-refinement families.
  • Test-time selection: FabriMAE uses MAE-guided branch sampling for verifier-free test-time action selection in PI-family latent-refinement VLAs, improving LIBERO-Plus success rate with small observed runtime overhead.The procedure targets action selection without a verifier.

2 Preliminary

The paper models VLA action generation as a Conditional Generative Markov Chain over latent action states conditioned on context, rather than as physical robot-environment dynamics. The final latent state is converted into an executable action through an architecture-specific action head, while the transition kernel provides a common abstraction across heterogeneous VLAs.

  • Conditional Generative Markov Chain: VLA action generation evolves hidden latent action states through internal steps before producing an executable action or action chunk.The latent states are not directly executed by the robot; the final state z_t,K is mapped to a_t through the policy’s action head.
  • Formal definition: The formal chain M = (Z, C, K, P_θ) comprises latent state space, conditioning space, internal generation-step count, and a neural transition kernel.Its internal process is z_t,0 → z_t,1 → · · · → z_t,K → a_t.
  • Formal definition: At each internal step, the next latent state is sampled from P_θ(· | z_t,k, C_t), with the initial state drawn from ρ_0.The kernel can represent autoregressive token updates, continuous readout, flow-style refinement, or deterministic updates as degenerate kernels.
  • Markov property: The Markov property applies to the complete latent action state, making the transition kernel a shared object across heterogeneous VLA action-generation mechanisms.Given z_t,k and C_t, the next state does not additionally depend on earlier latent states.

3 Methodology

MAE interprets VLA action generation as conditional Markov information routing and measures how action-query tokens address visual context through transformer attention entropy. It aggregates oriented entropy across steps and selected heads, using architecture-specific orientations for different VLA generation mechanisms.

  • Markov Attention Entropy: Under the Conditional Generative Markov Chain view, each latent action transition is an information-routing step that queries conditioning context.Transformer attention from action-query tokens to conditioning-context tokens measures this routing behavior at each generation step, layer, and head.
  • Visual Attention Entropy: Visual attention entropy measures uncertainty in visual addressing: high entropy spreads attention across patches, whereas low entropy concentrates it.In the Markov formulation, the entropy reflects how the transition kernel accesses visual perception while updating the action state.
  • Episode MAE Score: Episode MAE averages visual entropy across execution steps and action queries, then selects m heads per layer using an orientation ω ∈ {−1, +1}.The selected heads are those with the largest oriented entropy, producing the episode-level MAE score.
  • Architecture-Specific Metrics: MAE-D applies to Latent-Readout VLAs, while MAE-C applies to Latent-Refinement VLAs with broader visual addressing retained during final refinement.MAE-D is used for OpenVLA and OpenVLA-OFT, whereas MAE-C is used for QwenPI-Flow; larger oriented MAE indicates a more reliable episode.

4 LIBERO-Reflect Benchmark

LIBERO-Reflect is a 4,000-episode benchmark for evaluating whether VLA reliability scores rank successful episodes above failed episodes under matched standard and challenging conditions. It spans four capability-oriented subsets and uses realized simulator success with AUROC, AUPR, and FPR@95 for assessment.

  • Benchmark construction: LIBERO-Reflect contains 2,000 standard and 2,000 challenging episodes, with 500 episodes per suite in each pool across four LIBERO suites.The standard pool uses LIBERO-Goal, LIBERO-Spatial, LIBERO-10, and LIBERO-Object; the challenging pool is sampled from LIBERO-PRO.
  • Capability subsets: The benchmark comprises four 1,000-episode subsets: Goal Semantics, Object Binding, Spatial Grounding, and Composite Generalization.Each subset combines 500 standard and 500 challenging episodes and targets distinct grounding, reasoning, or generalization capabilities.
  • Assessment protocol: Each episode receives a scalar reliability score, and methods are evaluated by whether higher scores identify successful rollouts while lower scores identify failed rollouts.Ground-truth labels are strictly the actual simulator success flags, not nominal dataset membership.
  • Evaluation metrics: AUROC measures global ranking, AUPR emphasizes precision under class imbalance, and FPR@95 measures false positives when 95% of successful episodes are recalled.Higher AUROC and AUPR are better, whereas lower FPR@95 indicates fewer failed episodes ranked as reliable.
  • Assessment protocol: Matched suite organization, task counts, initializations, simulator, and rollout protocol mitigate first-order shortcuts from nominal standard-versus-challenging source membership.Metrics are computed from realized simulator success within the matched mixed-difficulty pool.

5 Experiments

Experiments evaluate MAE across heterogeneous VLA architectures, LIBERO-Reflect task subsets, efficiency conditions, and design choices. Results show strong cross-architecture reliability, low overhead, robust visual-head and all-layer choices, and effective verifier-free action selection with FabriMAE.

  • Experimental setup: Experiments evaluate OpenVLA, OpenVLA-OFT, and QwenPI-Flow across four LIBERO-Reflect subsets using black-box and white-box self-evaluation baselines.MAE-D is reported for Latent-Readout VLAs and MAE-C for Latent-Refinement VLAs, using all layers and Top-1 head selection.
  • Reliability performance: MAE improves OpenVLA by up to 90.1% AUROC on Object Binding versus Random and reduces FPR@95 by 70.5%.The comparison uses the full black-box and white-box baseline protocol across three VLA backbones and four LIBERO-Reflect subsets.
  • Cross-domain transfer: MAE transfers across architectures, initialization families, and task families, with gains on Goal Semantics, Object Binding, and Spatial Grounding.The evaluated policies differ in VLM backbones, perception stacks, training data, and action-generation mechanisms.
  • Cross-task monitoring: On Reflect-10, OpenVLA-OFT reaches 78.57 AUROC and QwenPI-Flow reaches 79.52 AUROC, supporting a shared execution-correctness signal across heterogeneous tasks.Reflect-10 combines diverse goals, objects, and layouts from LIBERO-10.
  • Efficiency: MAE adds < 0.1× overhead, whereas Self-Consistency and Verbal Confidence incur ≥1× latency overhead, placing MAE in the high-efficiency zone.MAE reuses internal attention maps, while the comparison methods require multiple sampled generations or external models.
  • Ablations: Visual-head entropy consistently improves AUROC over text-head entropy, Top-1 provides the most consistent ranking, and all-layer aggregation gives the best overall AUROC.Top-1 also minimizes the selected-head set and entropy aggregation cost; layer ablations compare shallow, middle, deep, and all-layer variants.
  • Verifier-free action selection: FabriMAE selects among sampled action chunks using candidate-level MAE scores and reaches 86.80% overall success rate on full LIBERO-Plus.For the measured PI0.5 policy with branch FabriMAE, Normal improves by 1.10 percentage points, with gains across Camera, Language, Light, Background, and Layout perturbations.

6 Conclusion … OpenVLA-OFT

The paper presents MAE as a white-box framework that uses internal attention dynamics for VLA self-evaluation, and FabriMAE applies the same signal to verifier-free test-time action selection. The appendices document related work, heterogeneous model configurations, LIBERO-Reflect construction, and OpenVLA-OFT evaluation details.

  • 6 Conclusion: MAE uses internal attention dynamics to improve VLA reliability ranking without external evaluators on LIBERO-Reflect.The framework is presented as a white-box approach to VLA self-evaluation.
  • 6 Conclusion: FabriMAE uses the same internal signal for verifier-free test-time action selection in PI-family Latent-Refinement VLAs.The conclusion identifies verifier-free action selection as an application of MAE.
  • A Related Work: Prior VLA self-evaluation studies estimate uncertainty from generated actions or add learned components, whereas this work evaluates action-generation reliability under visual observations and language instructions.The related-work passages describe calibration, agreement-based self-evaluation, output-action uncertainty, and additional learned failure-analysis components.
  • B Model Configuration Details: The experiments deliberately pair two VLM backbone families with different action-generation mechanisms and training or adaptation regimes to test MAE across heterogeneous policies.Table 4 maps experimental model names to evaluated checkpoints and configurations.
  • C LIBERO-Reflect Construction Details: LIBERO-Reflect contains 4,000 episodes, with each subset contributing 500 standard and 500 challenging episodes.The benchmark combines 2,000 standard and 2,000 challenging episodes across four subsets.
  • OpenVLA-OFT: OpenVLA-OFT uses official checkpoints with efficient OFT adaptation, continuous actions, action chunking, and L1 regression.The configuration uses an OpenVLA-7B visual and language backbone based on DINOv2, SigLIP, and Llama 2 7B.
  • OpenVLA-OFT: The evaluated configurations also include Qwen3-VL-PI-LIBERO-4in1 with DiT-B, 7-DoF actions, horizon 8, and data_mix=libero_all, testing cross-family and cross-generation generality.The appendix identifies Qwen3-VL flow-style refinement and LIBERO-specific training as part of the configuration diversity.
  • C LIBERO-Reflect Construction Details: The benchmark’s challenging episodes preserve task language while swapping target-object and surrounding-object placements, and nominal labels can differ from actual success.Nominal standard episodes may fail and nominal challenging episodes may succeed depending on policy capability.

D Ablation Details and Exact Values

The ablations examine whether MAE depends on head averaging, head modality, or manually selected layer ranges. The main protocol uses visual heads, Top-1 selection, and all layers to preserve stable architecture-aware signals with low aggregation overhead and avoid task-specific tuning.

  • Main configuration: The main configuration combines all layers with Top-1 head selection to preserve the strongest, most stable signal while minimizing aggregation overhead.Larger Top-m settings are reserved for the head-selection ablation, with half-head defined as Top-16 for 32-head OpenVLA-family policies and Top-20 for the 40-head QwenPI-Flow policy.
  • Text versus visual heads: Text-side entropy is insufficient, while visual heads provide large advantages on OpenVLA Object Binding and QwenPI-Flow Goal Semantics/R.The comparison retains text-head Top-1 counterparts and uses MAE-D for OpenVLA and MAE-C for QwenPI-Flow.
  • Head selection: Across all three models, Top-1 head selection is the most reliable default when considering both AUROC and AUPR.Additional heads can dilute the oriented entropy signal, while half-head settings are less consistent and require more aggregation.
  • Layer bands: No restricted layer band dominates across task families, whereas all-layer scoring gives the best overall AUROC.Shallow, middle, or deep bands can preserve parts of the signal, especially where object grounding is localized, but all-layer scoring avoids architecture- or task-specific tuning.

E Evaluation Protocol

The evaluation protocol assigns each LIBERO-Reflect episode a label from simulator success and compares self-evaluation scores by ranking performance. It also specifies MAE’s internal-step correspondence, layer-band ablations, and a uniform random ranking baseline.

  • Episode-level evaluation: Each policy executes a LIBERO-Reflect episode, whose simulator success flag supplies the binary label while the evaluator assigns a label-independent scalar reliability score.Methods are compared under the same episode-level protocol.
  • Episode-level evaluation: Self-evaluation quality is measured by whether scores rank successful executions above failures using AUROC, AUPR, and FPR@95.For MAE, attention maps come from the same policy forward passes that generate the robot actions.
  • Implementation correspondence: The transformer layer index is ℓ, k denotes the internal action-generation step, and Table 11 maps k to MAE’s final internal step K for each model.This correspondence is model-specific.
  • Layer-band analysis: The main protocol aggregates visual attention entropy across all layers, while layer-band ablations recompute MAE-D or MAE-C within shallow, middle, and deep regions.The ablation tests whether reliability is concentrated at one depth or benefits from integrating the full action-generation process.
  • Baselines: The random baseline assigns every episode an independent uniform score under the same success and failure labels, serving only as an untuned ranking lower bound.It is not tuned per subset or model.

F Baseline Protocols and Applicability · G Model Input Templates

The appendix defines baseline protocols according to each VLA’s action interface, limiting token-statistic methods to policies exposing discrete action-token distributions. It also specifies the model input templates used to condition the evaluated backbones.

  • F Baseline Protocols and Applicability: Random assigns an i.i.d. uniform episode score and provides a lower-bound ranking baseline under success and failure labels.It is evaluated using the same episode labels as the other reliability methods.
  • F Baseline Protocols and Applicability: Verbal Confidence uses gpt-4.1 as an external multimodal evaluator, assigning each episode a scalar confidence from a stitched contact sheet and task instruction.The scalar confidence lies in [0, 1] and serves directly as the episode-level reliability score.
  • F Baseline Protocols and Applicability: Self-Consistency estimates reliability from sampled action-token agreement and is defined only for discrete OpenVLA.The score averages modal token frequencies across action-token dimensions and execution steps, while executing the first sampled action.
  • F Baseline Protocols and Applicability: Maximum Softmax Probability, Perplexity, Entropy, and Length-normalized Entropy use token-level categorical distributions and therefore are not directly comparable for continuous or flow-based action heads.Reported scores are direction-adjusted so larger values indicate higher estimated reliability; Length-normalized Entropy uses n = 5 candidates.
  • F Baseline Protocols and Applicability: Discrete OpenVLA supports token-probability baselines, whereas continuous OpenVLA-OFT and QwenPI-Flow do not expose comparable categorical action-token sequences.The incompatibility arises from OpenVLA-OFT’s continuous regression head and QwenPI-Flow’s flow-matching action trajectories.
  • G Model Input Templates: OpenVLA-family policies use a shared action-query prompt with lower-cased LIBERO instructions, while OpenVLA-OFT additionally receives proprioceptive State Input.These templates reproduce the policy-conditioning inputs for the evaluated backbones.
  • G Model Input Templates: QwenPI-Flow uses a Qwen3-VL-style multimodal message and appends the grounding text expected by the StarVLA policy.Table 12 records these prompt forms and the non-text inputs required for reproducibility.

H Dataset-Source Considerations

LIBERO-Reflect is designed as an episode-level reliability test that reduces source-level shortcuts by matching standard and challenging pools across construction and evaluation. MAE is interpreted as ranking reliability over realized executions, using internal attention dynamics rather than nominal dataset origin.

  • Benchmark construction: LIBERO-Reflect matches nominal pools by suite organization, task count, initializations, simulator, policy interface, and rollout protocol to reduce source-level shortcuts.The design targets differences in evaluation code, control horizon, observation, and related protocol factors.
  • Source-shortcut analysis: A source-only rule assigning uniform reliability by dataset origin is penalized when nominal source and realized outcome disagree and misses within-source variation.Episodes from the same dataset origin can have different execution outcomes, which source-level assignment cannot capture.
  • Source-shortcut analysis: MAE is computed from the policy’s internal attention dynamics during execution, supporting episode-level reliability estimation beyond dataset source.This contrasts with assigning reliability uniformly to standard or challenging episodes based only on nominal origin.
  • Evaluation protocol: LIBERO-PRO supplies challenging rollouts to increase failure density for reliability ranking, while simulator success remains the evaluation label.The benchmark therefore evaluates realized execution outcomes rather than treating source membership as the target.
  • Evaluation interpretation: The benchmark is an episode-level reliability test under a matched mixed-difficulty pool, with diagnostics reporting remaining source-level differences.Metrics are interpreted as reliability ranking over realized executions, not source-invariant classification.

I Architecture-Determined Entropy Orientation

MAE-D and MAE-C use opposite entropy orientations determined by each policy’s action-generation interface, not by episode-level data fitting. The rule distinguishes latent-readout policies from latent-refinement policies and uses fixed aggregation procedures for evaluation.

  • Architecture-Determined Entropy Orientation: MAE-D and MAE-C orient entropy according to whether the policy reads out a final latent state or iteratively refines an explicit action representation.This distinction follows the visual-attention role in the conditional generative Markov-chain transition kernel.
  • Latent-Readout VLAs: Latent-Readout VLAs require progressive consolidation of task-relevant visual evidence into a latent state before executable-action readout.Reliable transitions tend to concentrate action-query attention on the relevant object, region, or spatial relation.
  • Latent-Refinement VLAs: Latent-Refinement VLAs repeatedly update an action or trajectory variable under visual-language conditioning, with MAE evaluated at the final internal step k = K.The final-step attention reflects visual addressing during flow- or denoising-style refinement before the trajectory is returned.
  • Architecture-Determined Entropy Orientation: The orientation assignment is architecture-level and can be determined from the model’s inference computation graph alone, without data fitting.Latent-Readout policies lack iterative refinement of an explicit action variable, whereas Latent-Refinement policies update one across multiple refinement or denoising steps.
  • Aggregation Protocol: Top-1 is a deterministic per-episode reduction over oriented entropy values, while all-layer aggregation is the default frozen scoring rule; Top-m and layer-band variants are sensitivity analyses.These procedures avoid selecting attention heads or layers using test labels or task-specific information.

J Cost Analysis Details

MAE’s evaluation overhead is bounded because it reuses attention matrices from the same forward pass, adding only entropy computation and head aggregation. Table 13 profiles its average per-episode latency and extra evaluation time.

  • Cost Analysis Details: MAE’s bounded overhead comes from reusing internal attention matrices from the same forward pass and computing entropy plus head aggregation.The efficiency analysis profiles average execution time per episode and the extra latency incurred during evaluation.
Loading 2608.16697v1…