PaperlayerView exact revision on arXiv

Source-linked AI summary

Prism-GRPO: Faster VLA Policy Optimization via Splitting Same-outcome Groups

Zeyun Deng, Yuzhe Lu, Yawei Wang, Linbo Liu, Qing Ping, Han Ding, Guande Wu, Panpan Xu, Jun Huan

arXiv:2608.17423v1cs.ROcs.LG

TL;DR

Binary-reward GRPO discards same-outcome rollout groups, wasting robotic sampling. Prism-GRPO adds trajectory-level execution quality to recover their training signal, reaching target success rates with up to 56% fewer rollouts across four RoboTwin tasks.

  • Problem

    Binary-reward GRPO discards all-success and all-failure groups because they provide no advantage spread, wasting collected rollouts.

  • Method

    Prism-GRPO augments binary outcome rewards with bounded trajectory-level execution quality to split same-outcome groups into a quality spectrum.

  • Results

    Up to 56% fewer rollouts were needed to reach target success rates across four RoboTwin tasks while execution quality improved.

  • Takeaways & Limitations

    Prism-GRPO improves rollout efficiency and execution quality across contact-, smoothness-, and VLM-derived quality signals.

  • Takeaways & Limitations

    Prism-GRPO requires an observable trajectory signal aligned with success, and verifying population-level alignment in full VLA models remains challenging.

Abstract

from arXiv · show

GRPO is increasingly used for reinforcement learning of vision-language-action (VLA) policies because, unlike PPO, it does not require training a critic. This simplification comes with a sampling cost: group-relative advantages require multiple rollouts from each scene. Under binary success rewards, groups whose rollouts all succeed or all fail have zero advantage and are discarded by dynamic sampling. These groups are especially common early in training, when most rollouts fail, wasting much of the expensive robotic rollout budget. We introduce Prism-GRPO, which augments binary outcome reward with a weighted trajectory-level execution-quality score. By splitting same-outcome groups into a quality spectrum, Prism-GRPO recovers training signal while ensuring that every success still outranks every failure. Quality scores can be derived from simulator contacts, executed actions, or visual observations, avoiding task-specific progress rewards. We prove that Prism-GRPO never increases the probability that a sampled group is discarded for having zero advantages, and derive a gradient-alignment condition under which its combined update remains a local ascent direction for task success. Across four RoboTwin tasks spanning different horizons and coordination patterns, Prism-GRPO improves success and quality at matched rollout budgets and reaches target success rates with up to 56% fewer rollouts. It also suppresses a reward-hacking shortcut, with the cleaner behavior transferring under direct deployment to a real robot. Through ablations, we show consistent gains across contact-, smoothness-, and VLM-derived quality signals.

1 Introduction

Prism-GRPO addresses GRPO’s sample inefficiency on VLA policies by using trajectory-level execution quality to recover signal from same-outcome groups while preserving success-over-failure ordering. It provides theoretical guarantees and improves rollout efficiency and execution quality across four RoboTwin tasks.

  • Problem: Binary-reward GRPO discards all-success or all-failure groups because identical rewards produce zero advantage and no gradient.This makes group-relative training sample-inefficient, especially when same-outcome groups are common.
  • Method: Prism-GRPO splits same-outcome rollouts using bounded trajectory quality while combining it with binary task success.Quality can reflect execution properties such as unintended contact, object disturbance, or motion smoothness, rather than task progress.
  • Theory: Prism-GRPO never increases the expected rollouts needed to obtain an informative group, and aligned gradients make its combined update a first-order ascent direction for both objectives.The guarantee depends on success–quality gradient alignment for the ascent result.
  • Results: Up to 56% fewer rollouts are needed to reach target success rates across four RoboTwin tasks, while execution quality improves.The gains hold across contact- and smoothness-based quality signals in the stated introduction.

2 Related Work

Prior work spans vision-language-action policy representations, critic-free reinforcement learning, graded reward design, and methods for reviving zero-variance groups. Prism-GRPO is positioned within these lines while addressing gradient conflict in multi-objective policy updates.

  • Vision-language-action models: VLA models map visual observations and language instructions to robot actions using pretrained vision-language backbones.Action representations differ; OpenVLA discretizes each degree of freedom into bins and emits one token per degree of freedom.
  • Reinforcement learning for VLA policies: GRPO removes PPO’s separate value network by scoring same-input trajectory groups relative to their mean reward, but applying it to VLA policies is challenging.The passage identifies PPO’s critic-based advantage estimation and GRPO’s critic-free alternative as the relevant foundations.
  • Enriching the binary reward: Prior methods enrich binary success rewards with learned vision-language rewards, task-specific stage-level process rewards, or other graded feedback.SimpleVLA-RL’s binary reward cannot distinguish trajectories within the success or failure outcome classes.
  • Reviving degenerate groups: RL-ZVP revives zero-variance groups using entropy from the policy’s own output distribution, while EDGE-GRPO adds guided error correction.These methods treat same-outcome groups as potentially useful training data despite their lack of advantage spread.
  • Gradient conflict in multi-objective learning: Multi-objective learning addresses opposing policy gradients through gating, projection, learned weighting, or strict priorities, whereas this work targets manipulation settings with higher-quality behavior.The cited methods generally assume gradient conflict between objectives.

3 Preliminary

This section formulates VLA control as an MDP and describes GRPO’s group-based optimization without a learned value function. It defines degenerate groups, success–quality gradient alignment, and a compositional reward procedure that retries until sampled outcomes are informative.

  • MDP and GRPO formulation: VLA control is modeled as an MDP in which a policy observes image-and-instruction states, takes robot-command actions, and maximizes expected trajectory return.The formulation uses states S, actions A, transition dynamics P, reward R, and initial-state distribution ρ0.
  • MDP and GRPO formulation: GRPO optimizes the expected-return objective without training a learned value function by sampling multiple trajectories for each scene.The supplied formulation introduces group sampling as the basis for relative advantages.
  • Degenerate groups: A group is degenerate when all rollout rewards are equal, producing zero advantages and no gradient; otherwise, it is informative.Formally, degeneracy occurs when R1 = ··· = RG.
  • Objective gradients and alignment: Success and quality objectives are aligned when their parameter-space gradients have a nonnegative inner product, and conflict otherwise.The objectives are J(θ) = Ex,τ[success(τ)] and Q(θ) = Ex,τ[q(τ)].
  • Compositional reward procedure: The compositional procedure assigns each trajectory Ri = success(τi) + λ q(τi) and retries sampling until the group is not uniformly rewarded.Each retry discards G rollouts when all Ri are equal, after which advantages are computed relative to the group.

4 Prism-GRPO

Prism-GRPO augments binary outcomes with trajectory-level quality to split otherwise discarded same-outcome groups while preserving success dominance. Its analysis shows improved rollout efficiency and quality-preserving success updates under measurable alignment conditions.

  • Quality-augmented reward: Prism-GRPO adds a continuous trajectory-quality score q(τ) to binary success rewards, with quality measuring execution properties such as smoothness.A GRPO group contains G trajectories generated on the same scene, each scored by success(τ) ∈ {0, 1} and q(τ) ∈ [0, 1].
  • Quality-augmented reward: With λ < 1, quality refines failures within [0, λ] and successes within [1, 1 + λ], so every success still outranks every failure.The quality term breaks ties in all-failure and all-success groups while only reordering trajectories within each outcome class.
  • Alignment analysis: Task-level correlation remains informative in degenerate groups because between-scene covariance can preserve the overall quality–success relationship when within-scene covariance vanishes.Saturated scenes can have zero within-scene covariance, while easier scenes may simultaneously exhibit higher quality and higher success rates.
  • Gradient alignment: Under ⟨gsuccess, gquality⟩ ≥ 0, the combined direction gcombined = gsuccess + λgquality is a first-order ascent direction for both success and quality.The directional derivatives satisfy ⟨∇θJ, gcombined⟩ ≥ ∥gsuccess∥2 and ⟨∇θQ, gcombined⟩ ≥ λ∥gquality∥2.
  • Sample efficiency: 1.76× and 2.91×: at p = 0.1 or 0.9, Binary GRPO requires approximately 1.76× as many rollouts for G = 8 and 2.91× for G = 4.For continuous conditional quality scores, exact ties occur with probability zero, yielding Ccombined(p) = G and making every group informative.

5 Experiments

Across RoboTwin 2.0 experiments, Prism-GRPO improves rollout efficiency and execution quality over Binary GRPO while avoiding degenerate-group waste. Gains persist across quality sources and suppress shortcut behaviors in simulation and real-robot deployment.

  • Benchmark and training setup: Experiments use RoboTwin 2.0’s randomized bimanual manipulation scenes across four tasks, with all rollouts in each GRPO group sampled from the same scene.Group degeneracy is therefore determined per scene, while training aggregates updates across scenes.
  • Quality signal: GT-Max Force is the default training quality signal, measuring the largest force applied to any non-target object during a rollout.It is used because unintended contacts are common, easy to measure, and can disrupt later execution.
  • Main results: 22–56% rollout savings over Binary GRPO are achieved at matched rollout budgets, with calibrated success following the same trend across four tasks.Prism-GRPO also shows higher execution quality at matched target success and improves Sum-Impulse Quality despite training with Max-Force Quality.
  • Main results: 70% of Lift Pot’s earliest rollout batches are discarded by Binary GRPO, while Prism-GRPO keeps early discard rates essentially at 0%.Binary GRPO later stabilizes at 20% discarded batches, whereas Prism-GRPO remains around 14%.
  • Reward-shortcut suppression: 0.6–1.3% is the shove-cheat range across Prism-GRPO seeds, compared with 0.6–20.0% for RL-ZVP and 0.7–7.0% for Binary GRPO.The highest RL-ZVP rate is over 15× that of Prism-GRPO, while Prism-GRPO maintains clean lift-and-place behavior under the imperfect geometric checker.
  • Quality-source generalization: 38–56% rollout savings are obtained with collision-quality signals, and 25–44% with motion-quality signals, including simulator- and VLM-derived sources.Adding quality costs only 0.04% wall-clock time for simulator signals and 4.8% for VLM signals; rollout generation remains dominant.
  • Sim-to-real reliability: Prism-GRPO produces cleaner sim-to-real behavior: RL-ZVP cheats most often, Binary GRPO occasionally, and Prism-GRPO not at all while maintaining comparable clean success.Low-quality simulator successes can fail on hardware because real contact dynamics and safety-limited actuation produce less displacement than simulation.

6 Discussion and Conclusion … A.2 Proof of Proposition 2

Prism-GRPO recovers training signal from same-outcome groups while preserving success ordering, reducing or avoiding wasted rollout sampling under suitable trajectory-quality signals. Its theoretical results establish no greater informative-group cost and a success-quality gradient improvement under alignment, with limitations when aligned quality signals are unavailable.

  • 6 Discussion and Conclusion: Prism-GRPO is positioned as the first VLA RL method to recover training signal from rollouts lost to same-outcome groups.Related methods target deployment, rollout-generation, reward-model, motivation, scalable-training, or guided-exploration efficiency instead.
  • 6 Discussion and Conclusion: Prism-GRPO requires an observable trajectory signal aligned with success, available through contacts, actions, or vision in robotics but potentially requiring proxies or external verifiers elsewhere.The conclusion identifies this observable-signal requirement as a limitation.
  • A.1 Proof of Theorem 1: Under binary rewards, a group is degenerate exactly when all G trajectories succeed or all G fail, with informative probability 1 −p^G −(1 −p)^G.Dynamic sampling draws fresh groups until the first informative group, making the number of groups geometric.
  • A.1 Proof of Theorem 1: With 0 < λ < 1, success and failure reward ranges are disjoint, so mixed-outcome groups cannot have equal combined rewards; degeneracy requires a shared outcome and coincident quality scores.The combined reward is R_combined = success + λq.
  • A.1 Proof of Theorem 1: C_combined(p) ≤C_binary(p), because quality tie-breaking lowers degeneracy probability and increases the informative fraction under dynamic sampling.The proof bounds each combined-reward degeneracy term by its binary counterpart.
  • A.1 Proof of Theorem 1: As p →1 or p →0, C_binary(p) → +∞, whereas C_combined remains finite when t_succ, t_fail < 1.The combined-cost limits are G/(1 −t_succ) as p →1 and G/(1 −t_fail) as p →0.
  • A.2 Proof of Proposition 2: Under the alignment hypothesis, the combined update improves both objectives when their corresponding gradients are nonzero.The proof gives lower bounds of ∥g_success∥^2 for the success derivative and λ∥g_quality∥^2 for the quality derivative.

A.3 Proof of Proposition 1 · B Quality · B.1 Quality Signals

Proposition 1 establishes a gradient-alignment condition linking task-success and quality updates through policy-weighted correlation and conditioning. Prism-GRPO defines bounded trajectory-level quality and implements it with fixed success-dominant rewards plus collision- and smoothness-based signals, including visual collision judgments when simulator contacts are unavailable.

  • A.3 Proof of Proposition 1: For a single softmax decision, success and quality logit gradients are gS = Dũ and gQ = Dṽ, with policy-weighted correlation represented as the Euclidean cosine of transformed centered values.The transformed vectors lie in the centered subspace Hπ, and the gradient cosine equals the Aπ-metric cosine.
  • A.3 Proof of Proposition 1: If ρ(s) ≥ (κπ −1)/(κπ + 1), the combined quality update has nonnegative alignment with the success gradient under the stated non-degeneracy assumptions.The condition follows from the metric-cosine bound; bounded conditioning is necessary because the bound can approach −1 as κπ grows.
  • B Quality: Prism-GRPO treats quality as a bounded trajectory-level score q(τ) ∈[0, 1], while all variants retain the same success-dominant reward and RLOO estimator.Only the quality measurement changes across variants, isolating the effect of the quality source while keeping policy, reward, and optimization fixed.
  • B.1 Quality Signals: Collision signals exclude required contact with the manipulated target and score robot contact with other objects as unintended non-target contact.For Lift Pot the target is the pot, while for Move Can Pot it is the can.
  • B.1 Quality Signals: Simulator logs provide Prism-Peak, the default GT-Max Force variant, for collision severity, and Prism-Count for collision frequency.Prism-Peak uses peak impulse, whereas Prism-Count weights every non-target contact equally regardless of magnitude.
  • B.1 Quality Signals: Prism-VLM-Contact uses eight temporally ordered frames and a zero-shot Qwen3-VL-235B judge to produce a binary collision-free quality signal from visual observations.The judge identifies the target, excludes intended manipulation, and inspects scene-layout changes and brief intermediate-frame contacts; evaluation used 256 trajectories.
  • B.1 Quality Signals: Smoothness signals use the twelve ALOHA arm-joint dimensions, excluding gripper dimensions, and measure velocity reversals or abrupt action changes without simulator contact data.Prism-Flips captures the most oscillatory joint, Prism-MeanFlips averages reversals across arm joints, and Prism-Jerk measures average third-order action-difference magnitude.

B.2 Metrics

Prism-GRPO evaluates trajectory quality using six collision- and smoothness-based signals, normalized to [0, 1] with task-calibrated thresholds. Calibration uses SFT trajectories to preserve ordinary variation while penalizing excessive contacts or motion.

  • Quality signals and normalization: Six signals cover collision severity and motion smoothness, including contact impulses or counts, VLM-estimated contacts, velocity reversals, and action jerk.Collision variants are Prism-Peak, Prism-Count, and Prism-VLM-Contact; smoothness variants are Prism-Flips, Prism-MeanFlips, and Prism-Jerk.
  • Quality signals and normalization: Each non-negative raw cost r(τ) is converted into a quality score in [0, 1], where larger values indicate cleaner or smoother execution.The threshold T sets the normalization scale, while r0 can remove motion inherently required by a task.
  • Quality signals and normalization: For Prism-Flips on Move Can Pot and Beat Block Hammer, r0 = 12 removes reversals naturally required by grasp-and-place and lift-and-strike motions.Collision signals always use r0 = 0.
  • Calibration from the SFT policy: Thresholds are calibrated from each task’s SFT policy on 256 validation scenes, then fixed for all subsequent reinforcement-learning runs.The calibration aims to retain useful variation in ordinary SFT trajectories while assigning low scores to clearly excessive collision or motion values.
  • Collision thresholds: Prism-Peak thresholds are 10 N s for Lift Pot, 6 N s for Move Can Pot, and 3 N s for both Handover Block and Beat Block Hammer.The thresholds separate a low-impulse region containing relatively gentle contacts from a high-impulse tail associated with severe unintended interactions.

C Experimental Setup · C.1 Tasks · D Advantage Calculation

The experiments use a shared SimpleVLA-RL initialization and controlled rollout configuration across four diverse RoboTwin tasks. Prism-GRPO combines binary success with quality-based rewards so leave-one-out advantages remain informative for same-outcome groups while preserving success-over-failure ordering.

  • C Experimental Setup: All methods start from the same released SimpleVLA-RL supervised fine-tuning checkpoint, isolating differences to the reinforcement-learning stage.This shared initialization applies to Prism-GRPO and every baseline for the corresponding task.
  • C Experimental Setup: 512 rollouts per step come from 64 scenes with group size G = 8, alongside a 5 × 10−6 learning rate and rollout temperature of 1.6.The configuration also uses constant warmup, gradient clipping, PPO clip bounds (0.2, 0.28), and no KL penalty or entropy bonus.
  • C Experimental Setup: Adaptive gap-fill generation targets 512 retained trajectories while estimating deficits from observed keep rates, adding headroom, and capping each refill at B scenes.All methods update on the same number of retained trajectories, while generated, filtered, and surplus trajectories count toward rollout usage.
  • C.1 Tasks: Four RoboTwin tasks span bimanual and one-arm coordination across lifting, pick-and-place, handover, and tool use, with randomized placements, clutter, lighting, and backgrounds.The tasks expose failures including unstable grasps, tipping, inaccurate placement, failed transfer, and reward shortcuts.
  • C.1 Tasks: Move Can Pot includes a shove-cheat failure in which the policy moves the pot toward a stationary can instead of grasping and moving the can.Success requires the can to be upright, released, and placed within the target region beside the pot.
  • D Advantage Calculation: Quality variation gives same-outcome groups nonzero leave-one-out advantages, recovering learning signal where binary rewards provide none without special implementation handling.The same estimator is applied to every group; unlike standard group-normalized GRPO, RLOO does not divide by within-group reward standard deviation.
  • D Advantage Calculation: λ = 0.2 defines the combined reward R(τ) = success(τ) + λq(τ), with q(τ) ∈[0, 1] distinguishing trajectories within outcome classes.The quality term lies in [0, λ], so every successful trajectory has reward at least 1 and every failed trajectory has reward at most λ.

E Baseline Implementation … G.1 Full Statistics

The paper implements baselines under a shared setup, varying only reward and advantage estimation, and evaluates how quality signals reduce dynamic-filter discards and rollout cost. Prism-GRPO recovers same-outcome groups for learning, while rollout savings translate nearly proportionally into compute savings.

  • E Baseline Implementation: All baselines share Appendix C’s setup and differ from Prism-GRPO only in reward and advantage estimation.
  • E Baseline Implementation: Binary GRPO uses success rewards with group-standardized advantages, discarding zero-variance groups through dynamic sampling.Binary RLOO keeps the same reward but replaces the estimator with a leave-one-out baseline, isolating the quality term’s contribution.
  • E Baseline Implementation: Random reward adds independent continuous noise to success, typically preventing zero-variance groups but providing no execution-quality information.RL-ZVP instead uses detached, trajectory-dependent policy entropy over action-chunk decisions and composes only with group-standardized GRPO.
  • G Detailed Experimental Results: Prism-GRPO consistently discards fewer groups and stays closer to the minimum cost of 512 rollouts per optimizer step across four tasks.The quality signal recovers same-outcome groups when their execution quality differs, allowing those groups to contribute to learning.
  • F Compute usage: Each task uses one node with 8× NVIDIA H100-80 GB GPUs, and a complete RL run takes approximately 12–16 hours.Rollout generation accounts for roughly 8–12 hours and dominates total wall time.
  • F Compute usage: 30% fewer rollouts than Binary GRPO’s 35,000 on Lift Pot avoids 10,500 rollouts and saves approximately 2.4 of 7.9 rollout-generation hours.Generation time averages approximately 0.81 s per rollout, so rollout reductions yield almost proportional wall-clock savings.
  • G.1 Full Statistics: Each optimizer update requires 64 retained groups with G = 8, or 512 retained trajectories, with surplus rollouts generated to refill discarded groups.Higher discard rates therefore increase the generation cost of each optimizer step.
  • G.1 Full Statistics: Random and RL-ZVP have zero discard rates because their independent random or trajectory-dependent confidence scores vary within every sampled group.Full collision and smoothness results are reported in Figures 15 and 16, with similar trends across the two collision metrics and consistent trends across smoothness metrics.

G.2 Quality Weight Ablation … H Empirical Analysis of Success–Quality Alignment

Ablations show that Prism-GRPO benefits from applying quality broadly, especially with smaller groups, while contact-based quality metrics yield consistent conclusions. RLOO provides more stable optimization than standard group normalization, and quality-weight decay offers limited gains.

  • G.2 Quality Weight Ablation: Success and calibrated success follow similar trends across λ ∈ {0.2, 0.5, 0.9, 1, 2} on Lift Pot and Move Can Pot.Max-Force Quality and Sum-Impulse Quality likewise exhibit consistent patterns; Max-Force Quality is the primary collision-quality metric.
  • G.3 Quality-Term Placement Ablation: Applying the quality term to every group nearly doubles success rate versus applying it only to same-outcome or all-failure groups.It also produces a more stable quality curve while preserving success dominance within mixed groups.
  • G.4 Effect of Group Size: The group-size ablation varies G ∈ {8, 16, 32} while fixing 512 generated rollouts per optimizer update and holding other training settings unchanged.G = 8 is the default setting in the main experiments.
  • G.4 Effect of Group Size: Prism-GRPO provides its largest improvement at G = 8, with smaller gains at G = 16 and nearly identical results at G = 32.Each optimizer update uses 512 generated rollouts, and larger groups make all-success and all-failure groups less frequent.
  • G.5 Effect of Quality-Weight Decay: Quality-weight decay does not improve performance for λ = 0.2 or 0.5, while λ = 0.9 reaches the target about 2K rollouts earlier with decay.The quality term is especially useful early in training when failures dominate and many groups are all-failure.
  • G.6 Effect of the Advantage Estimator: RLOO produces more stable learning and consistently achieves expected rollout-saving gains, whereas standard group-normalized GRPO is less stable with smaller, less consistent savings.RLOO preserves the quality-term scale in same-outcome groups, while standard group normalization largely cancels it.

H.1 Analysis Protocol … I.1 Deployment Setup

The paper evaluates quality–success alignment with within-scene and cross-scene diagnostics, finding robust associations across collision and smoothness signals. It also tests zero-shot real-robot deployment using camera preprocessing, chunked actions, and joint-rate limiting.

  • H.1 Analysis Protocol: The protocol combines within-scene quality gaps with cross-scene Spearman correlations to reduce confounding from scene difficulty.Within-scene comparisons hold simulator initialization fixed, while cross-scene analysis relates scene success rates to mean trajectory quality.
  • H.1 Analysis Protocol: Each task’s best-success checkpoint is evaluated on 256 randomized scenes with 56 stochastic rollouts per scene, producing 14,336 trajectories.All generated trajectories are included without dynamic filtering, and mixed-outcome scenes receive equal weight with bootstrap confidence intervals.
  • H.2 Collision-Based Alignment: Across collision statistics, success is strongly associated with cleaner execution both within identical scenes and across the task distribution.Max-Force Quality is the default signal, while Contact-Count Quality and Sum-Impulse Quality test robustness to the collision definition.
  • H.3 Smoothness-Based Alignment: 87%–100% of mixed-outcome scenes have positive within-scene quality gaps across all task–smoothness-signal combinations.Cross-scene correlations are consistently positive and strong, supporting the success–smoothness relationship across multiple definitions.
  • H.3 Smoothness-Based Alignment: Prism-Flips subtracts a data-driven reversal floor on Move Can Pot and Beat Block Hammer to measure excess oscillation beyond task-required reversals.The floor is estimated as the median reversal count among successful trajectories before normalization.
  • I Real-World Experiments: The policies are evaluated zero-shot on a Piper robot without additional real-world fine-tuning for moving a can next to a pot.An Orbbec DaBai DC1 camera captures 640 × 480 observations, resized to 240 × 320 before standard policy preprocessing.
  • I.1 Deployment Setup: The deployed policy predicts 25-step action chunks, with each action containing six arm-joint targets and one gripper target.A proportional rate limit constrains arm-joint motion while leaving the gripper dimension unrestricted.
  • I.1 Deployment Setup: Arm-joint changes are limited to 0.05 rad per control tick, scaling all six arm dimensions together when requested displacements exceed the limit.Joint scaling preserves the relative direction of commanded motion; the gripper is exempt.

I.2 Evaluation Protocol · I.3 Real-World Results

The real-world evaluation stress-tests whether simulated shove-cheating transfers to deployment, using matched-success checkpoints and 25 trials per method. Prism-GRPO records more clean successes than the baselines, while the evaluation emphasizes shove-cheat occurrence rather than treating the small success-count difference as conclusive.

  • I.2 Evaluation Protocol: I.2 Evaluation Protocol: Checkpoints were selected at approximately 65% simulated success, choosing each method’s highest simulated shove-cheat rate among similarly successful checkpoints.The protocol is designed to stress-test transfer of the simulated shortcut in which the policy moves the pot toward the can instead of lifting and placing the can.
  • I.2 Evaluation Protocol: I.2 Evaluation Protocol: Each policy was evaluated over 25 real-world trials.
  • I.3 Real-World Results: I.3 Real-World Results: Prism-GRPO achieved 6/25 clean successes, compared with 4/25 for Binary GRPO and 2/25 for RL-ZVP.
  • I.3 Real-World Results: I.3 Real-World Results: The Prism-GRPO versus Binary GRPO clean-success difference is not interpreted as conclusive because the counts differ by only two trials.
  • I.3 Real-World Results: I.3 Real-World Results: The evaluation emphasizes shove-cheat occurrence, defined as moving the can beside the pot instead of performing the instructed placement.
  • I.3 Real-World Results: I.3 Real-World Results: Collision-based diagnostics compare successful and failed trajectory quality within scenes and correlate scene success rate with mean quality across scenes.The reported quality signals are Max-Force, Contact-Count, and Sum-Impulse Quality; positive within-scene gaps indicate cleaner successful trajectories.
  • I.3 Real-World Results: I.3 Real-World Results: Smoothness-based diagnostics use maximum joint reversals, mean joint reversals, and action jerk with the same within-scene and cross-scene definitions.
Loading 2608.17423v1…