PaperlayerView exact revision on arXiv

Source-linked AI summary

$π\mathbf{R}^2$: Reactive Real-time Flow Policies

Sungjae Park, Shubham Tulsiani

arXiv:2607.26055v1cs.ROcs.AIcs.LG

TL;DR

Action-chunking flow policies struggle to react promptly because open-loop execution and slow replanning leave actions stale. πR2 separates fast proprioception from asynchronously updated vision-language features and uses latency-adaptive one-step flow inference, achieving roughly 4× faster closed-loop replanning and up to 30% higher real-world success than the strongest baseline.

  • Problem

    Open-loop action chunks and high perception-to-action latency limit the reactivity of large-backbone flow policies for dynamic manipulation.

  • Method

    πR2 combines asynchronous fast proprioceptive conditioning with slower vision-language features and a latency-adaptive per-position flow schedule for reactive inference.

  • Results

    Up to 23% higher simulation success and 30% higher real-world success were achieved over the strongest baseline, with roughly 4× faster closed-loop replanning at 25 Hz.

  • Takeaways & Limitations

    πR2 enables large-backbone policies to perform reactive, closed-loop manipulation with fresh observations every 40 ms while retaining multi-action prediction.

  • Takeaways & Limitations

    πR2 does not address latency external to the model, such as communication delays between the inference server and robot client.

Abstract

from arXiv · show

Generalist manipulation policies increasingly take the form of action-chunking flow policies built on large pretrained backbones. Such chunks run open-loop, so the policy cannot react to sensory input arriving mid-execution, sacrificing \emph{reactivity}. Replanning more often would restore it, but the perception-to-action pipeline (a large backbone plus multiple denoising steps) is too slow: this \emph{latency} forbids frequent replanning and leaves committed actions stale, making such policies ill-suited for dynamic, closed-loop control. We present $π\mathbf{R}^2$, which makes these policies reactive and real-time while retaining large backbones, expressive multi-modal policies, and multi-action prediction. Built on the per-position noise schedule of diffusion forcing, $π\mathbf{R}^2$ contributes two ideas. First, it splits conditioning into a fast channel (proprioception, fresh every tick) and an asynchronously updated slow channel (vision-language features), so the policy reacts to proprioception within a chunk while tolerating stale vision. Second, a latency-adaptive flow schedule treats in-flight actions as inpainting conditioning and emits actions in one denoising step per call, letting one trained model adapt to varying hardware latency. Requiring minimal modification to existing architectures, $π\mathbf{R}^2$ can be finetuned from a pretrained policy: applied to GR00T-N1.7 on a real xArm6+XHand platform, it replans closed-loop roughly $4\times$ faster than the base policy (~$25$Hz on an A5000 GPU), acting on a fresh observation every $40$ms. Across simulation and real-world manipulation tasks, $π\mathbf{R}^2$ improves the success rate by up to $23\%$ in simulation and $30\%$ in the real world over the strongest baseline. Project page: https://pi-r2-flow.github.io/

1 Introduction

πR2 addresses the reduced reactivity and increased latency of action-chunking flow policies by separating fast proprioceptive conditioning from slower vision-language processing and using a latency-adaptive diffusion-forcing schedule. On GR00T N1.7, it enables closed-loop replanning at 25 Hz, roughly 4× faster than the base policy, with fresh observations every 40 ms.

  • Motivation: Action chunks execute open-loop while perception-to-action inference processes large backbones and multiple denoising iterations, limiting reactivity and increasing latency.The resulting actions depend on stale sensory inputs, making these policies ill-suited to dynamic manipulation.
  • Approach: πR2 preserves large pretrained backbones, expressive multimodal policies, and multi-action prediction while enabling reactive real-time inference.Its framework leverages the flexible per-position schedule of diffusion forcing rather than mapping uniform noise to action chunks over multiple denoising iterations.
  • Approach: πR2 separates conditioning into a fast proprioception channel and a slower asynchronously processed vision-language feature channel.Joint positions, velocities, torques, and contact forces can be updated rapidly, while backbone features need not be processed at every tick.
  • Approach: A delay-adaptive noise schedule lets πR2 continue executing previous actions during latency while producing smooth, temporally coherent outputs from subsequent denoising iterations.The schedule incorporates in-flight actions and adapts execution to latency.
  • Results: 25 Hz on A5000 GPUs enables fresh observations every 40 ms, making πR2 roughly 4× faster than the base policy for closed-loop replanning.The comparison is against sparse replanning at approximately 7 Hz, while using the large-scale VLA GR00T N1.7.

2 Related Work

Prior work established action chunking and flow-based VLA policies, but open-loop execution limits reactivity in dynamic manipulation. πR^2 builds on diffusion forcing with asynchronous conditioning and latency-adaptive scheduling to make large-backbone flow policies real-time and closed-loop.

  • Action Chunking and Flow Policies: Action chunking improves temporal coherence and is standard in expressive diffusion and flow-matching manipulation policies, but committed sub-chunks cannot react to new observations.Chunked policies jointly generate H future actions and execute the first h ≤ H before replanning, leaving those actions open-loop.
  • Generalist Manipulation Policies: Vision-language-action models pair large vision-language backbones with chunked flow-matching heads, conditioning actions on proprioception across diverse tasks and embodiments.Examples include π0/π0.5 and GR00T N1, whose vision-language modules feed flow-matching action heads.
  • Diffusion Forcing: Diffusion forcing assigns independent noise levels to chunk positions, enabling successive denoising steps to incorporate progressively newer observations within one action chunk.Streaming diffusion uses a linearly increasing schedule so leading positions are denoised early and each step incorporates a fresh observation.
  • Reactive Conditioning: πR^2 separates conditioning into a slow vision-language channel and a fast proprioception channel, allowing fresh state reactions every tick while asynchronously refreshing cached visual features.Training supplies a learned embedding for bounded vision-language delay, enabling proprioception-reactive control despite stale slow-channel features.
  • Latency-Adaptive Scheduling: A latency-adaptive staircase schedule treats d in-flight actions as clean inpainting conditioning, uses a ramped interior, and appends d pure-noise tail slots for single-step emission.The schedule addresses variable inference delay from model components, GPUs, and network latency, whereas naive streaming diffusion assumes d = 0.

4 Experiments

Experiments show that πR² preserves reactive performance with one denoising step per call and improves performance under realistic inference latency. In simulation and real-world dexterous manipulation, asynchronous processing reduces effective delay and yields substantial gains over flow-based baselines.

  • 4.1.2 Simulation: Deployment latency: πR² with asynchronous processing achieves success rates of 0.43, 0.42, and 0.45 across increasing delays, outperforming naive-async at 0.33, 0.29, and 0.22 and Train-time RTC at 0.36, 0.32, and 0.19.Its effective action delay remains d=1 with asynchronous processing, while baselines pay d=⌈1.75d0⌉ per call.
  • 4.2 Real-world experiments: πR² operates at d=1 at 25 Hz, with occasional d=2 under network delays, whereas all flow-based baselines incur d=4 or 5.The real-world study uses 25-Hz control ticks, with each tick lasting approximately 40 ms.
  • 4.2 Real-world experiments: πR² improves success across metrics, with approximately 20–30% absolute gains on Tidy Up Book, Insert Box, and Catch Book.The evaluated tasks require dynamic grasping, precise contact interactions, proprioceptive feedback, or rapid reactions to falling objects.
  • 4.3 Reactivity analysis: πR² modulates grip from live fingertip-force feedback and stops near 50 N, while Train-Time RTC continues pushing and reaches approximately 120 N.The analysis on Tidy Up Book links fresh proprioception to timely force adjustment and shows how stale in-flight actions cause overshoot.

5 Discussion

πR2 modifies existing VLA architectures to enable real-time, closed-loop policies through asynchronous vision/text processing and a latency-adaptive flow schedule. Together, these contributions reduce per-call delay by up to 4× while remaining compatible with any VLA action head.

  • Discussion: πR2 combines asynchronous vision/text processing with a latency-adaptive flow schedule to produce real-time, closed-loop policies.The two contributions decouple the slow VLM backbone from the action loop and absorb variable inference delay through a per-position noise schedule parameterized by d.
  • Discussion: Asynchronous vision/text processing decouples the slow VLM backbone from the action loop.This is one of πR2’s two orthogonal contributions.
  • Discussion: Up to 4× lower per-call delay is achieved while retaining compatibility with any VLA action head.The latency-adaptive flow schedule absorbs variable inference delay via a per-position noise schedule parameterized by d.

Limitations

The approach leaves communication latency outside the model unaddressed and retains the base architecture, leaving stronger proprioceptive emphasis for future work.

  • Limitations: The approach does not address external latency sources, including communication delays between the inference server and robot client.These delays are outside the model itself.
  • Limitations: The base policy architecture remains unchanged, while dedicated attention heads for proprioceptive tokens could further amplify reactivity.The authors leave this architectural direction to future work.

A Implementation Details · A.1 Simulation

The simulation evaluates reactive cube reorientation with a 16-DoF Leap Hand at 50 Hz, using noisy proprioceptive and vision-derived state inputs. Policies predict 16-action chunks with a Conditional U-Net, while πR2 uses per-position conditioning and one denoising step per policy call.

  • A.1 Simulation: 50 Hz control and four target yaw poses define the modified Leap Cube Reorientation task.The cube must reach one of 0, π/2, π, or 3π/2 yaw poses within 0.2 rad in 600 steps.
  • A.1 Simulation: 200 trajectories from four PPO experts train the policies under independently injected observation noise.Noise affects joint angles, cube position, and cube orientation quaternion, with quaternion re-normalization after perturbation.
  • A.1 Simulation: 32-dimensional proprioception combines noisy joint angles with tracking errors, while 9-dimensional vision-derived state represents the slow channel.The vision-derived subset contains palm-to-cube position error and absolute cube orientation; evaluation can apply staleness only to this subset.
  • A.1 Simulation: 16-dimensional relative joint commands are issued at 50 Hz and scaled by 0.5 before being added to the current motor target.Each action lies in [−1, 1]^16.
  • A.1 Simulation: H=16 action chunks use a Conditional U-Net 1D head, with πR2 replacing shared FiLM projections by one modulation pair per chunk position.All variants share the rest of the architecture, including noise-level modulation at every residual block.
  • A.1 Simulation: 0.2 probability standard-flow mixing lets the same πR2 network initialize an action buffer from pure noise at episode start.The remaining 0.8 of batches use the three-region per-position staircase schedule with front actions clamped and masked from the loss.
  • A.1 Simulation: A single πR2 checkpoint serves all simulated slow-channel delays, with evaluation supplying dvis-step-old versions of the 9-dimensional subset.The simulation uses state-based inputs rather than image or language observations, and sensor noise makes additional delayed-state training unnecessary.
  • A.1 Simulation: 1 denoising step per policy call enables πR2 inference, whereas train-time RTC and standard flow use K=15 steps for whole-chunk prediction.Each method-and-delay cell reports mean success over 100 episodes.

A.2 Real-World

The real-world evaluation uses an xArm6–XHand platform with overhead RGB sensing and deploys πR2 at 25 Hz using separate GPUs for asynchronous vision and fast action inference. Policies emit 2-second action chunks, are fine-tuned from GR00T-N1.7, and are evaluated over randomized trials using task success and progress metrics.

  • Hardware and Workspace: πR2 deploys on a 6-DoF xArm6 carrying a 12-DoF XHand, with one overhead 640 × 480 RGB camera and two RTX A5000 GPUs.The separate GPUs prevent the slow VLM and fast DiT action head from sharing compute and memory bandwidth.
  • Control rate: 25 Hz control executes one action target every 40 ms, while teleoperation records demonstrations at the same rate.The rate is bottlenecked by the camera and the XHand–workstation USB link.
  • Base model: The real-world policy is fine-tuned from GR00T-N1.7 while freezing the Qwen-VL backbone and Eagle-2 vision encoder.Training updates the action head, and the architectural change replaces shared AdaLN modulation with per-position (γp, βp) heads.
  • Action space and low-level drivers: Each policy call emits H=50 absolute joint-position targets for the 18 robot joints, spanning 2 s of motion.One target is sent per control tick, and onboard firmware interpolates between consecutive commands.

A.3 Training and Inference Algorithms

The training algorithm samples delays, constructs a jittered staircase schedule, masks front actions as inpainting conditioning, and optionally simulates slow-channel staleness. At deployment, asynchronous workers continuously update cached vision-language features and action chunks while the action head uses fresh proprioception and latency-adaptive one-step updates.

  • Training: Training samples a delay d, builds the three-region staircase τ⋆,d, masks the front d slots, and applies timing jitter.The front slots are replaced with ground-truth actions and excluded from the loss.
  • Training: Real-world training additionally samples slow-channel staleness and appends its delay embedding to the slow representation.The slow feature is selected from an earlier time t−dvis.
  • Inference: Deployment runs a 25 Hz control loop with continuously refreshed proprioception, asynchronous vision-language updates, and action-head queries in separate workers.The VLM worker forwards fresh images and language, then atomically updates the cached slow feature.
  • Inference: Each new action chunk swaps into the loop while skipping already in-flight front actions, then sends the next action to the robot.The loop increments the chunk index after swapping and advances it after every action dispatch.
  • Inference: The action worker auto-derives delay d from rolling query latency and performs one Euler denoising step per call using in-flight actions as inpainting conditioning.It updates the rolling latency window after each query and hands the resulting chunk back to the main loop.

A.4 Additional Reactivity Analysis

Across the remaining tasks, πR2 reacts promptly to force feedback and maintains controlled manipulation, while Train-Time RTC reacts too late, causing slips, force overshoot, or missed contact.

  • Additional Reactivity Analysis: On Catch Book, πR2 closes on the force spike as the book lands, whereas Train-Time RTC reacts too late and the book slips.Figure 8 compares fingertip force and emitted actions over time, with overhead frames at marked times.
  • Additional Reactivity Analysis: On Insert Box and Don’t Spill, πR2 stays controlled and grasps with both fingers, while Train-Time RTC overshoots force and continues without thumb contact.RTC’s index force overshoots out of distribution on Insert Box; on Don’t Spill, thumb force is approximately 0.
Loading 2607.26055v1…