Source-linked AI summary
Learning Safe Flow Matching Robot Policies from Binary Feasibility Oracles
Jianming Ma, Rongjun Jin, Yizhi Chen, Yue Gao
TL;DR
Flow-based robot policies must satisfy physical constraints even when feasible sets are accessible only through binary queries rather than analytic models. The paper uses an oracle-guided safeguard for forward correction and boundary-aware backward learning, with the same intervention during training and inference. In obstacle-aware manipulation, it reports a 100% step safety rate while maintaining success close to the unconstrained baseline.
Problem
Method
An oracle-guided safeguard corrects unsafe updates using binary feasibility queries and supplies local boundary information for backward learning during both training and inference.
Source: §1 · pp. 1–2Results
100% step safety rate while maintaining a success rate close to the unconstrained baseline for π0.5 + safeguard.
Source: §4 · p. 4Takeaways & Limitations
Obstacle-aware manipulation experiments show improved safety while preserving task performance and motion quality.
Source: §5 · p. 4The discrete flow relies on simple interpolation paths that may become unreliable in highly non-convex feasible regions; evaluation across more diverse tasks, robots, and real-world constraints remains future work.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Robot foundation models increasingly rely on flow-based action generation for general-purpose manipulation, yet the generated actions are not guaranteed to satisfy physical constraints such as joint limits and collision avoidance. This problem is especially challenging when the feasible set cannot be described analytically and is accessible only through black-box feasibility oracles. We study this practical oracle-only setting and propose an oracle-guided differentiable safeguard for safe flow-matching robot policies. The safeguard uses binary feasibility queries to correct unsafe updates in the forward pass, while extracting local boundary information to provide constraint-aware learning signals in the backward pass. Combined with discrete-time flow matching, the same safeguard is used during training and inference, enabling forward--backward aligned policy learning without explicit or differentiable constraint models. Experiments on obstacle-aware robotic manipulation demonstrate 100% constraint satisfaction while preserving task performance.
1 Introduction
Flow-based robot policies must satisfy physical constraints, but scene-dependent feasible sets may be accessible only through binary feasibility queries. The paper proposes an oracle-guided safeguard that corrects updates and supplies boundary-aware learning signals, using the same intervention during training and inference.
1 Introduction
- The safeguard uses binary feasibility queries to restrict unsafe forward updates and local boundary information to construct constraint-aware surrogate gradients.It targets settings where explicit or differentiable constraint models are unavailable.Source: §1 · pp. 1–2
- Using the same safeguard during training and inference aligns policy learning with the safety intervention used at deployment.The framework is integrated with discrete-time flow matching.Source: §1 · p. 2
- The oracle-only formulation avoids requiring explicit or differentiable constraint models for safe flow-matching policy learning.This addresses robotics settings where a collision checker can judge feasibility but an analytical description of the collision-free set may be unavailable.Source: §1 · pp. 1–2
- On obstacle-aware robotic manipulation, the approach demonstrates strong constraint satisfaction while maintaining task performance.The introduction reports this result without giving specific numerical values.Source: §1 · p. 2
2 Preliminaries and Problem Formulation
The policy generates an H-step future joint-position action chunk through a discrete-time flow conditioned on the current observation and language instruction. Task-dependent safety is exposed only through a binary feasibility oracle, and safeguarding each update from a feasible initialization preserves feasibility through the final sample.
2 Preliminaries and Problem Formulation
- The policy predicts an H-step future joint-position trajectory conditioned on the current multimodal observation and language instruction.The trajectory is represented as an action chunk evolving through discrete flow steps.Source: §2 · p. 2
- Task-dependent constraints are available only through a binary feasibility oracle, which need not be differentiable or analytically describe the feasible set.The goal is for every configuration in the predicted action chunk to be feasible.Source: §2 · p. 2
- Starting from a feasible initial sample, safeguarding each discrete update preserves feasibility recursively and keeps the final generated sample feasible.The discrete flow uses linear interpolation between the initial sample and target action chunk.Source: §2 · p. 2
3 Oracle-Guided Differentiable Safeguard
The safeguard corrects each predicted joint configuration using binary feasibility queries, then supplies boundary-aware surrogate gradients for learning. It uses the same oracle for forward feasibility enforcement and backward estimation of local boundary geometry.
3 Oracle-Guided Differentiable Safeguard
- The safeguard is applied independently to each future-horizon element because the oracle evaluates individual joint configurations.The section omits the horizon index when describing a single element.Source: §3 · p. 2
- Forward correction searches along the nominal update ray for its first exit from the feasible region, then scales the update by the estimated exit factor.The ray search can use a coarse scan followed by local bisection.Source: §3 · p. 3
- For a truncated update, a boundary-aware surrogate Jacobian exposes the encountered constraint boundary to the action expert during training.For an untruncated update, the Jacobian reduces to I.Source: §3 · p. 3
- Symmetric binary feasibility queries near the detected boundary estimate its otherwise unavailable outward normal for the practical surrogate Jacobian.The same oracle thus supports both forward feasibility enforcement and backward estimation of local boundary geometry.Source: §3 · p. 3
4 Experimental Results
In the obstacle-aware pick-and-place evaluation, π0.5 + safeguard achieves a 100% step safety rate while maintaining success close to the unconstrained baseline. The evaluation compares task success, safety, motion smoothness, and inference time.
4 Experimental Results
- The task moves a bottle left to right while avoiding a stationary bottle obstacle and the tabletop, subject to joint-position limits.The feasibility oracle uses privileged simulator information.Source: §4 · pp. 3–4
- 100% step safety rate: π0.5 + safeguard maintains success close to the unconstrained baseline and improves LDLJ from −22.99 to −18.35.The local-init baseline has lower success and worse smoothness than π0.5 + safeguard; the stop-gradient ablation degrades both success and smoothness.Source: Table 1 · p. 4, §4 · p. 4
- The study fine-tunes pretrained π0.5 models with LoRA on 150 expert obstacle-avoidance trajectories for 30K steps using one NVIDIA H100 GPU.Actions are absolute joint positions with horizon H = 20.Source: §4 · p. 4
- The comparison includes the original π0.5 baseline, Gaussian and local initializations, the safeguarded policy, and a stop-gradient ablation.The method samples initial points around the observed joint configuration and uses the oracle to filter infeasible samples.Source: §4 · p. 4
5 Conclusion
The paper presents an oracle-guided safeguard for safe flow-matching robot policies using binary feasibility feedback. Its current discrete flow relies on simple interpolation paths, which may be unreliable in highly non-convex feasible regions.
5 Conclusion
- Binary feasibility feedback supports forward feasibility enforcement and local boundary-aware backward learning without explicit or differentiable constraint models.The same safeguard is used during training and inference.Source: §5 · p. 4
- Simple interpolation paths may become unreliable in highly non-convex feasible regions.The paper identifies constraint-aware flow paths and evaluation across more diverse tasks, robots, and real-world constraints as future work.Source: §5 · p. 4
A Derivation of the Boundary-Aware Backward Pass
The derivation obtains the boundary-truncated update's Jacobian by perturbing the nominal update while holding the feasible starting point fixed. The boundary intersection's displacement lies in the local tangent plane, and the Jacobian reduces to I when the update is untruncated.
A Derivation of the Boundary-Aware Backward Pass
- Holding the feasible point fixed, perturbing a truncated nominal update moves its boundary intersection within the local tangent plane, orthogonal to the outward normal.The derivation assumes a transverse intersection.Source: A Derivation of the Boundary-Aware… · p. 7
- Applying the product rule to the ideal corrected update yields the boundary-truncated Jacobian with respect to the nominal update.The displayed derivation relates the Jacobian to the intersection factor and outward normal.Source: A Derivation of the Boundary-Aware… · p. 7
- When the update is untruncated, the intersection factor is locally constant at 1 and the Jacobian reduces to I.In practice, the estimated intersection factor and normal approximate those used in the ideal derivation.Source: A Derivation of the Boundary-Aware… · p. 7
B Practical Implementation
The practical safeguard combines coarse oracle scanning and local bisection with binary probes that estimate nearby boundary direction for the backward pass.
Forward: coarse scanning and local bisection
- Uniformly spaced oracle queries accept the full update when feasible or bracket the first infeasible point for local bisection.The final feasible endpoint approximates the first-exit point.Source: B Practical Implementation · p. 7
Backward: local boundary estimation
- Symmetric probes around the search-interval midpoint use coordinate axes and random unit directions to estimate the local boundary normal.Contributions toward the infeasible side are aggregated and normalized; the sign is chosen so the estimated normal has a nonnegative dot product with the nominal update.Source: B Practical Implementation · p. 7
- The boundary estimate supplies the surrogate backward pass, with batched horizon queries and normal estimation required only during training.The binary-label estimate is related to decision-based methods such as HopSkipJumpAttack [18].Source: B Practical Implementation · p. 7
C Experimental Details
The experimental pipeline collects collision-aware demonstrations, implements a binary feasibility oracle, initializes from feasible joint configurations, and fine-tunes policy variants with LoRA.
Expert data collection
- 150 successful, safety-checked episodes provide synchronized observations, actions, and privileged scene information for Table 1.Demonstrations use a collision-aware planner, including a 4 cm inflated obstacle envelope, and stored joint paths are replayed for recording.Source: C Experimental Details · p. 7
Oracle implementation
- A batched JAX oracle combines joint-limit and collision checks, including carried-bottle collisions, and returns binary labels without differentiation through the checker.It checks sphere approximations of arm links against scene geometry using forward kinematics.Source: C Experimental Details · p. 7
Feasible initial distribution
- Feasible initial samples come from oracle-filtered joint configurations near the observed state, repeated across the horizon for both training and inference.Source: C Experimental Details · p. 8
Training parameters
- All variants are fine-tuned for 30K updates with LoRA on one H100 GPU, using batch size 32 and AdamW.The schedule uses 1K warmup steps, cosine decay, and an exponential moving average with decay 0.99.Source: C Experimental Details · p. 8
D Discussion and Limitations
The method preserves feasibility across discrete generation steps from a feasible start, and applying the same correction during training aligns learning with inference interventions.
D.1 Why Maintain Feasibility at Every Generation Step?
- Intermediate flow states are not physically executed, so maintaining their feasibility is an algorithmic choice for ensuring the final output satisfies constraints.Starting feasible and safeguarding each update keeps the generation process within the feasible set.Source: §D.1 · p. 8
- For a fixed context, a feasible initial sample remains feasible at every step when each update returns a verified feasible configuration or retains its input.The proposition covers X_k for k = 0, . . . , T.Source: §D.1 · p. 8
- Applying the correction during training exposes the policy to inference-time interventions and makes final feasibility a sampling-procedure property without a separate terminal projection.Source: §D.1 · p. 8
D.2 What Are the Main Limitations?
The discussion identifies feasibility and local-geometry assumptions, finite-query accuracy and cost, and possible path-design and query-efficiency directions.
D.2 What Are the Main Limitations?
- Straight reference paths require locally feasible connectivity; feasible endpoints alone may still yield paths through obstacles that the direction-preserving correction cannot route around.The formulation also assumes feasible target actions and obtainable feasible initial samples; a sufficient connectivity condition is a common convex feasible neighborhood for each source–target pair.Source: §D.2 · p. 8
- The full method takes 1.345 s per chunk versus 0.071 s for the Gaussian baseline, approximately 18.9× as long.Finite scans and probes limit boundary resolution, and safety scope depends on the oracle and configurations actually checked.Source: §D.2 · p. 8
D.3 Future Directions
- Nonlinear reference paths following feasible expert trajectories could route around obstacles, though connecting random initial configurations and discretizing without infeasible shortcuts remains a challenge.The paper proposes this direction for extending the approach to non-convex constraint regions while retaining the oracle-guided safeguard.Source: §D.3 · p. 9
- Adaptive coarse scanning, selective local refinement, and batched collision checking are proposed to reduce oracle-query cost while balancing computation and safety.The proposed strategies should be evaluated jointly with task success and constraint satisfaction for online manipulation.Source: §D.3 · p. 9