PaperlayerView exact revision on arXiv

Source-linked AI summary

GloVLA: Let Geometry Move and Local VLA Interact for Robust Object-Centric Manipulation in Unstructured Environments

Truong Thanh Nguyen, Huy Hoang Nguyen, Ha Anh Nguyen, Binh Khanh Dinh, Ngo Anh Vien, Duy Nguyen Ho Minh, Minh Nhat Vu, Ngan Le

arXiv:2609.06256v1cs.RO

TL;DR

Single-policy VLAs must handle long-range transport and contact-rich interaction, a combination that can be brittle and inefficient in unstructured environments. GloVLA assigns transport to a geometric controller and local interaction to VLA policies; across backbones it preserves or improves standard benchmark performance and reports larger gains under distribution shift.

  • Problem

    A single VLA policy must learn both long-range geometric transport and local contact-rich manipulation, despite visual and geometric shifts that can move execution beyond its training distribution.

    Source: §I · p. 1

  • Method

    GloVLA uses a geometric controller for transport to object-centric handoff regions, then invokes local VLA policies for contact-sensitive interaction.

    Source: §I · p. 2

  • Results

    Across four VLA backbones, GloVLA preserves or improves LIBERO and LIBERO-Plus performance, with larger gains under distribution shift; π0 average success rises from 87.4% to 98.6%.

    Source: §I · p. 2, B. Standard LIBERO Object Results · p. 5

  • Takeaways & Limitations

    The factorization improves robustness and efficiency across simulation and physical-robot settings while preserving or improving standard LIBERO performance.

    Source: §VI · p. 8

    GloVLA depends on reliable target localization and reasonable fixed handoff offsets, and LIBERO-Challenge does not model all real-contact effects of clutter.

    Source: §V · p. 8

Abstract

from arXiv · show

Vision-language-action (VLA) models have shown promising generalization for language-conditioned robot manipulation, but deploying them in unstructured environments remains challenging. A single end-to-end VLA policy must simultaneously solve long-range transport of the end effector to task-relevant regions and short-horizon, contact-rich interaction upon arrival. This formulation is inefficient and brittle: small visual shifts, distractors, clutter, occlusions, or unfavorable initial gripper poses can push the policy outside the local state distribution in which it was trained, leading to task failure. We introduce GloVLA, a hybrid framework that explicitly separates object-centric manipulation into two complementary regimes: a geometric transport controller moves the end-effector into interaction-centric handoff regions, and local VLA policies handle only the short-horizon interaction phases. GloVLA is model-agnostic and can be integrated with different VLA backbones with no additional demonstrations and no changes to the action space or success predicate. Experiments on standard LIBERO and LIBERO-Plus Object tasks together with a newly introduced LIBERO-Challenge benchmark ettings with clutter, distractors,illumination changes, visual shifts, and obstruction show that GloVLA improves task success and substantially lowers VLA inference cost compared with full end-to-Challenge, full-trajectory GR00T N1.6execution degrades to 20.9% average success while GloVLA retains 88.5%; on a physical UR10e, overall success improves from 35.6% to 90.0% while mean inference time is more than halved. Videos and additional results are available at https://glovla-project.github.io/

I. INTRODUCTION

GloVLA separates object-centric manipulation into geometric transport and local VLA interaction to address brittleness and inefficiency in unstructured environments. The paper introduces LIBERO-Challenge and reports gains across simulation and physical-robot evaluations.

I. INTRODUCTION

  • Single-policy VLAs must handle both long-range transport and contact-rich interaction, increasing computation and data requirements while risking errors under visual shifts, clutter, or unfavorable gripper poses.These conditions can move execution outside the local state distribution represented in training demonstrations.Source: §I · p. 1
  • GloVLA delegates repeatable transport to a geometric controller and reserves closed-loop VLA control for local, contact-sensitive interaction.A fixed object-centric offset defines the handoff, and the framework wraps existing VLA backbones without architectural modification.Source: §I · p. 2
  • LIBERO-Challenge extends LIBERO Object with clutter, distractors, obstruction, illumination changes, visual shifts, and graded combinations of perturbations.Its easy, medium, and hard settings combine one, two to three, and four to five perturbations, respectively.Source: §I · p. 2
  • Across four VLA backbones, GloVLA preserves or improves LIBERO and LIBERO-Plus performance, with larger gains under distribution shift and more than halved inference time on UR10e.The reported advantage is also larger under matched source-demo budgets.Source: §I · p. 2

II. RELATED WORK

Related work motivates separating geometric transport from learned interaction rather than adding robustness machinery around a monolithic VLA. GloVLA uses a deterministic, backbone-agnostic boundary and trains separate local policies for grasping and placing.

II. RELATED WORK

  • GloVLA complements VLA robustness methods by removing learned transport from the object-centric task, while reserving VLA control for visual and contact reasoning.Classical planners can fill the transport role when collision-aware motion is required; the approach uses a deterministic object-centric handoff rather than LLM-sequenced subgoals.Source: §II · pp. 2–3

III. METHODOLOGY

  • The framework uses original full-task demonstrations, preserves the local policies’ observation-action interface and success predicate, and adds no learned component at the fixed boundary.It is backbone-agnostic for VLAs with the same control interface.Source: §III · p. 3

A. Problem Formulation

  • Two independently trained local policies handle grasp and place clips, while object- and basket-center estimates define the transport controller’s handoff targets.The phase policies share a backbone and interface but not parameters; extraction regions delimit the positional support of their training clips.Source: A. Problem Formulation · p. 3

B. Sphere-Conditioned Demonstration Extraction

Demonstrations are segmented into local grasp and placement clips using end-effector distance and gripper-state signals. The local dataset combines both clip types and retains only demonstrations with finite endpoints.

B. Sphere-Conditioned Demonstration Extraction

  • Extraction tracks end-effector distance to the object and basket centers, commanded gripper state, and an aperture proxy from the observed finger joints.Observed open is defined by an aperture of at least 0.02 m; commanded open is a gripper coordinate above 0.5.Source: B. Sphere-Conditioned Demonstration… · p. 3
  • A grasp clip begins at the first object-region entry with both gripper signals open, includes the first subsequent close command, and ends at the first exit.The exit step is included; if the end effector never exits, the clip covers the remaining trajectory.Source: B. Sphere-Conditioned Demonstration… · p. 3
  • A place clip starts on basket-region entry while holding the object and ends after the first release command plus eight subsequent steps.The local dataset is the union of grasp and place clips, and a demonstration is retained only when both clip endpoints are finite.Source: B. Sphere-Conditioned Demonstration… · p. 3

C. Transport Controller

GloVLA uses a closed-loop Cartesian controller for global transport to object- and basket-centered handoff positions. Local VLA policies retain responsibility for grasping and releasing.

C. Transport Controller

  • A lightweight closed-loop Cartesian controller handles global transport in LIBERO’s native 7D OSC-pose action space, without collision checking or trajectory optimization.A full motion planner can replace it when collision-aware transport is required.Source: C. Transport Controller · p. 4
  • Object and basket handoff positions are their estimated centers plus fixed offsets that place the end effector near the local grasp and placement policy distributions.The offsets comprise horizontal grasp and placement offsets and corresponding approach heights.Source: C. Transport Controller · p. 4
  • The controller regulates position while holding orientation at mode entry; the gripper stays open during object approach and closed during basket transport, leaving grasping and release to the local VLA policies.Its translational action uses a clipped proportional law, and it exits on handoff-set entry or step-budget exhaustion.Source: C. Transport Controller · p. 4

D. Hybrid Switching Policy

The hybrid policy switches deterministically between geometric transport and local VLA interaction. Its handoff design relates localization error, controller tolerance, and offsets to the positional support of local training clips, but does not guarantee full in-distribution states.

D. Hybrid Switching Policy

  • The switched policy has four modes: object approach, local grasping, basket transport, and local placement.The geometric controller handles approach and transport, while VLA policies handle grasp and placement.Source: D. Hybrid Switching Policy · p. 4
  • Mode transitions use deterministic guards, and budget exhaustion passes control to the next mode rather than terminating the episode; the official LIBERO success predicate is unchanged.A timed-out transport can therefore hand the local policy a suboptimal start.Source: D. Hybrid Switching Policy · p. 4
  • For handoffs triggered by handoff-set entry, ε_j + η_j + ∥δ_j∥2 ≤ r_j guarantees positional containment in the extraction region, not an in-distribution state.Orientation, gripper state, and visual context remain unconstrained, and budget-exhausted transport may hand off outside that region.Source: D. Hybrid Switching Policy · p. 4

IV. EXPERIMENTS

The experiments address task success, robustness to unstructured-scene perturbations, demonstration efficiency, and transfer to a physical robot. LIBERO-Challenge scenes span easy, medium, and hard perturbation combinations.

IV. EXPERIMENTS

  • The experiments ask whether GloVLA improves success over full end-to-end VLA execution and robustness under unstructured-scene perturbations.Source: §IV · p. 4
  • They also test demonstration efficiency and whether simulation results transfer to a physical robot.Source: §IV · p. 4
  • LIBERO-Challenge examples range from easy scenes with one perturbation to medium scenes with 2–3 and hard scenes with 4–5 combined perturbations.The perturbations include clutter, distraction, illumination, obstruction, and visual shift.Source: Figure 3 · p. 5

A. Experimental Setup

The experiments compare several VLA backbones across LIBERO suites and a stress-test benchmark with progressively composed scene perturbations.

  • The evaluated backbones are π0, π0.5, GR00T N1.6, and GR00T N1.7; detailed robustness tests use GR00T N1.6 for its broader checkpoint coverage.Source: A. Experimental Setup · p. 5
  • Evaluation covers LIBERO Object, LIBERO-Plus Object, and LIBERO-Challenge, which varies clutter, distractors, illumination, camera or appearance, and approach-corridor obstruction.Source: A. Experimental Setup · p. 5
  • LIBERO-Challenge has 50 evaluation-only scenes: 30 easy, 10 medium, and 10 hard, with one, two or three, and four or five perturbations, respectively.Source: A. Experimental Setup · p. 5
  • Both methods use the same RGB observations, proprioception, 7D OSC-pose action space, and official success predicate; simulation uses environment-state centers, while UR10e localization uses SAM 3.Source: A. Experimental Setup · p. 5

B. Standard LIBERO Object Results

On standard LIBERO Object tasks, GloVLA matches or improves on FullVLA across all ten tasks and four tested backbones.

C. Robustness Under Unstructured Environments

Across simulated distribution shifts, matched demonstration budgets, and physical-robot trials, GloVLA retains stronger task success than FullVLA.

D. Data Efficiency

  • 99.9% average success across unstructured conditions with 50 demos compares with 48.5% for FullVLA under matched source-demonstration budgets.GloVLA also reaches 100% success on clean tasks with 10 demos.Source: D. Data Efficiency · p. 5

C. Robustness Under Unstructured Environments

E. Real-World Demonstrations

  • 35.6% to 90.0% overall success and 60.1 s to 28.2 s inference time mark the UR10e comparison between FullVLA and GloVLA.GloVLA retains 80% and 70% success on medium and hard compositions, compared with 10% and 0% for FullVLA.Source: E. Real-World Demonstrations · p. 6

F. Qualitative Results

The qualitative results, parameter sweeps, and discussion show where the handoff is robust, what drives its benefits, and which assumptions still constrain it.

F. Qualitative Results

  • FullVLA fails from the cluttered real-robot scene onward, while GloVLA succeeds through the hardest combined obstruction-and-distraction setting.Remaining failures include an erroneous object center that misplaces the handoff and an oversized offset that leaves the local policy to recover transport.Source: F. Qualitative Results · p. 6, Figure 5 · p. 6

G. Ablation Study

  • Success stays above 80% across tested extraction radii; offset tolerance is asymmetric, with horizontal grasp offset ρo the tightest constraint.ρb, zg, and zp stay near perfect across ±8λ but degrade at ±16λ; ρo preserves success within ±2λ, degrades to 40–80% at −4λ, and falls to 0–50% at −8λ.Source: G. Ablation Study · p. 6, Figure 6 · p. 6, Figure 7 · p. 7

V. DISCUSSION AND FUTURE WORK

  • The hybrid yields modest reliability gains on standard benchmarks but large, broadly distributed robustness gains under realistic perturbations in simulation and on a real robot.Source: §V · pp. 6–7
  • Object-centric transport maps varied starts into a compact local interaction distribution, shortens the VLA-controlled horizon, and concentrates demonstrations on contact-sensitive actions.As the extraction region expands, the local policy recovers more transport and GloVLA approaches FullVLA; gains arise near the local-policy end of this continuum.Source: §V · p. 7
  • The implementation uses fixed offsets, no learned handoff-selection model, and no action-space modification; its simple proportional controller makes factorization the most plausible source of robustness gains.The passage notes that fully isolating this attribution requires decoupled ablations.Source: §V · pp. 7–8
  • GloVLA depends on reliable object and basket centers and reasonable handoff offsets, while LIBERO-Challenge does not model all real-contact effects of clutter.The implementation uses simulator centers in LIBERO and calibrated object-centric targets in the real setup; future work includes learned localization, adaptive handoffs, recovery, collision-aware transport, and longer multi-object tasks.Source: §V · p. 8

VI. CONCLUSION

GloVLA separates object-centric manipulation into geometric transport and local VLA interaction. Across four VLA backbones, this factorization preserves or improves standard LIBERO performance while yielding larger gains under LIBERO-Challenge perturbations, with improved demo efficiency, lower inference cost, and effective transfer to a physical robot.

VI. CONCLUSION

  • Across four VLA backbones, GloVLA preserves or improves standard LIBERO performance and provides larger gains under LIBERO-Challenge perturbations.The tested perturbations include clutter, distractors, obstruction, illumination variation, visual shift, and composed perturbations.Source: §VI · p. 8
  • GloVLA factorizes object-centric manipulation into geometric transport and local VLA interaction.The conclusion assigns structured, repeatable motion to geometry and uncertain, contact-rich interaction to the VLA.Source: §VI · p. 8
  • The framework improves demo efficiency, reduces VLA inference cost, and transfers effectively to a physical robot.The authors argue that robust and efficient robot foundation models may depend on better boundaries between explicit structure and learned intelligence.Source: §VI · p. 8
Loading 2609.06256v1…