PaperlayerView exact revision on arXiv

Source-linked AI summary

SUN: Persistent Programs For Language-Grounded Control-to-Learning-to-Real Policies

Weiqi Wang, Zhi Li, Yudong Lei, David Martinez, Xiaofeng Gao, Yuxin Jiang, Chenfanfu Jiang, Yingnian Wu, Demetri Terzopoulos, Ran Gong

arXiv:2608.31167v1cs.ROcs.AI

TL;DR

Long-horizon manipulation pipelines often lose the task semantics validated by control, leaving rewards and stage logic vulnerable to drift. Kuafu preserves typed SUN Programs through MPC screening, policy learning, and data generation; across nine tasks, it reports higher success than sparse-reward and alternative learners and transfers to physical robots. The paper concludes that control-validated semantics remain effective after behavior is amortized into learned policies.

  • Problem

    Existing handoffs from model-based control to learned policies discard objectives, constraints, transitions, and satisfaction boundaries, allowing rewards to drift from control-validated specifications.

    Source: §I · p. 1

  • Method

    Kuafu synthesizes typed SUN Programs from language and scene semantics, screens them with MPC, and retains their compiled interfaces while training stage-conditioned and residual policies.

    Source: §I · p. 1, A. Semantically UNified (SUN) Programs · p. 3, C. SUN-Guided Policy Amortization · p. 5

  • Results

    82.03% task-macro success across nine tasks, versus 35.67% for the sparse-reward baseline, while Kuafu-generated data supports downstream visual policies and zero-shot physical-robot transfer.

    Source: §I · p. 2, B. Performance and Ablation Analysis · p. 6, §V · p. 8

  • Takeaways & Limitations

    Task semantics validated through control remain effective after behavior is amortized into learned policies, supporting robust multi-stage manipulation and sim-to-real transfer.

    Source: §V · pp. 8–9

    SUN Programs depend on registered scene interfaces with informative object frames and axes and a finite typed-operator library; deformable objects and fluids require new operators.

    Source: §V · p. 9

Abstract

from arXiv · show

Bridging model-based control and learned policies in long-horizon manipulation has harbored a silent disagreement: control executes specified objectives, learning amortizes that behavior into a reactive policy, yet existing protocols discard task semantics, leaving rewards hand-crafted and behavior drifting from what control verified.We introduce Semantically UNified (SUN) Programs, typed executables where geometric and contact relations are defined once and compiled into aligned Model Predictive Control (MPC) costs, satisfaction predicates, RL rewards, transition guards, and diagnostics. Our system, Kuafu, driven by large vision language systems, automatically synthesizes SUN Programs from language and scene semantics, screens feasibility via MPC, and retains semantics while training stage-conditioned policies. Across nine tasks, Kuafu achieves 82.03% macro-success, outperforming sparse-reward (35.67%) and Stage-BC (24.75%) baselines. At 8192-way scale, it generates 10.57x the successful trajectory time per hour of human teleoperation. With 500 trajectories per task, Kuafu data trains DP3 policies to 46.0% simulation success (vs. 22.4% for alternatives) and 34.7% on physical Franka and Kinova robots. These results establish that simulation-screened task semantics can effectively amortize control into robust policies, without demonstrations or manual dense rewards, unifying symbolic planning and data-driven execution.

I. INTRODUCTION

Kuafu addresses semantic drift between model-based control and learned policies by preserving typed SUN Programs across language grounding, MPC screening, policy learning, and deployment. The framework improves multi-stage manipulation success while reducing manual reward engineering and supporting zero-shot sim-to-real transfer.

Motivation

  • RL for multi-stage manipulation needs dense, structured rewards, but manually crafting them is laborious, brittle, and expensive to revise.Candidate reward effects are only evaluated after full policy training and rollout.Source: §I · p. 1
  • Model-based control verifies geometric objectives and transitions, yet handoffs to learning often discard the task semantics that justified the actions.This can cause policy rewards and stage logic to drift from control-validated specifications.Source: §I · p. 1

Results

  • Across nine manipulation tasks, Kuafu achieves higher success with fewer language-model interactions and lower computational cost than competing methods, while enabling physical-robot sim-to-real transfer.The evaluation includes Franka and Kinova robots and reports significantly stronger downstream policy success from Kuafu-generated simulation data.Source: §I · p. 2

SUN Programs

  • SUN Programs are typed executable artifacts that preserve task semantics across control, learning, and deployment.The same relations are compiled into aligned interfaces for MPC, monitoring, and reinforcement learning.Source: §I · pp. 1–2

Kuafu

  • Kuafu synthesizes SUN Programs from language and scene frames, screens feasibility with MPC, and trains stage-conditioned policies without manual reward engineering or seed demonstrations.Optimization rejects faulty objectives and transitions before repeated policy-training cycles, while Residual RL learns deployable contact-rich execution.Source: §I · p. 2

II. RELATED WORK

Prior work connects symbolic task structure, language grounding, optimization, and learning through executable costs, constraints, rewards, and trajectories. However, these approaches differ in how they expose task semantics to control and policy learning, and iterative reward refinement remains costly.

Executable Task Representations

  • Executable task representations connect discrete task progression with continuous motion through symbolic planning, program generation, geometric feasibility, and scene grounding.Related approaches also encode temporal progress with finite-state rewards or signal-temporal-logic constraints.Source: §II · p. 2

Language-Grounded Optimization

  • Language-grounded optimization converts task descriptions into composable 3D value maps or reward parameters evaluated by MPC.These methods expose spatial objectives to closed-loop control.Source: §II · p. 2

Automated Reward Generation

  • Language models automate executable rewards and sim-to-real domain-randomization configurations, but optimized rewards can overfit observed behavior or encode invalid task specifications.A high return may therefore fail to translate into strong performance on the true task metric.Source: §II · p. 3

Closest Precedents

  • ReKep, OmniManip, LATO, and MPC-Net respectively connect grounded costs, spatial constraints, learner-induced state distributions, or control objectives to manipulation learning.These precedents span closed-loop optimization, VLM checking, adaptive MPC teaching, and Hamiltonian-loss policy training.Source: §II · p. 3

III. METHODS

Kuafu constructs typed SUN Programs from language, scene frames, and operators, then preserves each grounded relation across control, monitoring, and learning. Stage-conditioned execution uses shared predicates and stage context to maintain task semantics.

SUN Program construction

Semantic Unification

  • Every grounded relation is compiled into an MPC objective, a stage-monitor predicate, and an RL learning term while retaining its identity, grounding, and stage membership.This shared correspondence is the paper’s Semantic Unification mechanism.Source: A. Semantically UNified (SUN) Programs · pp. 3–4

Running example

Stage progression

  • The stage monitor evaluates current-stage predicates after each action and updates the stage only when the associated completion conditions are satisfied.For drawer opening, progression waits until predicates such as directional movement and fixed orientation are satisfied.Source: A. Semantically UNified (SUN) Programs · p. 4

B. Program Validation and Acceptance

Kuafu screens candidate SUN Programs through closed-loop nonlinear MPC simulation, independently checks physical success, and repairs failures before freezing accepted semantics for policy learning.

Independent task success

  • An independent evaluator gext provides a frozen sparse terminal outcome and is never exposed to program synthesis or learned policies.This separates external task success from the SUN monitor’s completion logic.Source: B. Program Validation and Acceptance · p. 4
  • For drawer opening, gext declares success only when the drawer joint reaches 12 cm displacement; otherwise geometric completion can be treated as false completion.Source: B. Program Validation and Acceptance · p. 4

MPC screening

  • Task-space nonlinear MPC consumes compiled SUN interfaces and screens each program in closed-loop simulation using direct multiple shooting.The controller replans from the updated runtime state after each observed action.Source: B. Program Validation and Acceptance · p. 4

Acceptance and repair

  • Each candidate is tested under 10 task-sampled initial conditions and accepted when at least one rollout achieves the physical outcome without any false completion.Failures can trigger up to five whole-program repair attempts based on measured violations.Source: B. Program Validation and Acceptance · p. 4
  • After acceptance, Kuafu freezes grounded relations, tolerances, predicates, term definitions, and stage order before handing successful MPC trajectories to policy learning.Source: B. Program Validation and Acceptance · p. 4

C. SUN-Guided Policy Amortization

Kuafu amortizes accepted MPC behavior into a reactive stage-conditioned policy, then adapts it with bounded residual reinforcement learning while retaining SUN-derived task supervision.

Stage-conditioned behavioral cloning

  • 1,000 physically successful MPC trajectories train a deterministic Stage-BC policy that maps observations and active stages to 10-step action chunks.During learned execution, the reactive policy replaces the MPC optimization loop at each control step.Source: C. SUN-Guided Policy Amortization · p. 4

Bounded residual reinforcement learning

Program-derived residual reward

  • Kuafu rewards monotonic within-stage progress using program-derived scores, stage transitions, physical success, false completion, and action terms.The score resets after stage transitions and terminates episodes at success, false completion, or timeout.Source: C. SUN-Guided Policy Amortization · p. 5

Learning parameter calibration

D. Data Production

For downstream visual-policy learning, Kuafu retains trajectories using a common independent physical-success criterion, carrying consistent outcome evaluation across data generators.

Data retention

  • The downstream visual-policy study retains trajectories according to the common physical-success evaluator gext across all data generators.The retained SUN Program continues providing stage progression, completion monitoring, and false-completion detection during data generation.Source: D. Data Production · p. 5

IV. EXPERIMENTS

The experiments evaluate SUN Programs across nine multi-stage manipulation tasks, covering varied motion, contact, grasping, and precision capabilities under standardized simulation conditions.

IV. EXPERIMENTS

  • Nine multi-stage tasks span pushing, pulling, twisting, reorienting, stacking, drawer manipulation, picking, placing, and precision contact behaviors.Source: §IV · pp. 5–6
  • Task-macro external-outcome success is the unweighted mean across tasks, so each task contributes equally regardless of rollout count.Source: §IV · p. 6

A. Formulation Reliability

Kuafu reliably forms and validates executable programs through MPC screening and diagnostic recovery, while converting language interaction into a task-level rather than recurring execution cost.

A. Formulation Reliability

  • 95.6% of 45 independent program-formation runs were accepted after diagnostic repair, up from 64.4% initially.Twenty-nine runs passed initially, and recovery accepted 14 of the remaining 16.Source: A. Formulation Reliability · p. 6
  • Kuafu uses 110.5k tokens per task, while VoxPoser’s cumulative interaction exceeds Kuafu’s after four rollouts.Persistent programs shift language interaction from a recurring execution cost to a task-level cost.Source: A. Formulation Reliability · p. 6

B. Performance and Ablation Analysis

Kuafu’s complete control-to-learning handoff substantially outperforms partial handoffs and competing reward-search or online-planning baselines, while retaining performance on longer task horizons at lower compute cost.

B. Performance and Ablation Analysis

  • 82.03% task-macro success for Kuafu exceeds 35.67% for sparse reward, a 46.36-point gain.The three-run robustness aggregate reaches 79.43% success, with false-completion rates of 0.60%.Source: B. Performance and Ablation Analysis · p. 6
  • Removing the accepted Stage-BC prior yields 0% success, while Stage-BC alone achieves 24.75% and residual RL on stage-free BC reaches 18.56%.Removing or broadening the residual bound also collapses all tasks to 0% success.Source: B. Performance and Ablation Analysis · p. 6
  • Eureka’s best candidate reaches 18.24% success but its final policy falls to 12.85%, while VoxPoser achieves 10.00% and fails on five tasks.Eureka’s Task T0 performance drops from 45.14% to 0.21%.Source: B. Performance and Ablation Analysis · p. 6
  • Kuafu retains 91.4% of shorter-task performance across paired horizon tests, compared with a drop from 65.14% to 22.21% for sparse-reward handoff.Kuafu reaches 87.25% on T6 and 95.21% on T7, while non-Kuafu maxima are 13.81% and 38.80%.Source: B. Performance and Ablation Analysis · p. 7
  • 122.67 GPU-hours for Kuafu versus 479.60 for Eureka yields a 3.91× efficiency gain under one-GPU worker-hour accounting.Kuafu’s program formation costs 1.55 GPU-hours, while MPC validation and mentor collection dominate its compute.Source: B. Performance and Ablation Analysis · p. 7

A. Downstream DP3 success (%) (500 successful trajectories per task)

Under matched successful-trajectory budgets, Kuafu-generated data produces stronger downstream DP3 policies and more mentor-aligned, smooth trajectories, while substantially increasing successful-data throughput.

A. Downstream DP3 success (%) (500 successful trajectories per task)

  • During data collection, the learned controller replaces online MPC while the retained SUN Program supplies stage annotations, completion monitoring, diagnostics, and false-completion filtering.Source: C. Data Efficiency · pp. 7–8
  • 46.02% task-macro success from Kuafu data exceeds SkillMimic’s 22.44% with 500 successful trajectories per task.The fixed DP3 learner and matched budget isolate downstream data utility; Kuafu has a 2.05× gap.Source: C. Data Efficiency · p. 8
  • 57.3% of Kuafu rollouts are Q95-qualified, versus 20.4% for MimicGen, a 2.81× gain.Among successful rollouts, Kuafu has normalized metrics of 1.04 completion time, 0.67 end-effector path, 0.52 action path, and 0.28 action second difference.Source: C. Data Efficiency · p. 8
  • 246.79 successful trajectory minutes per GPU-hour is 10.57× human teleoperation throughput.Human teleoperation produces 23.34 successful trajectory minutes per hour.Source: C. Data Efficiency · p. 8

D. Visual Learning and Zero-Shot Transfer

Kuafu’s persistent SUN Programs support visual policies after privileged state and stage inputs are removed, achieving substantial simulation and physical transfer success. The evaluation also identifies grasp acquisition and limited generalization scope as important remaining boundaries.

Visual Learning

  • 46.02% DP3 task-macro success was achieved after removing privileged state and stage inputs, exceeding the strongest RGB policy at 38.52%.ManiFlow-PC reached 46.46%, while DP3 reached 46.02% across the evaluated single-task policy families.Source: D. Visual Learning and Zero-Shot… · p. 8

Zero-Shot Transfer

  • 34.72% task-macro success was achieved across 106 physical trials and six tasks without real-world fine-tuning.Success was nonzero on every tested task, with calibrated RGBD-camera setups and constrained randomization.Source: D. Visual Learning and Zero-Shot… · p. 8

Limitations

  • Precise grasp acquisition remains the main observed physical failure mode, and the evaluation does not test held-out instructions, operators, scene semantics, or task compositions.SUN Programs also require informative registered scene interfaces and finite typed-operator libraries; deformable objects and fluids require new operators.Source: D. Visual Learning and Zero-Shot… · p. 8, §V · p. 9

End-to-End Results

  • The complete control-to-learning handoff reached 82.03% success, while fixed-volume DP3 training reached 46.02% success and enabled zero-shot transfer across 106 physical trials.The framework preserves an MPC-screened SUN Program alongside the learned policy.Source: §V · pp. 8–9

Related papers

Loading 2608.31167v1…