Source-linked AI summary
VLCP: Vision Language Control Policy Closed-Loop Code Replanning for Robot Manipulation
Dhia Naouali, Minghan Wu, Claudia Wong, Abhinav Puthran, Omar G. Younis
TL;DR
Robot policies built from vision-language models often require adapting them to unfamiliar action representations, while one-shot generated control code cannot respond when execution fails. VLCP keeps the model frozen and repeatedly rewrites its control function during each episode, raising pooled success from 3.5% to 35.1% across 57 tasks.
Problem
One-shot control code cannot revise faulty logic after execution begins, leaving manipulation failures to compound within an episode.
Method
VLCP periodically re-queries a frozen VLM to rewrite the robot’s control function from live observations within the episode.
Results
35.1% pooled success versus 3.5% for open-loop control, with non-overlapping confidence intervals across scene families in a 57-task sweep.
Takeaways & Limitations
Within-episode code replanning recovers 27.3% of failed grasps while requiring a median 10 replans per episode and caching 84% of input tokens.
Takeaways & Limitations
Repeated near-misses can exhaust the fixed block budget, especially in long-horizon tasks where recovered grasps leave placement sub-goals unfinished.
Abstract
from arXiv · showhide
Turning a frontier vision-language model into a robot policy usually means fine-tuning it to emit an action representation it never saw in pretraining, which throws away much of the reasoning that made the model worth reaching for. We go the other way and keep the VLM frozen. It writes the policy as a short Python control function, with no demonstrations and no fine-tuning. Writing that code once is open-loop, though. Existing closed-loop methods react at the wrong level: they retry a fixed policy or pick a different subtask, but never rewrite the code that failed. VLCP closes the loop where the failure actually lives, on the control code, within a single episode. Every $K$ steps the VLM re-observes the scene from multi-view RGB, proprioceptive state, and a state delta, then rewrites the control function from what it just saw, so a failure is caught before it compounds. We evaluate on a 57-task MuJoCo/RoboVerse sweep. This training-free policy reaches $35.1\%$ pooled success, against $3.5\%$ for the identical system queried once per episode. That tenfold gap holds with non-overlapping confidence intervals in every scene family. The gain traces to a $27.3\%$ within-episode recovery rate on failed grasps: a miss an open-loop controller would carry to the end of the episode gets re-observed and fixed at the next replan. And the loop stays cheap. A median $84\%$ of input tokens hit cache, an episode needs only about $10$ compact queries, and control blocks written during any replan persist to a cross-episode skill library reused in later prompts.
1 Introduction
VLCP keeps a frozen vision-language model in its pretrained code-and-reasoning regime by having it write Python control functions, then closes the loop by rewriting failed code during the same episode from live observations. It also preserves replanned control blocks in a cross-episode skill library for later reuse.
- Motivation and approach: Code policies let a frontier vision-language model control robots in a familiar Python representation without demonstration fine-tuning.This approach aims to preserve the model’s pretrained coding and reasoning abilities rather than teaching a new action language.
- Limitations of prior methods: Open-loop code generation cannot detect runtime failures, while task-level monitoring retries policies or replans subtasks without rewriting the failed controller.A grasp failure can therefore persist silently through the remainder of an episode.
- VLCP: VLCP re-queries a frozen VLM every K steps using multi-view RGB, proprioceptive state, and an explicit state delta, then rewrites the control function within the episode.This places the revision boundary at the control-code level, so failures can be corrected before they compound.
- Cross-episode skill library: Control-code blocks created during replanning are saved to a shared on-disk skill library and injected verbatim into later prompts for reuse across episodes.A later control function can import a skill written during an earlier replan.
2 Related Work
Prior work either generates robot-control code once before execution or monitors a fixed policy and invokes recovery, while vision-language-action models map observations directly to actions. VLCP instead places closed-loop correction at the control-code level, measuring its isolated contribution and recovery benefits while accumulating reusable helper modules.
- Code generation for robot control: Code-as-Policies established executable code as a robot action representation; later systems added control flow, retrieval, feedback checks, or video-conditioned generation, but queried only before execution.These methods leave errors occurring after motion starts uncorrected.
- Closed-loop monitoring and replanning: Closed-loop alternatives monitor fixed policies and trigger retries, constraint-based fallbacks, or fixed recovery routines rather than rewriting the failed control code.Examples include Code-as-Monitor, DoReMi, and Inner Monologue.
- Closed-loop monitoring and replanning: VLCP isolates the loop with a controlled K=T ablation and directly measures the recovery it provides.The paper identifies within-episode code rewriting as the locus of correction and evaluates it in Sections 4.3 and 4.5.
- Vision-language-action models: VLA-0, OpenVLA, and π0 map observations directly to actions, leaving no point to inspect or correct reasoning mid-task and depending on large demonstration datasets.Their action representations use specialized tokens or plain text, unlike the training-free methods described here.
- Skill library: VLCP saves helper modules written during replanning to a shared on-disk library and surfaces them verbatim in later prompts, adapting Voyager’s accumulate-and-reuse idea at finer grain.Unlike Voyager, VLCP does not use a curated library, promotion criteria, or an open-ended curriculum.
3 Method
VLCP represents manipulation policies as generated Python control functions that receive multi-view observations, proprioception, object poses, and an analytic Jacobian. It closes the loop both within fixed-length control blocks and across blocks by letting the VLM rewrite failing code mid-episode, while persisting helper skills across episodes.
- Policy interface: VLCP receives two fixed-camera RGB views, proprioceptive state, task-relevant object poses, and an analytic end-effector Jacobian at every control step.A task is solved when the binary terminal predicate G is satisfied.
- Policy interface: The policy is a Python function control(obs, J) → actions that outputs per-joint position targets for the simulator’s PD controller.The Jacobian is passed as a ready-made primitive so generated code can perform velocity-IK.
- Code-level replanning: VLCP rewrites the control function at block boundaries, closing the loop at the level of the failing code rather than merely retrying a fixed policy or escalating to a planner.Within each block, the compiled function remains closed-loop on fresh observations and J, but its logic is frozen until the next replan.
- Replan cadence: K=50 divides the 500-step horizon into about 10 replans, trading earlier failure correction against additional VLM query cost.Each generated function runs for the remaining steps in its block without further VLM queries.
- Skill persistence: Helper modules are written atomically to a shared skill directory, imported immediately, and persisted across episodes for later replans.At each replan, up to 12 recently modified module sources are injected into the prompt, while failed imports are reverted and surfaced to the VLM.
4 Experiments
Across a 57-task MuJoCo/RoboVerse sweep, within-episode code replanning is the central driver of VLCP’s performance: closed-loop VLCP reaches 35.1% pooled success versus 3.5% open-loop. The experiments also show that replanning recovers many failed grasps while remaining computationally practical through caching, compact outputs, and roughly ten cache-warm calls per episode.
- Experimental setup: 57 tasks span Object Picks, Kitchen Scenes, and Living-Room Scenes in MuJoCo via RoboVerse, with VLCP replanning every K=50 steps.The sweep includes 10 Object Picks, 30 Kitchen Scenes, and 17 Living-Room Scenes; each task runs for one episode.
- Closed-loop ablation: 35.1% pooled success for closed-loop VLCP versus 3.5% for its matched open-loop K=T baseline, a tenfold gap with non-overlapping confidence intervals across scene families.Closed-loop success is 60.0% on Object Picks, 33.3% on Kitchen Scenes, and 23.5% on Living-Room Scenes; open-loop reaches 10.0%, 3.3%, and 0.0%.
- Failure recovery: 27.3% of failed-grasp episodes later recover the object after reapproach, with recovery occurring at a median of 3 replans after the miss.Recovery occurred in 15 of 55 episodes containing at least one failed grasp; 40 episodes never secured the object, while open-loop recovery is 0 by construction.
- Efficiency: 84% median input-token cache reuse and approximately ten cache-warm VLM calls per episode keep within-episode replanning affordable.A static ≈8k-token prefix is cached, and the model emits a median of ≈350 tokens per replan.
- Reliability: 100% of 778 generated control blocks parsed and compiled successfully, so malformed code never triggered the compile-failure fallback.The fallback fired only on transient runtime exceptions.
5 Conclusion
VLCP closes the manipulation loop by having a frozen VLM rewrite its control code every K steps from live observations, achieving a tenfold pooled-success gain over open-loop execution. The approach remains simulation-limited, but its cross-episode skill library offers a path to continual learning without retraining.
- Core contribution: VLCP re-queries a frozen VLM every K steps to rewrite the control function from multi-view RGB, proprioception, and an explicit state delta.This catches and corrects failures while the episode is still unfolding.
- Core contribution: The loop edits the failing control artifact directly rather than retrying a fixed policy or reselecting a subtask.The authors identify the loop itself, rather than any single generated program, as the contribution.
- Evidence: 35.1% pooled success versus 3.5% for the open-loop K=T ablation yields a tenfold gain across a 57-task sweep.The ablation holds the simulator, control API, privileged poses, task set, model, and episode budget fixed while varying only within-episode replanning; confidence intervals do not overlap in any scene family.
- Limitations: VLCP has been tested only in simulation with simulator-provided object poses, leaving robustness to perception noise and real-robot dynamics untested.The authors intentionally factor out localization to isolate the closed-loop question, while noting that a learned detector is an engineering substitution.
- Outlook: Successful control blocks are saved and re-injected into later prompts, allowing the cross-episode skill library to grow and support continual learning without retraining.Later episodes can reuse and compose behaviors written earlier.
A Prompt assets
The prompt assembles documentation/API/examples, a skill library, and a goal pin, with strict instructions governing the VLM’s code output. It also supplies control examples and directs replanning around the next phase using synchronized visual observations.
- Prompt structure: The system prompt combines three cache-structured blocks: documentation/API/examples, the skill library, and the goal pin.Block A includes the output contract and worked control examples.
- Output contract: The VLM must emit only fenced Python blocks, with exactly one # control block and optional reusable # skills/<snake_case_name>.py modules.The control block defines def control(obs, J) -> list[dict], while skill modules persist and remain importable across replans.
- Replanning contract: Control may retain module-level state across K calls, while skill files must remain free of mutable state.Each replan handles only the next phase, approximately K closed-loop steps, rather than the full multi-phase plan.
- Observation and examples: Synchronized camera views should be inspected carefully because they often reveal the real scene state, including whether grasps or pushes succeeded and what changed.The prompt’s worked example demonstrates the control(obs, J) contract and explicit gripper control through move_to_target.
B Generated-code sample
The sample shows a frozen VLM-generated Python skill module for chocolate-pudding pick-and-place, written during replanning and reused verbatim in later replans. It combines documented world-frame assumptions with typed helper functions for grasp, lift, and basket placement targets.
- Module reuse: The skill was written during a replan and re-injected verbatim into every later replan’s Block B, where the episode’s control function imports it.The sample illustrates a module docstring, full type hints, and per-function Args/Retu documentation.
- Module scope: The module computes simple world-frame panda_hand position targets for top-down manipulation of chocolate pudding and placement into a nearby basket.Its docstring states the scope as chocolate-pudding picking and basket placement in LIBERO grocery-style scenes.
- Control assumptions: The assumptions specify that the pudding root approximates the footprint center, the grasp is top-down, and the panda_hand frame sits roughly 10 cm above the fingertip pads.Returned targets are panda_hand positions in world coordinates rather than object poses or fingertip positions.
- Pick control: The pick helper returns hover, low-grasp, and lift targets from the pudding root position, with optional XY offset and world-frame height parameters.Its defaults include grasp_z = 0.118 and lift_z = 0.255, and it returns three (3,) world-coordinate arrays.
- Basket placement: The placement helper computes a world-frame panda_hand target inside the basket opening from the basket root, hand height, and optional XY offset.If no offset is supplied, the placement point is the basket center; hand_z defaults to 0.175.