PaperlayerView exact revision on arXiv

Source-linked AI summary

Instruction-Conditioned Motion Control for Robotic Arms: Language-to-Code Decomposition and Reinforcement Learning

Fan Huang, Dezhong Tong, Changyu Chen, Haewoon Kwak, Jisun An

arXiv:2610.19617v1cs.ROcs.AI

TL;DR

Language-driven robot programs can be syntactically executable without showing how instructions shape downstream execution, and reward labels may not match intended performance. ICMC separates parameter specification, instruction generation, and frozen code synthesis, then trains an instruction policy through that generator. The main trajectory’s logged positive-reward rate rose from 31.6% to 35.3%, while audits found positive labels alongside geometric-failure logs; an independent evaluation protocol is introduced to check task completion against fixed specifications.

  • Problem

    The paper examines how to trace task instructions into executable robot programs and how reward labels from program execution relate to intended task performance.

    Source: §I · p. 1, §VIII · p. 6

  • Method

    ICMC separates task parameters, a trainable instruction policy, and a frozen code generator, and introduces independent task verification against fixed specifications.

    Source: §I · p. 1

  • Results

    The main trajectory’s logged positive-reward rate rose from 31.6% to 35.3%, while audits found at least 19, 18, and 5 positive rewards with geometric-failure logs across three runs.

    Source: §IX · pp. 6–7

  • Takeaways & Limitations

    The independent protocol separates robot control from success assessment; its CPU implementation passed ten negative controls, and a hand-authored controller completed all 40 predeclared scenes.

    Source: §I · p. 1

    Recovered code-generation charges are partial API costs that exclude local training compute, while aggregate compute and cost estimates are unreconciled with those ledgers.

    Source: §VIII · p. 6

Abstract

from arXiv · show

Instruction-Conditioned Motion Control (ICMC) separates parameter specification, instruction generation, and code synthesis for language-driven robot control, making execution outcomes traceable to instructions and programs. We compare one-shot generation with ICMC under o1 and o4-mini on cube pick-and-place and letter-reshaping tasks for a UR5e arm in MuJoCo. We then train an open-weight instruction policy by GRPO through a frozen code generator on cube placement onto two- and five-cube letter layouts, averaging the rewards of five programs per instruction. The logged positive-reward rate rose from 31.6% to 35.3% (253/800 to 282/800) between the first and last five training steps. A log audit identifies positive binary labels associated with geometric-failure messages: at least 19 in that run, 18 in the dense-reward run, and 5 in the run with one program per instruction. In one case, all five rewards were 1.0 while the log reports a 0.9515 m miss against a 0.05 m tolerance; the verdict came from the program's own output stream. We distinguish logged reward rates from independently verified task success and report descriptive results for each configuration's single training trajectory. We introduce an independent evaluation protocol that checks task completion against fixed specifications, separating robot control from success assessment to support reproducible evaluation.

I. INTRODUCTION

The paper introduces ICMC as a language-to-code decomposition that makes downstream execution traceable to generated instructions and programs. It motivates this design alongside comparisons of one-shot and staged generation, reward sampling, and evaluation methods.

II. RELATED WORK AND SCOPE

  • Prior work grounds robot actions in capabilities and state, uses feedback for recovery, and evaluates rearrangement against a specified goal; this study examines how tasks become executable programs,,,, [6].ICMC differs from cited code-generation and prompting approaches by inserting an instruction stage between task and program –,.Source: §I · p. 1, §II · p. 1

I. INTRODUCTION

  • ICMC separates task parameters, instruction generation, and code synthesis over a fixed UR5e API, making execution outcomes traceable to instructions and programs.The full instruction text, including prose and a reference-code block, is the trainable action; code synthesis remains frozen.Source: §I · p. 1, A. Instruction-conditioned control · p. 2
  • The study asks how one-shot generation compares with ICMC across two models and tasks, how GRPO training performs through a frozen code generator, and what archived reward-sampling and optimization comparisons establish.The paper also asks which learning-rate and update-epoch effects remain confounded.Source: §I · p. 1
  • The main training trajectory’s logged positive-reward rate rose from 31.6% to 35.3%, while audits found at least 19, 18, and 5 positive labels with geometric-failure logs across three runs.The paper distinguishes these logged rates from validated task performance.Source: §I · p. 1
  • The study characterizes simulated execution through one archived training trajectory per configuration; independent evaluation of generated programs and replication across training seeds remain future work.Matched learning-rate and epoch comparisons also remain future work.Source: §I · p. 1

A. Instruction-conditioned control

  • ICMC’s three stages specify task parameters, generate instructions for perception, fetching, and placing, then synthesize a complete program from those instructions and a fixed API.Exposing the intermediate artifact helps identify when a program ignores an instruction, changes a function contract, or terminates before the requested scene.Source: A. Instruction-conditioned control · p. 2

B. Reward sampling and optimization

The section analyzes five-program reward averaging and a centered group-based update, while noting that the archived reward labels and optimization settings constrain interpretation.

Reward sampling

  • For fixed instructions, averaging M conditionally independent binary rewards estimates the evaluator’s label probability and reduces variance, at M times the nominal sampling budget.That probability represents task success only if the evaluator is valid.Source: B. Reward sampling and optimization · p. 2
  • 302 of 640 instructions (47.2%) received mixed rewards, so a single program would have differed from the five-program mean for those instructions.These counts describe logged-label variation, not the sources of training-curve changes.Source: B. Reward sampling and optimization · pp. 2–3

Optimization

  • The centered reward coefficient weights each instruction’s score-function gradient; groups with equal rewards contribute no centered signal before normalization, clipping, and regularization.The archive does not identify the source version governing clipping, the KL coefficient, or advantage normalization.Source: B. Reward sampling and optimization · p. 3
  • Archived runs combine M=5 averaging with learning rate 10−6 and E=3 update passes, while binary and dense reward variants were trained separately.The dense reward combines a heuristic bug detector, distance progress, and completion; detector precision and recall remain unmeasured, and false positives can penalize valid programs.Source: B. Reward sampling and optimization · p. 3

IV. TASKS, CONFIGURATIONS AND EVALUATION

This section defines the archived runs and task configurations, then reports recovered planning outputs and the scope of the archived comparisons.

IV. TASKS, CONFIGURATIONS AND EVALUATION

  • An archived run is an analyzed recorded run scored from parsed program output; logged rate counts binary labels equal to 1, while validated success uses external geometric checks.The denoised reward is the mean of M=5 program rewards for one instruction; I and M denote archived two- and five-cube layouts.Source: §IV · p. 3

A. Environment, tasks and configurations

  • The study uses MuJoCo with a UR5e arm and parallel gripper; RQ1 includes image-conditioned prompting, while RQ2/RQ3 use textual task and scene descriptions.The figure depicts cube transfer from the rear table to a disposal area on the front table.Source: A. Environment, tasks and configurations · p. 3, Figure 2 · p. 3
  • The task suite covers cube pick-and-place and letter reshaping, with 12 targets yielding 66 unordered or 132 directed transformations.Shaping’s workspace-bounds metric is distinct from correct letter geometry and robot execution.Source: A. Environment, tasks and configurations · p. 3
  • The symmetric task-distance score bands target pairs by coordinate-set difference and archived enclosure and symmetry labels, but is not a measure of execution difficulty.On 16 planning pairs, its Spearman correlation with an assignment-derived operation proxy is 0.919; the coordinate-set term alone gives 0.975.Source: A. Environment, tasks and configurations · p. 3
  • RQ1 reports o1 and o4-mini prompting; RQ2/RQ3 use a Qwen3.5-35B-A3B instruction policy with rank-32 LoRA and a frozen Qwen3-Coder-Next service.Central settings include K=32, M=5, E=3, and 20 training steps.Source: A. Environment, tasks and configurations · p. 3
  • 60 saved o4-mini coordinate outputs met strict workspace bounds; 76/80 plans met coverage and uniqueness, and 38/80 also met maximal reuse.Plans are counted rather than executions.Source: A. Environment, tasks and configurations · p. 3
  • In the five-to-two-node example, all five plans from each model were coverage-feasible; maximal reuse held for 4/5 o1 and 5/5 o4-mini plans.The Hungarian solver and listed model assignments had the same padded objective (0.653), while the quoted real-to-real subtotal measures a different quantity.Source: A. Environment, tasks and configurations · p. 4

B. Evaluation protocol and interpretation of logged rewards

Archived program-output labels can conflict with geometric checks, motivating an independent protocol that separates robot control from success assessment. Prompting comparisons are descriptive and constrained by differing task sets, archived values, and wide uncertainty intervals.

B. Evaluation protocol and interpretation of logged rewards

  • At least 19, 18, and 5 positive labels accompany geometric-failure messages across the three archived runs, while retained records cover only one program per instruction.The figure shows 19 of 1,055 positive labels in the binary M=5 run with geometric-failure messages; the other runs have 18 and 5.Source: B. Evaluation protocol and… · p. 4, Figure 3 · p. 4
  • The independent protocol gives simulator state, fixed task data, and success reporting to a supervisor while generated programs issue control actions.Primary success also requires required disposal, reserve preservation, and clean execution; reserves cannot substitute for task cubes.Source: B. Evaluation protocol and… · p. 4
  • The protocol passed ten negative controls, and a hand-authored controller passed 40 of 40 predeclared scenes; evaluating generated programs remains future work.Re-scoring retained programs would cover one program per instruction, not reconstruct the unsaved samples behind averaged rewards.Source: B. Evaluation protocol and… · p. 4

V. PROMPTING COMPARISONS

  • Table II’s baseline and ICMC rows use different task sets and generation budgets, so their reported outcomes are descriptive rather than matched comparisons.The base ICMC notebook recount gives 14/20 code-valid and 2/20 explicit success messages; the final row recount is 4/20 or 13/20 depending on which saved manual execution is used.Source: Table II · p. 5, §V · p. 5
  • With o4-mini, ICMC records 12/20 against 7/20 for one-shot pick-and-place, while one-shot records 5/20 against 3/20 on reshaping.These outcomes characterize the respective task protocols and configurations; matched conditions are needed to isolate reasoning effects across models.Source: §V · p. 5
  • The task-success intervals for base ICMC, combined ICMC, and Chain-of-Code overlap pairwise, and testing differences between rows remains future work.Their reported counts are 3/20, 10/20, and 4/20, respectively.Source: §V · p. 5

VI. TRAINING THROUGH A FROZEN CODE GENERATOR

The three archived training trajectories show rising logged positive-reward rates, but reward labels can conflict with retained geometric-failure logs. Traced samples expose syntax, naming, and reward-verdict discrepancies between instructions, generated programs, and execution records.

VI. TRAINING THROUGH A FROZEN CODE GENERATOR

  • 31.6% to 35.3% (253/800 to 282/800) was the binary run’s logged positive-reward rate across its first and last five steps.The dense run rose from 30.0% to 33.7%, and the M=1 run from 25.0% to 30.6%; each configuration has one trajectory.Source: §VI · pp. 5–6
  • The five-step M=1 archive record duplicates the run’s first five steps byte-for-byte and is counted once.The traced samples come from step 20; the recorded pick-and-place task and stage-2 request specify the task and instruction-generation context.Source: §VI · p. 6, §VII · p. 6

VII. TRACEABLE INSTRUCTIONS AND FAILURE ANALYSIS

  • A retained zero-reward sample contains a bracket-mismatch SyntaxError, while another contains a NameError for undefined pattern_B.The instruction sample paired with the SyntaxError has five zero rewards; the NameError sample also has five zeros.Source: §VII · p. 6
  • Sample 026 had five ones, but its retained log reports distance 0.9515 m against 0.05 m tolerance and failed placement; the other four programs behind those rewards were not saved.Sample 001 had rewards (0, 1, 1, 0, 0), a retained geometric-success message, and maximum matched distance 0.0005 m; independent task verification is pending.Source: §VII · p. 6
  • Five rewards of 1.0 conflicted with a retained program log reporting a 0.9515 m miss against a 0.05 m tolerance.Only the first reward-maximizing program was retained, so the contradiction is established for that program, not the other four programs behind the rewards.Source: §VII · p. 6
  • The 32 step-20 instruction outputs differ in both prose and code, but their behavioral diversity remains untested.All code interiors and outside-code texts have different hashes; median code-line fraction is 79.3%.Source: §VII · p. 6

VIII. DISCUSSION, LIMITATIONS AND REPRODUCIBILITY

The discussion distinguishes reward-log diagnostics from validated success and describes an independent evaluation protocol alongside its limits. The conclusion reports protocol checks, simulation-scope boundaries, and partial API-cost ledgers.

VIII. DISCUSSION, LIMITATIONS AND REPRODUCIBILITY

  • The audit found 19 positive-label records with reported geometric failure in the binary run and 18 in the dense run.These discrepancies motivate independent task verification rather than treating logged reward increases as validated task-performance gains.Source: §VIII · p. 6
  • The study covers simulated manipulation, a subset of 132 transformations, and text-only RL; held-out prompts, scenes, and hardware remain untested.Hardware transfer would require calibration, perception and state-estimation testing, robustness and latency measurement, and a separate safety review.Source: §VIII · p. 6
  • Reproducibility scripts regenerate recounted values, traced examples, and Fig. 5 without model calls, while also testing geometric matching and protocol checks.Further work includes validating the target host and establishing archived model and checkpoint provenance.Source: §VIII · p. 6
  • Recovered code-generation ledgers total $6.88 for binary M=5, $6.95 for dense M=5, and $1.40 for M=1.These are partial API charges; they exclude local training compute, and the 160 A100-hour and 200 USD aggregates are unreconciled estimates.Source: §VIII · p. 6

IX. CONCLUSION

  • ICMC exposes the intermediate instruction so execution outcomes can be traced to specific instructions and programs.The conclusion pairs this traceability with the need to distinguish logged rates from validated success.Source: §IX · pp. 6–7
  • The CPU protocol passed ten negative controls, and a hand-authored reference controller completed all 40 predeclared scenes.Evaluating model-generated programs under the independent protocol remains future work.Source: §IX · p. 7

APPENDIX A RECOVERED PROMPT COMPONENTS AND A TRACED

Appendix A reproduces archived prompt components and a traced record, preserving original spelling and line ranges. The supplied task strings specify pick-and-place and reshaping requests.

APPENDIX A RECOVERED PROMPT COMPONENTS AND A TRACED

  • The appendix listings reproduce cited archived line ranges without edits, retaining original spelling while wrapping long lines.Source: Appendix A · p. 7

A. Task request strings

B. Stage requests

The archived stage-request components combine task parameters and a reference-code context with instructions to generate logical control guidance. The RL code builder then supplies the sampled instruction to a frozen-code completion request.

B. Stage requests

  • The archived stage-2 template includes a task string, reference code, parameters, and an instruction request; the text-only runs supplied no image.Source: B. Stage requests · p. 7
  • The stage-2 request asks for specific logical control instructions for each motion-control parameter, especially for three core motion-planning functions.Source: B. Stage requests · p. 7
  • The RL stage-3 builder uses the stage-2 template without line 158 as trimmed_prompt and the sampled instruction as logical_suggestions.The denoised driver appends an API addendum and sends one user message.Source: B. Stage requests · p. 7
  • The RL stage-3 builder uses the stage-2 template without line 158 and appends the full sampled instruction as logical_suggestions.Source: B. Stage requests · p. 7
  • The code-completion request asks for completed Python code based on the logical control instructions and says not to change the basic operational-code functions.Source: B. Stage requests · p. 7
  • The RQ1 code-builder request uses a different request and separator, and its original image bytes are absent from the retained archive.Source: B. Stage requests · p. 7

C. API-documentation addendum

The addendum specifies how generated control code must use the simulation API and structure its per-cube control loop. It also standardizes the loaded pattern variable used for target positions.

C. API-documentation addendum

  • env.step() requires the full (N, 8) trajectory array, with time in column 0 and pose [x, y, z, qx, qy, qz, qw] in columns 1–7.The code should pass the full robotic_arm trajectory to env.step(), rather than iterating over rows.Source: C. API-documentation addendum · p. 7
  • The main loop must repeatedly call fetching_cube() and move_to_pattern() until cube_id reaches pattern_B.shape[0].Each call advances middle_step by one phase when terminated==True.Source: C. API-documentation addendum · p. 7
  • Use pattern_B consistently for target positions, replacing pattern in get_target_pos() or defining pattern = pattern_B.The instructions identify pattern_B as the variable loaded from the .txt file.Source: C. API-documentation addendum · p. 7

D. One traced record: step 20, instruction sample 001

The traced step-20 instruction sample has five program rewards averaging 0.4, while the retained sample’s checker output reports correctly placed cubes. The trace also shows the generated main loop and a runner-appended checker sharing the program process.

D. One traced record: step 20, instruction sample 001

  • The five program rewards were [0,1,1,0,0], for a mean of 0.4; only sample 1, the first reward maximum, and its logs were retained.The record is step_020/rollout_001 from the denoised-binary archive.Source: D. One traced record: step 20,… · p. 8
  • The instruction describes fetching red cubes from the back table and placing them in the central disposal area on the right.Its control-flow instructions fill the replacement blocks and call fetching_cube and move_to_pattern sequentially for each cube.Source: D. One traced record: step 20,… · p. 8
  • The runner appended the placement checker after env.close(), and the checker shared the program’s process.The model-written main loop is delimited in lines 122–154 of code.py.Source: D. One traced record: step 20,… · p. 8
  • 0.0005 m was the reported max_match_dist for the retained sample, whose checker output said all cubes were correctly placed.The checker output lists planned positions [[0.4, -0.1], [0.6, -0.1]] and cube positions [[0.40042537, -0.09990171], [0.60047451, -0.09992712]]. Only this sample’s placement evidence was retained; the other four programs and logs were unsaved.Source: D. One traced record: step 20,… · p. 8

E. Provenance boundary

The retained provenance is incomplete: several original training and input artifacts are unavailable, although templates, selected records, and an evidence manifest are retained for release.

E. Provenance boundary

  • The original trainer, chat serializer, any system message, tokenizer version, and RQ1 image bytes are unavailable in the retained archive.Recovered templates, builder source, and outputs are user-text components rather than complete API transcripts.Source: E. Provenance boundary · p. 8
  • Complete templates, four full step-20 records, and a SHA-256 evidence manifest are retained for release.Source: E. Provenance boundary · p. 8

Related papers

Loading 2610.19617v1…