Source-linked AI summary
Retrieval-grounded robot program generation and simulation-based correction via Model Context Protocol
Zhichao Zhou, Siyuan Chen, Omkar Salunkhe, Ebru Turanoglu Bekar, Johan Stahre, Anders Skoogh
TL;DR
Industrial robot reprogramming remains difficult because domain-specific code must also behave correctly in a configured cell. The paper combines dual-stream RAG with MCP-connected RobotStudio execution and correction, showing that simulation feedback reveals failures missed by text-level checks while reducing, not eliminating, expert involvement.
Problem
Rapid product changes require robot reprogramming, but reliable industrial programs demand specialist knowledge and execution validation beyond natural-language code generation.
Method
The workflow grounds ABB RAPID generation in technical documentation and production templates, then uses MCP to upload, simulate, diagnose, and iteratively correct programs in RobotStudio.
Results
The RobotStudio cases showed that execution feedback exposed release-height, placement, and recovery-motion failures that static and semantic checks could miss.
Takeaways & Limitations
Grounded generation becomes more useful for industrial robot programming when executable simulator feedback is incorporated into the correction loop.
Takeaways & Limitations
The evaluation is preliminary, limited to one RobotStudio cell and few tasks, with safety PLCs, welding constraints, multi-robot coordination, and other platforms unvalidated.
Abstract
from arXiv · showhide
Flexible manufacturing requires industrial robots to be reprogrammed rapidly as product variants change. This paper presents a language-model-based workflow that generates, validates, and iteratively corrects ABB RAPID robot programs from natural language task descriptions. A dual-stream retrieval-augmented generation (RAG) pipeline grounds code generation in verified technical documentation and production templates, reducing domain-specific errors produced by ungrounded language models. A custom Model Context Protocol (MCP) server connects the language-model client directly to ABB RobotStudio for automated code upload, simulation execution, and diagnostic feedback. The evaluation combines a 30-query retrieval benchmark, scoped code-generation checks, and RobotStudio case studies in a simulated pickand- place manufacturing cell. The simulation loop exposes execution failures that static and semantic checks alone cannot catch, including suction release-height errors, unreachable placement targets, and configuration-dependent recovery motions. The results show how RAG and MCP can connect grounded code generation with executable feedback from industrial robot simulation software, while reducing but not eliminating expert setup and final supervision.
1. Introduction
Flexible manufacturing requires rapid robot reprogramming, but specialist programming and execution validation remain bottlenecks. The paper addresses this gap with retrieval-grounded ABB RAPID generation connected to RobotStudio feedback for iterative correction.
- Motivation: Frequent product changes make repeated manual robot programming a practical flexibility bottleneck in high-mix manufacturing.Robot behavior must be programmed, checked against cell layout, and validated before production use.
- Motivation: LLMs can translate natural-language intent into robot control logic, but reliability and domain specificity remain limited.RAG can reduce hallucinated instructions by grounding generation in manuals and validated examples.
- Problem: Textually plausible robot programs can still fail because reachability, path geometry, gripper behavior, I/O state, and recovery depend on the configured cell.This creates a gap between grounded code generation and execution-level validation.
- Approach: The workflow combines dual-stream RAG for ABB RAPID generation with an MCP connection to RobotStudio for upload, simulation, monitoring, and correction.Feedback includes execution status, event logs, joint readings, I/O signals, scene information, and program variables.
- Contribution: The paper evaluates retrieval, generation quality, and RobotStudio case studies to show how execution feedback reveals failures missed by text-level checks.Its contributions include a RobotStudio-connected workflow, dual-stream RAG evaluation, and execution-based failure evidence.
2. Literature Review
Prior work supports robot-program synthesis, offline simulation, retrieval grounding, and tool-using LLMs, but leaves a validation gap between executable-looking code and behavior in industrial cells. This paper closes that gap by connecting generation to RobotStudio execution feedback.
- Industrial programming: Industrial robot programming remains constrained by proprietary controller languages and closed platform ecosystems.ABB systems use RAPID modules for motion, workobjects, tool data, I/O behavior, and execution routines.
- Simulation and validation: Offline simulation can expose unreachable motions, singularities, invalid geometry, and failed end-effector release that structural checks miss.This motivates acceptance testing based on system behavior rather than program structure alone.
- Retrieval grounding: RAG supplies specialized manuals and examples at inference time to reduce hallucinated technical details in robot code generation.Retrieval quality matters because relevant material must enter context early enough to influence generation.
- Tool use: Tool-using LLM workflows combine retrieval, typed tool calls, and iterative correction, while MCP standardizes access to external tools and data.In this paper, MCP provides the interface between the language-model client and RobotStudio.
- Research gap: The paper’s novelty is connecting LLM-generated RAPID programs to RobotStudio feedback for diagnosing and correcting execution failures.Controller state, event logs, I/O signals, RAPID variables, and scene objects provide execution-level evidence.
3. Proposed Workflow
The proposed workflow separates grounded RAPID planning from RobotStudio validation and execution. Dual retrieval, structured prompting, staged checks, and MCP diagnostics support iterative program correction.
- Architecture: The architecture has a retrieval-and-planner layer and a validation-and-execution layer connected through RobotStudio.Figure 1 summarizes these two connected layers.
- Retrieval: Documentation retrieval covers RAPID semantics and data types, while 106 validated templates show how instructions are combined in production-style code.The two streams answer different questions: what instructions mean and how they are assembled.
- Retrieval: Query decomposition retrieves targeted material for motion rules, I/O patterns, tools, workobjects, and error handling before structured prompt assembly.The prompt requires a complete RAPID module, explicit entry procedure, balanced delimiters, no unresolved placeholders, and specified object names.
- MCP integration: An MCP bridge translates language-model tool calls into RobotStudio operations through a local HTTP API and virtual-controller interfaces.The bridge exposes RobotStudio state and execution operations as callable tools.
- Validation loop: Validation proceeds from structural and semantic checks to RobotStudio upload, execution, and controller-feedback retrieval.Structural checks inspect delimiters and placeholders; semantic checks inspect requested objects, tools, signals, and motion intent.
- Correction: Execution failures trigger revisions using controller status, event logs, joint readings, I/O values, and scene-object geometry.Scene introspection can replace hardcoded assumptions by reading workobjects and object bounding boxes.
4. Experimental Setup
The evaluation combines retrieval benchmarking, code-generation checks, and RobotStudio cases at distinct scopes. The cases test whether simulation feedback exposes failures that syntax and semantic checks do not detect.
- Evaluation design: The evaluation uses retrieval metrics, generation-quality checks, and RobotStudio execution cases with scope-specific units of analysis.Retrieval compares configurations over 30 queries, while code and simulation analyses use logged generation sets and case iterations.
- Evaluation design: 30 queries cover pick-and-place, pallet flow, soldering, signal processing, and error-recovery logic, while 21 trials assess RAPID module quality.The generation categories include path following, pick and place, gripper libraries, motion, arc welding, tool definition, and target programming.
- RobotStudio cases: The conditional pick-and-place case tests suction release for boxes of different heights despite compilable code and reachable targets.Its critical failure is execution behavior rather than syntax.
- RobotStudio cases: The product-variant case runs two 14-block pyramid tasks with different box heights and pallets to test geometry and configuration constraints.The green variant was twice as tall as the orange variant and used a different pallet.
- Comparisons: The study compares a pure LLM baseline, naive dual-stream RAG, and a full pipeline with decomposition, semantic review, and MCP simulation feedback.Metrics target retrieval relevance, structural and content validity, validation issues, and generate-test-correct outcomes.
5. Results
Dual-stream retrieval improved RAPID retrieval and scoped generation checks, while RobotStudio simulation exposed execution failures that text-level validation missed. Iterative corrections resolved release geometry, reachability, and joint-configuration problems in simulated product-variant tasks.
- Semantic validation: 20% content pass rate was achieved by the pure LLM baseline, with hallucinated instructions, invalid signal declarations, and non-standard RAPID syntax among typical failures.Only one of five baseline modules passed content validation.
- Semantic validation: All 21 retrieval-grounded modules passed structural validation, while I/O instruction coverage reached 50% in the full pipeline versus 36% in naive RAG.Semantic review still flagged 4 of 10 full-pipeline modules for correction, so these were scoped quality checks rather than a statistically balanced benchmark.
- Release-height validation in RobotStudio: The suction release failure persisted despite structurally valid code and reachable targets because the +40 mm release offset left the TCP inside the taller green box.The corrected offset was +140 mm, accounting for the 100 mm height difference between the variants.
- Execution feedback: MCP closed the validation loop by uploading generated programs, executing them in RobotStudio, and returning controller, I/O, joint, and scene diagnostics for correction.The simulation environment exposed cell-constraint failures beyond static and semantic code checks.
- Product-variant reconfiguration: Scene introspection enabled the orange 14-block pyramid to succeed on its first attempt, whereas the green variant required reach adjustments and joint-configuration recovery before all 14 blocks were placed.The successful recovery used MoveAbsJ to reach a calibration pose, then MoveJ to return to the named home pose without forcing the fifth axis toward its stored high-angle pose.
6. Discussion
The workflow extends retrieval-grounded robot code generation with executable RobotStudio feedback, allowing failures missed by text-level checks to inform correction. Its scope remains bounded by prepared stations, curated resources, limited evaluation, and human confirmation.
- Execution-grounded correction: RobotStudio feedback is treated as part of generation rather than only a final manual check, connecting code correction to controller and station behavior.The workflow exposes event logs, joint readings, I/O values, RAPID variables, and scene-object data through MCP tools.
- Execution-grounded correction: RAG reduces invalid RAPID constructs, but execution is still needed to detect infeasible release heights, placement poses, and recovery motions.The release-height and pyramid cases reveal failures that retrieval alone cannot determine from domain documentation or templates.
- Human involvement: The reported system reduces manual program debugging rather than autonomously engineering the robot cell.Station design, tools, workobjects, I/O mapping, Smart Components, documentation, templates, prompts, and task descriptions were prepared or curated beforehand.
- Scope and limitations: The evaluation is preliminary: it uses limited tasks and one RobotStudio cell, includes five pure-LLM trials and 10 semantic-review modules, and reports no significance test.The authors frame the RobotStudio evidence as case-based mechanism validation rather than a broad benchmark.
- Scope and limitations: The bridge is platform-specific and does not yet persist learned corrections across sessions or validate safety PLC integration, welding constraints, or multi-robot coordination.Applying the approach to KUKA, FANUC, or PLC environments would require corresponding MCP implementations.
7. Conclusion
The paper combines retrieval-grounded ABB RAPID generation with MCP access to RobotStudio for simulation-based validation and correction. Results indicate fewer domain-specific generation errors and show that execution feedback exposes failures missed by textual checks, while prepared stations and expert supervision remain necessary.
- Conclusion: Combining technical documentation with production-style templates improves access to relevant RAPID knowledge and yields fewer domain-specific errors than an ungrounded LLM baseline.The conclusion reports this result across retrieval and generation checks.
- Conclusion: RobotStudio cases expose failures caused by release geometry, unreachable targets, and unsafe recovery motions despite textual plausibility.These cases demonstrate the added value of execution feedback during validation.
- Conclusion: MCP connects natural-language task descriptions, grounded RAPID generation, simulation execution, and diagnostic correction in one workflow.RobotStudio operations are exposed as callable tools for the correction loop.
- Conclusion: The approach does not remove the need for a prepared station or expert supervision, but it offers a route toward faster industrial-cell reprogramming and debugging.The supported scope is reduced manual effort rather than fully autonomous cell engineering.