PaperlayerView exact revision on arXiv

Source-linked AI summary

Agentic AI-enabled Semantic Commissioning of a Cognitive Digital Twin for Reconfigurable Manufacturing

Yangyang Liu, Xun Xu, Jan Polzer

arXiv:2609.09503v1cs.ROcs.MA

TL;DR

Bespoke commissioning remains a bottleneck because CDT construction must integrate semantic specifications with live industrial states rather than geometry alone. The paper proposes a LangGraph-orchestrated agentic workflow combining RAG for static documentation and MCP for dynamic telemetry binding, with HITL support. The system achieved 97.2% mAP@0.5 and reduced deployment lead time by 95% to an average of 2 hours, while the authors identify dependence on digital documentation as a limitation.

  • Problem

    Bespoke CDT commissioning remains a severe industrial bottleneck because semantic mapping between heterogeneous physical assets and virtual counterparts is largely manual and time-consuming.

    Source: A. Evolution and Architecture of CDT · p. 2

  • Method

    The paper orchestrates LLM-based agents through LangGraph, combining RAG-based specification extraction, MCP-based telemetry binding, and HITL feedback within a CDT workflow.

    Source: §I · p. 1

  • Results

    97.2% mAP@0.5 and a 95% reduction in deployment lead time were achieved, completing reconfiguration in an average of 2 hours.

    Source: C. Experimental Results and Evaluation · p. 5, §IV · p. 6

  • Takeaways & Limitations

    The results demonstrate feasible autonomous orchestration of high-fidelity, live-synchronised CDT commissioning in a robotic machining cell.

    Source: C. Experimental Results and Evaluation · p. 5, §IV · p. 6

    The RAG-based enrichment assumes that digital documentation is available.

    Source: §IV · p. 6

Abstract

from arXiv · show

Rapid bespoke commissioning of the Cognitive Digital Twin (CDT) is a major challenge in reconfigurable manufacturing. Traditional digital twin (DT) construction methods primarily focus on geometric reconstruction, often neglecting the deep semantic integration and functional interoperability necessary for autonomous reasoning. This paper proposes an agent-based, AI-driven workflow to automate end-to-end CDT debugging. The system utilises LangGraph as a multi-agent orchestration engine to achieve dual-path synthesis: the semantic path extracts technical specifications from unstructured documents using Retrieval Augmented Generation (RAG), while the functional path autonomously discovers and binds to real-time industrial telemetry data using Model Context Protocol (MCP). Experimental validation in a robotic machining cell demonstrates that the system achieves a mean average accuracy (mAP) of 97.2% in perception and reduces the deployment cycle from several weeks to an average of 2 hours, marking a paradigm shift from manual scripting to autonomous orchestration.

I. INTRODUCTION

The paper proposes an agentic workflow for rapidly commissioning a semantically rich, functionally interoperable CDT in reconfigurable manufacturing. It combines perception, document-based semantic enrichment, live telemetry binding, knowledge integration, and human feedback.

  • Existing DT construction often prioritises geometric reconstruction while overlooking semantic enrichment and functional interoperability.A functional CDT must represent technical specifications and connect to live operational states, not merely mirror physical geometry.Source: §I · p. 1
  • The proposed workflow uses vision-based monitoring to detect layout changes and retrieve corresponding 3D models for spatial reconstruction.Source: §I · p. 1
  • RAG extracts static technical parameters from unstructured PDFs, while MCP enables autonomous discovery and subscription to OPC UA telemetry nodes.Together, the two paths provide static semantic enrichment and dynamic state synchronisation.Source: §I · p. 1
  • Extracted spatial, static, and dynamic information is synthesised into a knowledge graph and rendered through a lightweight Three.js interface.Source: §I · p. 1
  • Human-in-the-Loop intervention handles perceptual uncertainty, with feedback stored in persistent memory to improve future autonomy and robustness.Source: §I · p. 1
  • 95% reduction in deployment lead time was reported, reducing commissioning from several weeks to several hours in a physical testbed.The implementation maintained high semantic fidelity while reducing bespoke commissioning time.Source: §I · p. 2

A. Evolution and Architecture of CDT

CDT research has evolved toward cognitive, knowledge-rich models, but bespoke commissioning remains a major barrier to industrial deployment. Semantic mapping is still largely manual and must address heterogeneous data, bespoke configurations, knowledge extraction, and system fragility.

B. LLM-driven Autonomous Agents in Manufacturing

Manufacturing agentic frameworks are progressing from high-level planning toward autonomous low-level execution. Graph-based orchestration, industrial interfaces, and HITL architectures support stateful, interoperable, and auditable workflows.

C. Knowledge Representation and Standardised Interfacing

Knowledge representation and standardised interfaces are central to CDT interoperability. The paper combines established representation standards with RAG and MCP to connect technical specifications and live industrial telemetry.

III. AGENTIC FRAMEWORK REALISATION AND EXPERIMENTAL VALIDATION

The section presents a physical robotic machining cell implementation of the cognitive-function CDT framework and evaluates it through perception and system-integration experiments.

  • The proposed framework is instantiated in a real-world–inspired robotic machining cell using an LLM-driven agentic workflow.Source: §III · p. 2

A. Experimental Setup & Bespoke Commissioning Context

The experimental setup is a reconfigurable machining cell with robots, conveyors, and RGB-D perception, designed to reflect frequent production-layout changes. Its stack combines YOLOv11, LangGraph-based reasoning, and Neo4j semantic memory.

B. The Agentic Bespoke Commissioning Workflow

The workflow treats bespoke commissioning as a dynamic reasoning task: it enriches detected equipment semantically, binds entities to telemetry through MCP, and persists the resulting configuration in a knowledge graph.

Stage I Hybrid Semantic Enrichment

Stage II Protocol Binding via MCP Tool-calling

Stage III HITL and Memory Persistent

C. Experimental Results and Evaluation

The evaluation compares the Agentic workflow with manual CDT commissioning and assesses perception, deployment efficiency, and semantic fidelity in a robotic machining context. The workflow combines visual monitoring with reconstructed and knowledge-enriched digital scenes.

Perception Performance

Commissioning Efficiency

  • The study compares autonomous commissioning with manual 3D asset alignment, parameter entry from manuals, and OPC UA node binding.These activities define the traditional bespoke commissioning baseline.Source: C. Experimental Results and Evaluation · p. 4
  • Several weeks are required for a skilled engineer to configure four major manufacturing entities manually.The manual process includes data binding and spatial configuration.Source: C. Experimental Results and Evaluation · p. 4
  • The Agentic pipeline completes commissioning from raw camera input to live Three.js rendering in an average of 2 hours.The cycle covers the autonomous pipeline through live visualisation.Source: C. Experimental Results and Evaluation · pp. 4–5

Results and Fidelity

  • 95% deployment-lead-time reduction is reported relative to the multi-week manual commissioning process.The generated CDT also integrates RAG-extracted parameters and live industrial data streams into knowledge-enriched virtual entities.Source: C. Experimental Results and Evaluation · p. 5

Visualisation

  • The evaluation materials include a DT Editor view for spatial reconstruction and initial geometric-semantic mapping, plus a live dashboard for MCP-enabled telemetry synchronisation.These views represent the transition from reconstructed scene geometry to live industrial data integration.Source: Figure 5 · p. 5, Figure 6 · p. 5

IV. CONCLUSION & FUTURE WORK

The paper presents a LangGraph-orchestrated, LLM-driven CDT workflow that connects physical perception with live-synchronised digital representations. In a robotic machining cell, it reports high perceptual reliability and substantially shorter deployment lead time, while identifying documentation availability as a current boundary.

  • LangGraph orchestrates a multi-agent CDT workflow using LLM reasoning to bridge physical perception and live-synchronised digital representations.Source: §IV · p. 6
  • The workflow demonstrates autonomous CDT commissioning through YOLOv11 perception, RAG-based static-parameter extraction, and MCP-based dynamic industrial binding.Source: §IV · p. 6
  • The RAG enrichment currently assumes that digital documentation is available.Future work targets multimodal MLLMs for legacy blueprints and hand-drawn diagrams, and a distributed KG for cross-factory learning.Source: §IV · p. 6

Related papers

Loading 2609.09503v1…