Source-linked AI summary
NDT Factory: Synthesizing Verified Network Digital Twins from Semantic Models via Multi-Agent LLM
Sudipta Acharya, Petar Djukic, Burak Kantarci
TL;DR
Behavioral NDTs rely on predefined analytical logic, requiring manual engineering as NSI constraints evolve. The NDT factory synthesizes and validates executable NDTs from semantic models, then reuses cached executables. In a CAC evaluation, it achieved complete compilation and test success and closely agreed with a reference implementation across 300 NSIs.
Problem
Existing behavioral NDTs rely on predefined analysis logic tied to NSI constraint semantics, while evolving intent types can require repeated manual engineering.
Source: §I · p. 1Method
The NDT factory uses semantic models and coordinated LLM-based synthesis to generate, compile, test, and cache executable behavioral NDTs.
Source: §I · p. 1, §III · p. 3Results
The CAC NDT achieved 100% compilation and test success across multiple runs, 99.3% decision agreement on 300 NSIs, and a 90% admission rate.
Source: §VI · p. 6Takeaways & Limitations
The CAC evaluation shows that the synthesized NDT can provide deterministic, verifiable execution with close decision agreement to a reference implementation.
Source: §VI · p. 6The reported evaluation is a CAC case study; systematic LLM-backend evaluation and extension toward 5G RAN scenarios remain ongoing work.
Source: §I · p. 1, §VI · p. 6
Abstract
from arXiv · showhide
Autonomous network management requires systems that can evaluate Network Service Intents (NSIs) under varying conditions without manual implementation of analysis logic, as envisioned in TM Forum Level~4 (L4) autonomy. Behavioral Network Digital Twins (NDTs) enable such evaluation, but existing NDTs rely on pre-defined analytical logic, limiting adaptability for evolving closed-loop control. This paper introduces the NDT factory, a multi-agent software system that synthesizes executable behavioral NDTs on demand from semantic models using Large Language Model (LLM). We validate the system using a Call Admission Control (CAC) case study, where deterministic what-if analysis serves as the admission decision process. The NDT factory generates a complete CAC NDT through parallel synthesis and orchestration, achieving 100% compilation and test pass rates across multiple runs. Simulation over 300 NSIs shows 99.3% decision agreement with a reference implementation, 90% admission rate, and correct attribution of all rejections, demonstrating reliable synthesis with deterministic, verifiable execution.
I. INTRODUCTION
Behavioral NDTs can evaluate NSIs against network state, but their predefined analysis logic requires manual engineering as intent constraints evolve. The NDT factory instead synthesizes executable NDTs from semantic models, distinguishing its semantics-driven approach from prior scenario-specific NDT analysis and prompt-driven code generation.
I. INTRODUCTION
- Existing behavioral NDTs use predefined, design-time logic tied to NSI constraint semantics, so new constraint types or policies can require repeated manual engineering.This coupling limits scalable closed-loop L4 autonomy and motivates synthesis from declarative specifications.Source: §I · p. 1
- The NDT factory uses semantic models to synthesize executable behavioral NDTs on demand, validates them with compile–test–debug, and caches them for deterministic runtime reuse.The semantic model specifies inputs, operations, constraints, and validation cases without prescribing source-code implementation.Source: §I · p. 1, Figure 1 · p. 2
II. RELATED WORK AND MOTIVATION
- Prior NDT studies emphasize architectures, analytics, and scenario-specific simulation, while on-demand synthesis of executable logic for unseen intent types remains an open opportunity.The paper builds on the digital-twin-factory vision in Djukic et al. with an executable multi-agent pipeline.Source: §II · pp. 1–2
- Unlike prompt-driven code-generation work, the approach makes explicit behavioral semantics the primary driver and restricts the LLM to synthesis while compiled executables perform runtime analysis.The semantic models specify domain inputs, operations, constraints, output conventions, and validation tests.Source: §II · p. 2
III. THE PROPOSED SYSTEM: NDT FACTORY
The NDT factory separates NSI handling, coordination, planning, and code synthesis across agents. It generates and verifies an NDT when needed, then caches the executable for deterministic reuse.
III. THE PROPOSED SYSTEM: NDT FACTORY
- The system has an Intent handler, NDT factory agent, Planner, and parallel Coding agents, with generation and execution as its two operating modes.In execution mode, the cached NDT produces decisions; in generation mode, agents synthesize and assemble the pipeline.Source: §III · p. 2
- The Intent handler validates each NSI and converts it into a structured behavioral query; the factory agent looks up or generates the required NDT and gathers runtime network and service state.The factory agent combines the query with collected metadata, invokes the executable, and handles state updates and resource release.Source: §III · pp. 2–3
- The Planner extracts component dependencies from the orchestrator model and distributes independent components to Coding agents for parallel synthesis and compilation.It assembles binary interfaces, data-flow contracts, and pipeline order as context for generating the orchestrator, then assembles and verifies the complete pipeline.Source: §III · p. 3
- A Coding agent builds prompts from semantic models, generates code with tool support, and retries with compiler or test feedback for up to four iterations.A component is accepted only if it compiles and passes all semantic-model-defined tests.Source: §III · p. 3
- On a cache miss, the Planner dispatches parallel component synthesis, assembles context, and has a Coding agent synthesize the orchestrator before the pipeline is verified and cached.For later NSIs requiring the same analysis, the cached NDT executes without LLM involvement.Source: §III · p. 3
- The semantic model is a verifiable YAML specification of intended NDT behavior, including inputs, data structures, operations, constraints, output conventions, and test cases.Domain specialists define the logic and constraints; system engineers encode them as specifications and validation cases.Source: §III · p. 3
IV. CASE STUDY
The CAC case study uses deterministic what-if analysis to evaluate an incoming NSI against projected network and service conditions.
IV. CASE STUDY
- The CAC NDT projects the post-admission network state for each incoming NSI using a deterministic what-if algorithm.It evaluates bandwidth availability, link utilization, end-to-end latency, and existing-service SLAs.Source: §IV · p. 4
- The admission decision requires all evaluated constraints to hold; otherwise, the request is rejected with per-constraint reasoning.The checks include network and existing-service conditions.Source: §IV · p. 4
- The case study applies the analysis to call admission control in packet-switched networks.The evaluated constraints cover bandwidth, utilization, latency, and existing-service SLAs.Source: §IV · p. 4
A. Problem Formulation
The formulation models a network and its active services, then evaluates candidate paths for an incoming request. It selects a feasible path by a combined latency-utilization cost or rejects the request when none is feasible.
A. Problem Formulation
- The network is a bidirected graph with link capacities, propagation delays, and utilization, while active services record paths, reserved bandwidth, and latency SLAs.Link service rates use an M/M/1 model with mean packet size 1 kbit and 1 Mbps equal to 1 kbit/ms.Source: A. Problem Formulation · p. 4
- Each NSI specifies source, destination, required bandwidth, latency bound, and duration; the operator sets a network-wide link-utilization threshold.The formulation uses K candidate paths between the request’s source and destination.Source: A. Problem Formulation · p. 4
- A candidate path must satisfy residual-capacity and post-admission utilization constraints, and its latency is computed from link delays.Link delay combines propagation delay with M/M/1 mean queueing delay; existing services’ projected latencies are also considered.Source: A. Problem Formulation · p. 4
- The decision selects the feasible path minimizing a composite normalized-latency and utilization cost, or rejects the request if the feasible set is empty.The cost weights latency and maximum post-admission utilization, with lower values preferred.Source: A. Problem Formulation · p. 4
B. Underlying Deterministic what-if Analysis for CAC
The CAC what-if analysis evaluates candidate paths against network and service constraints before selecting an admissible path and updating state. Its logic is decomposed into six sub-components and an orchestrator.
B. Underlying Deterministic what-if Analysis for CAC
- The algorithm prunes the topology, enumerates candidate paths, filters them against constraints, scores feasible paths, and updates network state after selection.The filtering checks utilization, latency, and existing-service SLA constraints; if no path remains, it rejects the request with reasons.Source: B. Underlying Deterministic what-if… · p. 4
- The CAC logic is decomposed into six sub-components and one orchestrator, represented by semantic models.The formulation uses rate-based utilization and M/M/1 queueing-delay models.Source: B. Underlying Deterministic what-if… · p. 4
- Go is used for the case study, although the NDT factory is language-agnostic.The stated reasons for Go are fast compilation, static typing, and self-contained binaries.Source: B. Underlying Deterministic what-if… · p. 4
V. EXPERIMENTAL EVALUATION
The primary experiments use Claude Sonnet 4.6 as the Coding-agent backend, while the system is architecturally LLM-backend-agnostic.
V. EXPERIMENTAL EVALUATION
- Claude Sonnet 4.6 is the Coding-agent backend for the primary experiments; Qwen3-Coder-30B-A3B-Instruct is used to assess open-source-backend sensitivity.The system is described as architecturally LLM-backend-agnostic.Source: §V · p. 4
1) Metrics:
The evaluation measures generated-code compilation, test performance, code quality, and semantic similarity, with a weighted correctness score. The CAC component table also reports code size, algorithmic complexity, and synthesis cost.
1) Metrics:
- Table I covers semantic models, generated-code size, algorithmic complexity, and synthesis cost in tokens and latency, reported as mean ± standard deviation over 10 runs.Its complexity notation defines N as nodes, M as links, K as candidate paths, |P| as path length, and |S| as active services.Source: Table I · p. 5
- The overall correctness score combines compilation success, test pass rate, code quality, and semantic similarity using weights of 0.30, 0.50, 0.15, and 0.05, respectively.The weights prioritize functional correctness over structural quality and semantic alignment.Source: 1) Metrics · p. 5
- Code quality is scored through Go AST features, while semantic similarity compares JSON fields, data structures, operations, argument style, and computation logic.The similarity weights are 30%, 20%, 20%, 15%, and 15%, respectively.Source: 1) Metrics · p. 5
2) Results:
Across repeated synthesis runs, the generated CAC sub-components compile and pass all semantic-model-defined tests. The reported synthesis costs rise across the evaluated component-complexity range.
2) Results:
- All six sub-components compile and pass every semantic-model-defined test across 10 runs, establishing test-defined correctness.Compilation and test outcomes remain invariant across runs, although LLM attempts vary slightly.Source: 2) Results · p. 5
- Code quality and semantic similarity remain stable across runs, with semantic similarity varying by only ±0.01–0.02.The path-finding component has slightly greater variance because Yen’s K-shortest-path algorithm is structurally more demanding to synthesize.Source: 2) Results · p. 5
- Token usage rises from 3,059 for the Bandwidth Pruner to 8,813 for the Orchestrator (2.9×), while latency rises from 54.3 to 189.8 s (3.5×).The paper characterizes the synthesis-cost increase as moderate across the evaluated complexity range.Source: 2) Results · p. 5
3) LLM-backend sensitivity:
The backend evaluation compares two LLMs across six sub-component semantic models, while the CAC simulation scenario tests 300 NSIs against a handcrafted reference. Qwen’s synthesis reliability varied substantially by component.
3) LLM-backend sensitivity:
- Claude Sonnet 4.6 succeeded in 60/60 runs, while Qwen produced compiling, test-passing implementations in 41/60 runs (68.3%).Qwen succeeded in 10/10 Dynamic Service Catalog runs but 0/10 Graph Path Finder runs; its per-model mean latency ranged from 86–488 s, versus 52–139 s for Claude Sonnet 4.6.Source: 3) LLM-backend sensitivity · p. 5
- Qwen’s Graph Path Finder generated only one of two required distinct shortest paths, preventing orchestrator and complete CAC NDT synthesis.The paper concludes that synthesis reliability strongly depends on model capability, although the architecture supports different LLM backends.Source: 3) LLM-backend sensitivity · p. 5
1) Simulation scenario:
- The simulation routes NSIs through a time-ordered priority queue and compares both twins on the same arrival sequence and initial network and service states.It covers 300 NSIs over 490.5 s on 14-node, 21-link NSFNet, with Poisson arrivals and a 33 s mean service duration; latency is prioritized with α = 0.6.Source: 1) Simulation scenario · p. 5
2) Results:
Across 300 NSIs, the synthesized CAC NDT achieved 99.3% decision agreement with the reference, with a 90% admission rate. The conclusion reports 100% compilation and test success across multiple runs and identifies further validation and extensions as ongoing work.
2) Results:
- Peak max-link utilization was 54.6% for the synthesized NDT and 56.9% for the reference, both below the 80% limit.The moderate-load scenario did not exercise the C2 boundary, motivating higher-load validation.Source: Figure 4 · p. 6, 2) Results · p. 6
- Every admitted NSI satisfied constraints C1–C4, while all rejections in both twins arose solely from C3.The two decision disagreements were latency-boundary cases, each exceeding the permitted latency by less than 1%.Source: 2) Results · p. 6
- Across 6,300 link-utilization comparisons, mean absolute error was 0.7 percentage points and maximum error was 24.4 points.The passage attributes most divergence to numerical tiebreaking: 22 of 271 jointly admitted requests selected different but equally admissible paths.Source: 2) Results · p. 6
- The synthesized CAC NDT achieved 99.3% decision agreement and a 90% admission rate across 300 NSIs, with 100% compilation and test success across multiple runs.The agreement corresponds to 298 matching admission decisions against the handcrafted Python reference.Source: 2) Results · p. 5, §VI · p. 6
VI. CONCLUSION
- The approach may extend to QoS enforcement, fault localization, and traffic engineering when these functions can be expressed as semantic models and integrated into the same pipeline.Ongoing work includes systematic backend evaluation, semantic-model refinement through execution feedback, dynamic intent classification, packet-level simulation, and extension toward 5G RAN scenarios.Source: §VI · p. 6