PaperlayerView exact revision on arXiv

Source-linked AI summary

Sequence Models for Layer-3 Protocol Emulation

Alix Jeannerot, Petko Petkov, Alvaro Valcarce Rial

arXiv:2609.37697v1cs.NI

TL;DR

Layer-3 processing time budgets range from 10 ms to 2 s, and distant-datacenter network RTT can often exceed that budget even if the model is instantaneous. The RSE treats RRC as a domain-specific language, using a sequence model for protocol-dependent message structure and placeholders to reserve configured or sensitive fields for deterministic handling. 0.90 EM and 115 ms median TTLT: the 11M-parameter Mamba engine outperforms fine-tuned Qwen3-0.6B, which reaches 0.82 EM and 610 ms median TTLT.

  • Problem

    Layer-3 processing time budgets range from 10 ms to 2 s, and distant-datacenter network RTT can often exceed that budget even if the model is instantaneous.

    Source: A. Compact Sequence Engines · p. 3

  • Method

    The RSE treats RRC as a domain-specific language, using a sequence model for protocol-dependent message structure and placeholders to reserve configured or sensitive fields for deterministic handling.

    Source: §I · p. 1, A. Compact Sequence Engines · p. 4

  • Results

    0.90 EM and 115 ms median TTLT: the 11M-parameter Mamba engine outperforms fine-tuned Qwen3-0.6B, which reaches 0.82 EM and 610 ms median TTLT.

    Source: C. Fine-tuning a compact transformer · p. 5, D. A minimal Mamba-based sequence engine · p. 6

  • Takeaways & Limitations

    RSEs point toward a trainable control plane that could be updated with model releases and specialized to deployments.

    Source: §VI · p. 6

    The 10 ms target is not met, and the authors identify validation, latency, robustness, and security challenges before operational use.

    Source: §VI · p. 6

Abstract

from arXiv · show

This article investigates whether Layer-3 radio-protocol behavior can be represented by compact sequence models suitable for deployment inside the RAN. We introduce the RRC sequence engine (RSE), a hybrid architecture in which a sequence model predicts protocol-dependent message structure while deterministic components retain control over security-sensitive or configured fields and over transport containers. Using NR protocol traces, we show that protocol-aware tokenization, cross-stack context, and explicit placeholders matter more than general-sequence model scale. A purpose-built 11M-parameter Mamba engine achieves 0.90 exact match on the evaluated next-generation Node B (gNB)-side test set with a median generation latency of 115 ms, sufficient for some timers of Layer-3, outperforming a fine-tuned 0.6B-parameter model. We discuss applications in testing, simulation, deployment specialization, and future trainable 6G control planes, as well as the validation, latency, robustness, and security challenges that remain before operational use.

I. INTRODUCTION

RRC’s accumulated procedures, slow evolution, and vendor-specific interpretations make implementation and interoperability costly. This article treats RRC as a domain-specific language and develops sequence engines that aim to improve emulation speed and reliability.

I. INTRODUCTION

  • Standardization accumulates mandatory and optional procedures, slows protocol evolution, and limits deployment-specific customization.Only a fraction of standardized procedures are effectively implemented and productized.Source: §I · p. 1
  • Vendor-specific interpretations make stacks difficult to compare, test, or replace, increasing interoperability-certification costs.Source: §I · p. 1
  • A prior LLM partially emulates RRC from traces, but its seconds-long inference and uncertain message correctness remain impractical for Layer 3.Source: §I · p. 1
  • The RSE treats RRC as a domain-specific language and uses compact sequence models with protocol-aware tokenization and curated inputs and outputs.The model predicts message content while deterministic components retain control over operator-provisioned or security-sensitive fields.Source: §I · p. 1

II. RRC SEQUENCE ENGINES: MOTIVATION AND EXISTING WORK

RSEs are proposed as trace-based alternatives for troubleshooting, testing, simulation, and deployment-specific protocol behavior. The longer-term vision is to make control-plane behavior trainable under deployment constraints.

A. Motivation

  • Trace replay can help isolate whether an anomaly originates in RRC logic or interactions with neighboring protocols, and support what-if analysis without a full network simulation or live-equipment changes.Source: A. Motivation · pp. 1–2
  • A vendor-trace-trained RSE can act as a black-box peer for interoperability testing without requiring a matching commercial stack.Source: A. Motivation · p. 2
  • An RSE trained on real traces could add Layer-3 protocol fidelity to simulators while avoiding the engineering cost of implementing the standard.The trade-off suits research and early integration testing, where realistic exchanged messages matter more than complete procedure coverage.Source: A. Motivation · p. 2
  • Fine-tuning or distilling on deployment traces can yield a lean implementation limited to observed procedures, reducing complexity and resource consumption.The passage gives fixed wireless access as an example where unobserved handover support can be omitted.Source: A. Motivation · p. 2
  • UE- and gNB-side RSEs could form a trainable closed loop whose behavior is optimized under deployment constraints, potentially discovering more efficient signaling.Source: A. Motivation · p. 2

B. Related Work

Prior cellular-network language-model work either analyzes traces or applies models in-network, but direct RRC emulation still faces speed, accuracy, and context limitations. This work targets those gaps with compact engines and protocol-aware inputs.

B. Related Work

  • Analytical models explain traces or answer questions about standards but do not emit the protocol messages exchanged by a network.Source: B. Related Work · p. 2
  • A fine-tuned Llama RSE reproduced control-plane procedures from multi-vendor traces, yet took several seconds, cosine similarity did not ensure message accuracy, and it lacked neighboring-protocol context.Source: B. Related Work · p. 2
  • The article addresses these gaps with compact RSEs, a domain-adapted tokenizer, placeholders for unpredictable fields, and cross-protocol context.Source: B. Related Work · p. 2

A. Where the Sequence Engine Sits

The RSE is positioned in the gNB central unit control plane and does not replace lower network layers or message transport. Training examples pair protocol history with the message actually sent, with different available context on the gNB and UE sides.

A. Where the Sequence Engine Sits

B. From Traces to Conversations

  • Before modeling, ASN.1 bit strings are decoded into human-readable JSON that preserves message nesting and facilitates debugging.Source: B. From Traces to Conversations · p. 3
  • Sessions are recast as multi-turn conversations: gNB inputs include RRC history and preceding external messages, and the target is the sent downlink RRC message.The UE-side version reverses uplink and downlink roles and omits external messages; whole-history question-answer training avoids hand-written state machines.Source: B. From Traces to Conversations · p. 3

IV. COMPACT SEQUENCE ENGINES: FINE-TUNING VERSUS TRAINING FROM SCRATCH

Compact RSEs must meet Layer-3 timing and resource constraints while learning protocol logic rather than configured or opaque message content. The design uses protocol-aware tokenization, external-protocol context, and placeholders to constrain what the model predicts.

A. Compact Sequence Engines

  • Layer-3 processing budgets span 10 ms to 2 s, while gNB-side and UE-side compute may be shared or constrained by device resources and battery.For example, an RrcSetupComplete is expected within 10 ms; UE devices have limited compute and finite battery.Source: A. Compact Sequence Engines · p. 3
  • 138 tokens with the base tokenizer fell to 41 with an RRC-augmented tokenizer for the example message.The shorter sequences reduce generation work; long RrcReconfiguration messages can otherwise span thousands of tokens.Source: A. Compact Sequence Engines · p. 3
  • RRC history alone may not signal a response triggered by another protocol, so external messages such as an AMF release command provide decision context.The command may lead the gNB to send RrcRelease even when preceding RRC history contains little indication that release is imminent.Source: A. Compact Sequence Engines · p. 3
  • Figure 2 depicts preprocessing the received message, applying a chat template and optional external messages, then restoring placeholders before encoding and sending the response.The pipeline replaces information not needed for RRC logic with placeholders before generation.Source: Figure 2 · p. 4, A. Compact Sequence Engines · p. 4
  • Opaque containers and exogenous IEs are replaced by placeholders and later populated from operator or manufacturer configuration, leaving the RSE to learn endogenous protocol logic.This keeps sensitive or unpredictable values outside the model’s decision boundary, including ciphering-algorithm selection.Source: A. Compact Sequence Engines · p. 4

B. Fine-Tuning a Foundation Model

The fine-tuning route starts from a small pre-trained model because the RSE task is specialized. A 0.6-billion-parameter Qwen3 model is trained on the prepared conversations.

B. Fine-Tuning a Foundation Model

  • The fine-tuning route uses a 0.6-billion-parameter Qwen3 model [3] trained on the conversations described in Section III.The authors begin with the smallest candidate they consider capable, guided by latency requirements.Source: B. Fine-Tuning a Foundation Model · p. 4
  • A pre-trained LLM already handles nested brace-delimited syntax and long-range dependencies, leaving fine-tuning to teach protocol logic.The authors frame fine-tuning as a direct route to obtaining an RSE.Source: B. Fine-Tuning a Foundation Model · p. 4
  • Because the RSE performs a highly specific task, much of a general-purpose model’s language ability may be unnecessary.This motivates asking whether a model designed solely for the task can be smaller.Source: B. Fine-Tuning a Foundation Model · p. 4

C. Training from Scratch

Training from scratch offers control over model size for RRC’s small, structured vocabulary. The approach pairs a compact embedding with Mamba’s linear sequence complexity to handle long protocol histories.

C. Training from Scratch

  • A word-based tokenizer keeps the RRC vocabulary compact because its standard-defined terms have no synonyms, helping keep the embedding dimension and parameter count small.Each vocabulary token has a dense connection into the embedding dimension, whose width propagates to later layers.Source: C. Training from Scratch · p. 4
  • Self-attention scales quadratically with sequence length, a costly fit for long RRC histories and messages that can span thousands of tokens.A single procedure may also require context from other control-plane protocols.Source: C. Training from Scratch · p. 4
  • Mamba maintains a selectively updated recurrent state, giving linear time and memory complexity in sequence length.Cached prior states avoid recomputing the full context at each autoregressive generation step.Source: C. Training from Scratch · pp. 4–5

V. RESULTS

The evaluation uses held-out gNB-side and UE-side protocol messages to assess correctness, syntactic validity, and inference latency. Exact match is the primary correctness measure, alongside JSON validity and median time to last token.

A. Dataset and evaluation method

  • The corpus contains 5,047 gNB-side sessions and 4,009 UE-side sessions, with contextual external protocol messages in the gNB-side sessions.Random 60/20/20% train-validation-test splits yield 14,744 gNB and 9,775 UE test messages.Source: A. Dataset and evaluation method · p. 5

B. Metrics

  • Correctness is primarily measured by exact match, the fraction of generated messages identical to references, because small structural errors can have highly variable protocol impact.The authors regard BLEU and cosine similarity as too permissive for protocol emulation.Source: B. Metrics · p. 5
  • Syntactic validity measures the fraction of outputs that form valid JSON, but does not indicate whether their content matches the reference.Exact match assesses that content agreement.Source: B. Metrics · p. 5
  • Latency is the median time to last token over sequential test-set inference with batch size one.Source: B. Metrics · p. 5

C. Fine-tuning a compact transformer

The compact Qwen reaches gNB-side accuracy with the full curated pipeline, while external-protocol context and UE-side evaluation show how context and direction relate to performance.

C. Fine-tuning a compact transformer

  • EM 0.82 and syntactic validity 0.92 on gNB, at median TTLT 610 ms, are the full-pipeline Qwen3-0.6B results; latency remains above the 10 ms target.The model is more than three times faster than the Llama-3.2 1B baseline of (2093 ms).Source: C. Fine-tuning a compact transformer · p. 5
  • EM 0.68 with RRC history alone rises to 0.79 when external protocol messages are interleaved, since some downlink messages are externally triggered.For example, RrcRelease after an NGAP UeContextReleaseCommand cannot be predicted from RRC context alone.Source: C. Fine-tuning a compact transformer · p. 5
  • EM 0.80 and syntactic validity 0.99 on UE are achieved without external protocol messages; median TTLT is 320 ms versus 610 ms on gNB.The passage attributes the faster UE generation to generally shorter uplink messages.Source: C. Fine-tuning a compact transformer · p. 5

D. A minimal Mamba-based sequence engine

A Mamba sequence model trained from scratch uses protocol-specific data preparation and architecture choices to improve accuracy and latency over the fine-tuned Qwen, with a trade-off among smaller variants.

D. A minimal Mamba-based sequence engine

  • An 80,000-token context window is 3× larger than Qwen’s, enabled by state space models instead of a self-attention matrix.The Mamba model is trained from scratch on the same dataset with a dictionary-based tokenizer.Source: D. A minimal Mamba-based sequence engine · p. 5
  • EM 0.90 with error-free syntactic validity and median TTLT 115 ms: the 11M-parameter Mamba exceeds fine-tuned Qwen using 98% fewer parameters and is 5 times faster.The passage also reports that it is 18 times faster than Llama-3.2 1B.Source: Table II · p. 6, D. A minimal Mamba-based sequence engine · p. 6
  • EM 0.20 without placeholders or external messages rises to 0.87 when both are added in the 22M-parameter Mamba data-preparation ablation.The figure is also described as highlighting benefits from custom tokenization and cross-layer context.Source: D. A minimal Mamba-based sequence engine · pp. 5–6
  • The 256×1 and 512×1 variants reduce median TTLT below 50 ms but reach EM 0.69 and 0.73, while more than four layers double latency without seeming to improve EM.The results describe an accuracy-latency trade-off across Mamba architectures.Source: D. A minimal Mamba-based sequence engine · p. 6
  • The UE-side Mamba also achieves higher EM and a 4 times faster TTLT than fine-tuned Qwen.The passage does not provide the specific UE-side metric values in this comparison.Source: D. A minimal Mamba-based sequence engine · p. 6

VI. CONCLUSION

The conclusion presents domain-specific sequence models as a path to faster, more accurate RRC emulation, while noting that the 10 ms target remains unmet. It also points to trainable control planes that could be updated and specialized for deployments.

VI. CONCLUSION

  • 55 times smaller models trained from scratch achieve higher EM and faster TTLT than fine-tuned models, but the 10 ms target is not met.The conclusion suggests quantization-aware training may further reduce latency without compromising accuracy.Source: §VI · p. 6
  • A trace-trained Layer-3 control plane could be updated with model releases and specialized to particular deployments rather than waiting for standardization cycles.Fig. 3 marks the Llama-3.2 1B baseline, the 10 ms Layer-3 target, and the T300 minimum and maximum.Source: §VI · p. 6, Figure 3 · p. 6

Related papers

Loading 2609.37697v1…