PaperlayerView exact revision on arXiv

Source-linked AI summary

CRAFT: Fine-Tuning Pre-hoc Explainability in AI-native 6G RAN

Pranshav Gajjar, Vijay K Shah

arXiv:2609.00590v1cs.LG

TL;DR

Telecom SLM training must produce decisions with genuinely pre-hoc, auditable reasoning, but direct reasoning-oriented optimization faces a cold-start barrier. CRAFT verifies reasoning traces offline, fine-tunes on the resulting triplets, and achieves strong, format-compliant results while reducing training energy. Its evidence is limited to classification-style decisions, and broader model-family coverage would strengthen the claims.

  • Problem

    Existing telecom training recipes produce direct or post-hoc explanations rather than reasoning traces causally tied to decisions, while naive GRPO faces a cold-start barrier.

  • Method

    CRAFT autonomously constructs verified (input, trace, label) triplets and fine-tunes compact SLMs on them with low-rank supervised adaptation.

  • Results

    86.5% and 94.6% macro-F1 are achieved on TRACTOR and IC xApp, respectively, with zero parse failures, while CRAFT remains robust under continued GRPO and uses up to 59% less energy than GRPO-based baselines.

  • Takeaways & Limitations

    CRAFT-trained models are presented as practical components for AI-RAN xApps and rApps requiring competent decisions and auditable rationales.

  • Takeaways & Limitations

    Evaluation covers classification-style decisions with discrete labels and only three SLMs from two model families.

Abstract

from arXiv · show

The next generation of mobile networks is envisioned as fully AI-native, with AI-RAN architectures embedding small language models (SLMs) to perform reasoning over real-time telemetry. The state-of-the-art training paradigms for telecom LLMs, exemplified by RANSTRUCT-style supervised fine-tuning (SFT) on curated instruction data, are limited to post hoc rationalization. Here, the explanations, when produced at all, are generated after or independently of the decision, leaving the decision process unauditable. Pre-hoc reasoning, where a causal reasoning trace is produced before the output label, is preferable, and the broader LLM reasoning literature has made real progress toward it via RL methods such as Group Relative Policy Optimization (GRPO). Here we observe that transplanting this recipe into the telecom setting runs into a cold-start barrier: SLMs either learn to output the desired format or learn to predict the label, but rarely both. We identify this barrier and propose CRAFT, which stands for Cold-start Reasoning Alignment via Fine-Tuning, a data-centric method to autonomously generate a verified dataset of (input, trace, label) triplets. CRAFT fine-tunes SLMs on this verified data using low-rank adaptation (LoRA), requiring substantially less compute and wall-clock time than GRPO-based methods. On the TRACTOR and IC xApp telecom datasets, CRAFT achieves up to 86.5% and 94.6% for accuracy and F1 with no parse failures, while direct GRPO and SFT+GRPO fail to exceed 28% and 53.5% F1 with multiple parse failures. We further show that CRAFT-initialized policies serve as a robust foundation for subsequent GRPO fine-tuning, as under diverse reward functions the performance remains consistent with no parse failures. Finally, we demonstrate that CRAFT consumes 59% less energy than GRPO-based baselines, making it a sustainable path to deployable, auditable AI in 6G RAN.

I. INTRODUCTION

6G AI-RAN embeds intelligence in network control loops, increasing demand for edge SLMs that produce auditable pre-hoc reasoning. Existing telecom training and naive GRPO approaches do not reliably provide both valid traces and correct decisions, motivating CRAFT.

  • I. INTRODUCTION: AI-RAN embeds intelligence in network control loops through O-RAN controllers hosting xApps and rApps.These components support automated, real-time decisions about resource allocation, slicing, and interference mitigation.
  • I. INTRODUCTION: Edge SLMs must interpret KPI telemetry, output correct actions, and provide human-readable justifications before operators trust decisions.The target SLMs are designed for edge performance and range from 100M to 5B parameters.
  • I. INTRODUCTION: Existing telecom methods generally emit answers directly, while explanation-producing systems generate explanations after decisions are made.This ordering does not meet the requirement for auditable, decision-linked reasoning.
  • I. INTRODUCTION: GRPO and chain-of-thought offer a template for pre-hoc reasoning, but telecom SLMs face a cold-start barrier because they lack priors over valid telecom traces.The trace should be generated first and the label derived from it.
  • I. INTRODUCTION: CRAFT autonomously constructs verified (input, trace, label) triplets and fine-tunes compact SLMs with low-rank supervised adaptation.Offline verification avoids online exploration during policy optimization.
  • I. INTRODUCTION: 86.5% and 94.6% macro-F1 were achieved on TRACTOR and IC xApp with zero parse failures, while GRPO-based baselines remained below 28% and 53.5% F1.CRAFT also used less compute and wall-clock time and was up to 59% more energy-efficient than GRPO-based baselines.

II. BACKGROUND

Telecom language-model research spans benchmark evaluation, cooperative agents, and domain-specific training, but causal ties between reasoning traces and decisions remain underexplored. LoRA-based telecom fine-tuning improves label prediction without providing auditable reasoning traces.

  • II. BACKGROUND: ORAN-Bench-13K provides nearly 14,000 O-RAN specification questions and shows substantial headroom for general-purpose LLMs on O-RAN-specific knowledge.This motivates retrieval-augmented and fine-tuned alternatives.
  • II. BACKGROUND: AI5GTest and GENESIS use cooperative LLM agents for specification testing and broader RAN engineering tasks, respectively.GENESIS was validated with over-the-air experiments on a production O-RAN testbed.
  • II. BACKGROUND: The remaining focus is whether telecom models produce reasoning traces causally tied to decisions rather than merely rhetorical explanations.The distinction concerns the relationship between the trace and the already-produced decision.
  • II. BACKGROUND: RANSTRUCT and TelecomGPT use LoRA, which freezes pretrained weights and adds trainable low-rank matrices.The injected matrices B and A have rank r much smaller than the layer dimensions.
  • II. BACKGROUND: LoRA-based training can yield strong label-prediction accuracy but provides no mechanism for auditable reasoning traces.This limitation separates decision accuracy from pre-hoc explanation quality.

B. Chain-of-Thought Reasoning

Chain-of-thought models generate a reasoning trace before decoding the label, making the trace potentially upstream of the output. GRPO trains this behavior by ranking sampled outputs with group-relative rewards and combining format and correctness objectives.

  • B. Chain-of-Thought Reasoning: CoT factorizes generation as pθ(t, y | x) = pθ(t | x) pθ(y | x, t), sampling the trace before the label.Self-consistency samples K independent traces and decodes labels from them.
  • B. Chain-of-Thought Reasoning: Because the label is conditioned on the preceding trace, pre-hoc reasoning can be causally upstream of the output rather than a post-hoc gloss.SHAP and LIME instead justify an already-fixed prediction and need not reflect its generating mechanism.
  • C. Group Relative Policy Optimization: GRPO samples a group of G outputs, scores them with rewards, and uses the group itself as the baseline instead of a critic.The policy update uses group-normalized advantages in a clipped PPO-style objective.
  • C. Group Relative Policy Optimization: The group-normalized advantage uses each reward’s deviation from the group mean divided by its standard deviation, stabilized by δ > 0.Under outcome supervision, the scalar advantage is broadcast across every token of an output.
  • C. Group Relative Policy Optimization: GRPO rewards both task-agnostic form compliance and task-specific correctness, such as classification accuracy or bounded regression error.A KL penalty anchors the updated policy to a reference policy.

III. CRAFT

CRAFT addresses cold-start failure by constructing verified reasoning examples before supervised fine-tuning. Naive reasoning prompts and GRPO baselines instead collapse into invalid formats and unparseable outputs on the motivating experiment.

  • III. CRAFT: CRAFT synthetically constructs (KPI input, reasoning trace, label) triples whose traces are verified to be causally informative for their labels.The resulting dataset is used for ordinary supervised fine-tuning of a target SLM.
  • A. Motivation: The motivating comparison evaluates label-only SFT, post-hoc think-field prompting, and SFT followed by GRPO on TRACTOR using Qwen 3.5 2B.The post-hoc template does not supervise actual reasoning content.
  • A. Motivation: Standalone GRPO and SFT followed by GRPO show the same 100% parse-failure rate and 0% accuracy in the cold-start setting.The failure involves repetition of format instructions instead of a parseable prediction.
  • A. Motivation: A representative failure repeats allowed-label and formatting instructions rather than producing a valid label for an eMBB example.The parsed prediction is None despite the stated ground truth.

B. Algorithm

CRAFT constructs a verified reasoning dataset by generating candidate traces with an Oracle and retaining only traces that enable a Predictor to recover the correct label. The resulting triplets support pre-hoc, auditable reasoning rather than post-hoc rationalization.

  • Algorithm: CRAFT uses an Oracle Reasoner and Predictor, instantiated from the same base model with different prompts, to construct verified reasoning examples.The Oracle receives the KPI window and ground-truth label; the Predictor receives the window and candidate trace without the label.
  • Algorithm: The algorithm returns a dataset of verified (input, trace, label) triplets that provides a pre-hoc and auditable reasoning path.The trace is generated before the Predictor produces the label.
  • Algorithm: For each labeled example, CRAFT serializes the KPI window, generates a candidate trace, and filters it for minimum length, valid structure, and successful extraction.Any failure in these checks discards the example.
  • Algorithm: A surviving trace is retained only when the Predictor can parse the correct label from the KPI window and trace.This verifies that the trace contains information needed for the decision rather than merely sounding plausible.

IV. EXPERIMENTAL SETUP

The experiments evaluate CRAFT across two near-RT RIC telecom classification datasets, three target SLMs, four alternative training paradigms, and fixed LoRA-based training conditions. Performance is assessed using classification, formatting, reasoning-output, solution-output, and runtime metrics.

  • Datasets: TRACTOR contains 1,575 four-class network-slice classification windows, while IC xApp contains 389 binary interference-detection windows.The datasets differ in KPI composition, temporal granularity, and label cardinality.
  • Datasets: Both datasets use fixed 70/15/15 train, validation, and test partitions shared across every compared method.The partitioning is stated across the dataset description and implementation details.
  • Models: The target models are Qwen 3.5 2B, Qwen 3.5 4B, and Nemotron-3-Nano 4B; Gemma 4 31B generates CRAFT data offline.The larger Gemma model is confined to dataset construction rather than deployment.
  • Baselines: CRAFT is compared with Zero Shot, SFT, GRPO, and SFT+GRPO training paradigms.GRPO uses a composite reward for output format, reasoning presence, and label accuracy.
  • Training and metrics: All runs use rank-16 LoRA adapters on one RTX 4090 under a fixed 12-hour compute budget, with accuracy, macro-F1, PF%, Think%, Solution%, and wall-time reported.GRPO samples four completions per prompt at each step.

A. Ablation Studies

The ablations test whether CRAFT remains effective under continued GRPO, transfers from TRACTOR to IC xApp, and improves energy efficiency. These studies compare performance and parse failures across reward schemes, datasets, and training approaches.

  • Reward robustness: CRAFT initialization is evaluated under three continued-GRPO reward weightings for macro-F1 and parse-failure percentage.The study starts from the best CRAFT-tuned Qwen 3.5 4B model on TRACTOR and continues training within the 12-hour budget.
  • Generalization: The IC xApp ablation repeats the baseline-versus-CRAFT comparison with the best TRACTOR SLM to test whether the cold-start remedy generalizes across datasets.Side-by-side macro-F1 and PF% plots make the comparison across methods and datasets direct.
  • Energy efficiency: The third ablation quantifies CRAFT’s energy efficiency against GRPO-based baselines.The supplied passage introduces the energy study but does not state its numerical outcome.

V. RESULTS AND DISCUSSION

On TRACTOR, the reported results show that model size and training paradigm affect both classification quality and output validity. The supplied results indicate that CRAFT is evaluated against zero-shot, SFT, GRPO, and SFT+GRPO approaches.

  • Main results: 32.9% accuracy and 26.3% macro-F1 are reported for Qwen 3.5 4B under zero-shot prompting, with 0% parse failures.Qwen 3.5 2B and Nemotron-3-Nano instead reach 0% accuracy with 100% parse-failure rates.
  • Main results: 54.0% accuracy and 51.1% macro-F1 are reported for Qwen 3.5 4B after one epoch of label-only SFT, with a 3.8% parse-failure rate.The model also has a 96.6% Think rate in this condition.
  • Main results: 35.0% accuracy and 28.0% macro-F1 are reported for Qwen 3.5 4B under balanced-reward GRPO.Its parse-failure rate is reported as 0%.
  • Main results: 56.5% accuracy and 53.5% macro-F1 are reported for Qwen 3.5 4B under balanced-reward SFT+GRPO.Its parse-failure rate is reported as 2.5%.

A. Ablation 1: reward robustness under continued GRPO.

Continued GRPO preserves CRAFT’s strong reasoning performance across balanced, correctness-priority, and random-dynamic reward weightings, but does not improve on the CRAFT baseline. On IC xApp, CRAFT likewise maintains high performance with valid output formatting while alternative approaches exhibit severe parse failures.

  • A. Ablation 1: reward robustness under continued GRPO.: 85-86% macro-F1 with 0% parse failures is sustained across all three CRAFT+GRPO reward-weight modes.The modes are balanced, correctness-priority, and random-dynamic.
  • A. Ablation 1: reward robustness under continued GRPO.: 81.9-82.7% accuracy and 85.1-86.1% macro-F1 remain below the CRAFT baseline after continued GRPO.The CRAFT baseline is 83.1% accuracy and 86.5% macro-F1.
  • B. Ablation 2: generalisation to IC xApp.: 94.6% macro-F1 and 0% parse failures are achieved by CRAFT on IC xApp, compared with substantially lower scores and multiple parse failures for alternatives.The comparison includes zero-shot prompting, label-only SFT, GRPO, and SFT+GRPO.
  • B. Ablation 2: generalisation to IC xApp.: Only CRAFT simultaneously maintains high accuracy and valid reasoning format on IC xApp, matching the qualitative pattern observed on TRACTOR.The datasets differ in KPIs, window lengths, and decision cardinality.

C. Ablation 3: energy efficiency.

CRAFT uses substantially less training energy than GRPO-based baselines while separating offline data preparation from LoRA fine-tuning. Its energy advantage supports more sustainable deployment of auditable reasoning models at the network edge.

  • C. Ablation 3: energy efficiency.: 1.13 kWh is consumed by CRAFT, compared with 2.73 kWh for GRPO and 2.75 kWh for SFT+GRPO.CRAFT uses 0.70 kWh for verified dataset preparation and 0.43 kWh for LoRA fine-tuning.
  • C. Ablation 3: energy efficiency.: 59% less energy is consumed by CRAFT than by either GRPO-based baseline.Energy is averaged across all three SLMs and measured from mean GPU power multiplied by recorded training time.
  • C. Ablation 3: energy efficiency.: CRAFT’s lower absolute compute and energy costs are relevant to sustainably deploying auditable reasoning models at the network edge.The result extends beyond wall-clock convenience to total training-resource consumption.

VI. LIMITATIONS

CRAFT is evaluated on two telecom classification benchmarks and three SLMs, but its evidence remains bounded by dataset, task, and model-family coverage. Future work targets broader decision tasks and hardware validation.

  • Dataset coverage: The evaluation spans TRACTOR and IC xApp, covering four-way slice classification and binary interference detection with discrete labels.These benchmarks use different KPI families and window lengths.
  • Dataset coverage: The study does not evaluate continuous-valued tasks such as KPI regression or forecasting.Extending CRAFT’s verification procedure to these settings is identified as future work.
  • Model family diversity: The three target SLMs represent only two underlying model families, Qwen and Nemotron.Broader family coverage would strengthen conclusions because tokenizer and instruction-tuning choices may interact with CRAFT.
  • Future work: Future work will broaden evaluation to additional telecom decision tasks and explore hardware validation with protocols such as TeleMCP.
Loading 2609.00590v1…