Source-linked AI summary
OTel: Building Domain-Specialized Telecom LLM Foundations for Intelligent Networks
Farbod Tavakkoli, Roderic Paulk, Jorden Terrazas, Kenneth Church, Mark Austin, Louis Powell, Gregory Diamos, Lina Bariah, Syed Ali Raza Zaidi, Maryam Hafeez, Ali Maatouk, Imtiaz Karim
TL;DR
Telecom AI systems need domain-specific resources because general-purpose language modeling is insufficient for navigating dense, interdependent, and evolving telecom standards. OTel unifies telecom datasets, reproducible baselines, and evaluation across retrieval, reranking, and language models, improving performance across all three model families, with embedding retrieval reaching 93.5% NDCG@10.
Problem
Telecom AI research lacks a unified open resource combining training data, evaluation setups, and reproducible baselines for domain-specific retrieval, reranking, and generation.
Method
OTel unifies four derived telecom datasets, held-out evaluation partitions, and 30 full-parameter post-trained baselines spanning embedding, reranking, and language models.
Results
OTel fine-tuning improves performance consistently across all three model families, with embedding retrieval reaching 93.5% NDCG@10.
Takeaways & Limitations
Researchers can start from OTel’s shared baselines and focus on targeted improvements to telecom AI systems.
Takeaways & Limitations
Future releases should expand multilingual and multimodal data, improve weaker academic-paper and GSMA PRD subsets, and validate beyond OTel’s held-out partitions.
Abstract
from arXiv · showhide
Frontier AI models have advanced rapidly, but they still struggle with telecom-specific tasks. We present Open Telco (OTel), an open telecom AI resource with derived datasets for retrieval, reranking, instruction tuning, and safety/abstention, plus 30 full-parameter post-trained baselines across embedding, reranking, and language models. The community has already engaged substantially with the resource: as of May 3, 2026, the released models have been downloaded over 16 million times, and the project has received 157+ pieces of media coverage worldwide. Building on prior open telecom datasets and benchmarks, OTel provides documented telecom data sources, held-out evaluation partitions, trained embedding models, rerankers, context-grounded LLMs, and safety/abstention data in one unified resource. OTel post-training improves performance across all three model families: embedding retrieval reaches 93.5% NDCG@10, reranking reaches 0.952 MRR@10, and language-model correctness reaches 88.2%. We release OTel as a reproducible starting point and invite the community to expand the data, improve embedding and reranking models, and build stronger context-grounded telecom LLMs.
1 Introduction
Telecom AI systems face dense, interdependent, continuously revised technical sources, while existing benchmarks lack a unified open resource for training and evaluating retrieval, reranking, and generation models. OTel addresses this gap with an end-to-end, reproducible resource spanning telecom data, derived datasets, and 30 post-trained baselines.
- Motivation: Telecom standards and reference documents are dense, interdependent, and continuously revised, making telecom infrastructure technically demanding for AI systems.The sources include 3GPP specifications, O-RAN documents, GSMA references, and IETF RFCs.
- Problem: Existing research provides telecom benchmarks for broad standards knowledge, mathematical reasoning, O-RAN, and operational root-cause analysis, but not unified training infrastructure.The benchmarks draw on public 5G datasets, 3GPP and related sources, O-RAN scenarios, and operational log data.
- Contribution: OTel covers the full Retrieval-Augmented Generation (RAG) pipeline from data collection through model release as an open telecom AI resource.Its collaborative effort involves more than 100 domain experts and spans roughly 1.1 million raw training points from standards, industry, and academic sources.
- Implementation: 326,767 high-confidence examples and 30 full-parameter post-trained baselines provide a cleaned, reproducible foundation across embeddings, rerankers, and language models.Each baseline uses the matching OTel dataset, a shared reproducible recipe, and held-out OTel evaluation partitions.
- Impact: Over 16 million model downloads and 157+ pieces of worldwide media coverage by May 2026 indicate substantial community engagement with OTel.The passage presents this response as evidence of demand for open telecom AI infrastructure and notes that baseline results further validate the approach.
2 Technical Contribution
OTel integrates curated telecom datasets, full-parameter post-trained baselines across the RAG pipeline, and a reproducible evaluation protocol. Its data-processing and post-training methods address corpus noise and improve embedding, reranking, and language-model performance across model scales.
- Technical contribution: OTel’s technical contribution combines a curated telecom dataset family, full-parameter post-trained RAG baselines, and a reproducible evaluation protocol.The three layers are designed to be independently useful and collectively coherent.
- Data curation: Six processing stages convert QA-source triples into retrieval supervision through document joining, passage creation, hard-negative mining, claim verification, and minimal-context selection.The pipeline uses sliding-window and semantic chunking, reranker rescoring, atomic claim checks, and greedy minimization.
- Data cleaning: 71.1% to 3.9% Acc@1 variation across data shards revealed substantial raw-corpus noise, motivating four sequential cleaning filters.Separate full-parameter embedding models trained on independent shards were evaluated on a common held-out set.
- Dataset formats: OTel releases four task-specific formats: OTel-Embedding, OTel-Reranker, OTel-LLM, and OTel-Safety.They support contrastive retrieval, cross-encoder relevance training, context-grounded instruction tuning, and abstention modeling across the RAG workflow.
- Post-training results: +9.6 to +60.2 percentage points in NDCG@10, +0.535 to +0.598 absolute in MRR@10, and +3.7 to +10.0 percentage points in correctness were achieved through OTel fine-tuning.All rerankers exceeded 0.944 MRR@10; OTel-LLM-27B-IT reached 88.2% correctness, OTel-LLM-8.3B-IT reached 79.6%, and OTel-LLM-1.2B-IT reached 74.4%.
- Diagnostic evaluation: Collaborator-led O-RAN retrieval stress tests across six telecom subdomains provide diagnostic evidence, but their protocols were not yet standardized for formal benchmark tables.The tests examined fixed-chunk question pools across 3GPP, GSMA PRD, O-RAN, RFC, whitepaper, and academic paper sources.
3 Broader Implications
OTel aims to make telecom AI research more accessible in a high-stakes, specialized domain by providing shared data, baselines, and evaluation infrastructure. Future releases are intended to expand coverage, strengthen evaluation, and support incremental community-driven growth.
- Broader significance: OTel targets telecom, a highly consequential and technically specialized domain where deployment errors can have severe consequences.Its challenges include vast standards, specialized terminology, interdependent documents, and risks from misconfigured networks, incorrect protocol interpretation, and unsafe automation.
- Democratizing telecom AI research: OTel lets researchers begin from shared baselines instead of separately negotiating data access, building cleaning pipelines, training models, and creating evaluation harnesses.This design makes targeted improvements more feasible for the broader AI research community.
- Democratizing telecom AI research: OTel is intended to accelerate telecom AI research as shared resources accelerated progress in medical, legal, and financial AI.The cited analogies are MedQA, LegalBench, and FinBen, where shared training data and reproducible baselines helped form coherent research agendas.
- Future directions: Future OTel releases should expand multilingual and multimodal data, improve selected subsets and evaluation pools, standardize per-subdomain metrics, and validate beyond OTel partitions.The resource is architected to support incremental community-driven growth through contributions using the same infrastructure.