Source-linked AI summary
SpecMind: Enabling Spectrum Intelligence via Multi-Agent Hybrid Retrieval-Augmented Generation
Songwei Dong, Bingyan Lu, Makayla Kienlen, J. Nicholas Laneman, Cong Shen
TL;DR
Spectrum intelligence requires integrating heterogeneous, fragmented data for increasingly complex management decisions. SpecMind addresses this need with a multi-agent hybrid RAG framework and SpecBench benchmark, and outperforms strong baselines across spectrum query types with over 80% win rate.
Problem
Spectrum management must interpret growing, heterogeneous information across licenses, proceedings, and regulations, while existing benchmark resources for such RAG tasks are limited.
Method
SpecMind coordinates specialized agents using modality-aware retrieval over license databases, proceeding graphs, and regulatory texts, evaluated with SpecBench.
Results
SpecMind consistently outperforms Web-search RAG and SpectrumRAG across question types under both backbone LLMs, with especially substantial gains on license and compound queries.
Takeaways & Limitations
Structured SQL retrieval is critical for license records, graph retrieval benefits multi-document proceedings, and coordinating these mechanisms yields gains on compound queries.
Abstract
from arXiv · showhide
The exponential growth of wireless devices is driving unprecedented spectrum demand, pushing spectrum management toward more fine-grained decisions across space, time, and device constraints. As a result, spectrum policymakers and engineers must process large volumes of data that come from diverse sources and take many different forms, such as text and tables. These data sources are often disaggregated and require significant time and effort to integrate, search, and interpret. Furthermore, most of this information is formatted for human understanding and is not readily accessible to automated systems. To address this challenge, we propose SpecMind, a novel Multi-Agent Retrieval-Augmented Generation (RAG) system for spectrum intelligence that performs reasoning over heterogeneous data sources. This system enables autonomous agents to coordinate specialized sub-agents that retrieve and synthesize knowledge across policy proceedings, legal regulations, and license databases. We develop SpecBench, a question and answer (Q&A) dataset based on real-world license records and policy proceedings, addressing the lack of evaluation resources for RAG systems in the spectrum domain. Experimental results demonstrate that SpecMind outperforms traditional, general-purpose RAG systems across spectrum-related tasks, achieving over 80% win rate against strong baselines. The agent-based design enables more accurate retrieval, better contextual reasoning, and improved task completion across diverse query types.
I. INTRODUCTION
Spectrum management must integrate rapidly growing, heterogeneous data across operational and regulatory constraints. SpecMind addresses this challenge with an agentic RAG framework, reusable spectrum databases, SpecBench, and coordinated retrieval across data modalities.
- Increasing spectrum demand and constraints across frequency, time, location, users, and regulation make spectrum data integration and interpretation more complex.
- SpecMind is an Agentic RAG framework for spectrum intelligence over heterogeneous data that supports complex reasoning and task execution.
- The corpus unifies FCC licensing records, proceeding documents, and regulatory texts as complementary machine-readable resources.
- SpecBench contains 450 curated question–answer pairs covering fact verification, cross-source synthesis, and multi-hop reasoning.
- SpecMind coordinates retrieval and reasoning over tabular, graph-structured, and textual sources, achieving over 80% win rate against strong baselines.
II. METHODOLOGY
The methodology builds modality-aware databases for structured licenses, graph-structured proceedings, and text-based regulations. Each source receives a retrieval strategy matched to its data characteristics.
- The system uses license tables, FCC proceeding documents, and regulatory documents as three primary spectrum data sources.
- Regulatory retrieval segments documents into 1000-token chunks with 200-token overlap, retrieves 20 embedding candidates, and reranks them to 5 contexts.
- Proceeding data are modeled with separate graph databases for each proceeding to support structural consistency and modular extension.
- License retrieval reformulates natural-language questions as executable SQL queries to enable exact matching over structured numerical attributes.
B. Framework Design
SpecMind represents each task with a query, heterogeneous knowledge sources, and coordinated agents. A Supervisor Agent iteratively invokes specialized agents or internal operations, accumulates results, and produces a final answer.
- SpecMind uses a multi-agent hybrid RAG system whose knowledge sources include license databases, proceeding graphs, and regulatory texts.
- The control loop initializes global task state, selects actions, executes agents or internal operations, and updates state until termination.
- The agent set contains a Supervisor Agent and specialized license, proceeding, and regulation agents.
- The Supervisor Agent coordinates task decomposition, agent invocation, and result integration through associated action policies.
- The system accumulates intermediate evidence and partial results across agent interactions before generating a final answer grounded in heterogeneous knowledge sources.
C. Prompt Engineering
Prompt engineering explicitly defines agent-level policies for decomposition, tool use, retrieval, and coordination. These policies tailor reasoning and retrieval to license, proceeding, and regulatory data.
- Agent-level policies Π = {πsup, πlic, πproc, πreg} govern task decomposition, tool usage, and cross-agent coordination.
- Supervisor prompt: The Supervisor prompt uses a Think–Act–Observe loop to decompose queries, route subtasks, and update global task state iteratively.
- License agent prompt: The license policy inspects schemas, generates executable SQL, validates queries, and refines results from tool outputs.
- Proceeding agent prompt: The proceeding policy selects proceedings by topic and chooses basic, local, or global GraphRAG tools according to query granularity.
- Regulation agent prompt: The regulation policy combines embedding retrieval with lightweight reranking to improve evidence precision for regulation queries.
III. SP E CBE N C H DATASET
SpecBench is a real-world Q&A benchmark for evaluating RAG over heterogeneous spectrum data and multiple reasoning capabilities. Its questions span source types, evidence structures, and answerability conditions to support controlled evaluation.
- SpecBench addresses the lack of benchmarks for heterogeneous spectrum data, including proceedings, license databases, and regulations.
- The benchmark evaluates noise robustness, information integration, and negative rejection.Negative rejection requires abstaining when supporting evidence is insufficient.
- An evidence unit is the minimal retrievable element: a document for unstructured data or a table cell for structured data.Questions are single-source when one evidence unit suffices and multi-source otherwise.
- Compound questions test cross-agent coordination through parallel or sequential evidence dependencies.Sequential questions are more susceptible to error propagation because intermediate results formulate later queries.
- 450 Q&A pairs cover proceedings (31.1%), licenses (31.1%), regulations (13.3%), compound queries (14.4%), and unanswerable questions (10.0%).Questions were identified through domain-expert interviews, with manually retrieved supporting documents and evidence-grounded reference answers.
A. Baselines
The evaluation compares SpecMind with Web-search RAG and SpectrumRAG as representative spectrum question-answering baselines. The baselines use publicly accessible search or iterative query rewriting to improve retrieval.
- Web-search RAG retrieves the top-20 Google Search results for each query as generation context.It provides a practical baseline using publicly accessible resources.
- SpectrumRAG is an iterative RAG framework that uses LLM-based query rewriting to refine retrieval for spectrum policy question answering.The design is included as a competitive advanced RAG baseline.
- SpectrumRAG is described as improving retrieval quality and downstream generation in spectrum policy question answering.
B. Experimental Setup
The experiments evaluate SpecMind and baselines under matched backbone and embedding conditions. They measure valid answering, abstention, and comparative response quality using success rate and win rate.
- All methods are evaluated with Qwen3-8B and GPT-5.2 as backbone LLMs, using text-embedding-3-small for retrieval.Each method uses the same backbone for both RAG and reasoning.
- Success rate requires a factually grounded answer when evidence exists or correct abstention when no valid evidence is available.The metric covers standard license, proceeding, regulation, and compound questions alongside negative-rejection questions.
- Win rate compares methods on correctness, completeness, and faithfulness relative to the ground-truth answer.If all responses are incorrect, no method is selected as the winner.
C. Main Results
SpecMind outperforms Web-search RAG and SpectrumRAG across question types under both tested backbones, with especially strong gains on license and compound queries. Ablations show that specialized agents contribute to performance, particularly through structured and graph-based retrieval.
- Overall performance: SpecMind consistently outperforms Web-search RAG and SpectrumRAG across question types under both backbone LLMs.The largest win-rate gains occur on license and compound queries.
- Success rate: SpecMind achieves near-perfect success rates with GPT-5.2 and consistently high accuracy with Qwen3-8B, while baselines perform substantially worse on license and compound tasks.Web-search RAG remains relatively strong on regulation queries but does not generalize as well to structured and multi-source settings.
- Unanswerable questions: All methods achieve perfect success rates on unanswerable questions because the task primarily requires abstention rather than evidence synthesis.Performance in this category is therefore more dependent on backbone-model calibration than retrieval framework.
- Ablation findings: Replacing any specialized agent with naive RAG causes consistent performance degradation, with the largest effects on corresponding question types and then compound queries.The pattern reflects error propagation in multi-step reasoning.
- Ablation findings: License-agent replacement causes drops of up to −87.8 win rate and −82.8 success rate, highlighting the importance of SQL-based retrieval for structured data.Proceeding-agent replacement also reduces proceeding performance by −27.3 win and −10.3 success, while the regulation agent has a smaller impact.
V. QUALITATIVE ANALYSIS
SpecMind answers compound spectrum questions by coordinating regulation, license, and proceeding agents in a sequential, state-driven workflow. Intermediate results constrain later retrieval and support cross-source reasoning over heterogeneous data.
- Coordination: The supervisor sequentially selects agents and subtasks while maintaining intermediate results in the task state.This state-driven process lets later actions depend on earlier retrieval results.
- Regulation retrieval: The regulation agent first identifies the relevant frequency band, establishing a constraint for subsequent steps.The example uses the 12.7–13.25 GHz band.
- License retrieval: The license agent then retrieves corresponding licensees through SQL queries, grounding the compound task in structured data.The example identifies INTELSAT and GLOBECOMM.
- Proceeding retrieval: The proceeding agent uses graph-based retrieval to compare entities’ spectrum-sharing positions and reveal consistent opposition among incumbents.This completes the cross-source chain after frequency and license information are established.
B. Failure Analysis
Remaining non-optimal cases arise mainly from limitations in evidence representation and query specification rather than systematic failures of the coordination framework. The paper identifies attribution ambiguity, sparse evidence, and underspecified structured queries as key boundaries.
- Evidence representation: Proceeding graphs may record entity–filing co-occurrence without explicitly distinguishing authorship from referenced entities, limiting fine-grained attribution.This makes precise identification of who authored a filing more challenging.
- Evidence representation: Long-tail stakeholders have sparse graph representations, so retrieval may return broader context instead of their most specific stated position.The issue affects entities appearing only a few times in a proceeding.
- Query specification: Identifiers that are unique only within service-specific subtables can make queries omitting service type admit multiple reasonable interpretations.Such ambiguity can produce broader aggregation than intended.
- Implication: SpecMind’s value depends on selecting and composing retrieval mechanisms according to evidence structure rather than relying on one universal pipeline.The conclusion contrasts SQL retrieval for licenses, graph retrieval for proceedings, and conventional RAG for regulation texts.
- Future directions: Future refinement should focus on fine-grained evidence representation and query disambiguation, while broader evaluation requires expanded policy-dataset scale and coverage.The stated expansion targets regulatory domains, frequency bands, and stakeholder interactions.