PaperlayerView exact revision on arXiv

Source-linked AI summary

WirelessAgent++: Automated Agentic Workflow Design and Benchmarking for Wireless Networks

Jingwen Tong, Zijian Li, Fang Liu, Wei Guo, Jun Zhang

arXiv:2603.00501v1cs.NIcs.AIeess.SP

TL;DR

WirelessAgent++ tackles the labor-intensive, static design of wireless agents by treating workflows as executable programs and searching them with domain-adapted MCTS. It also introduces WIRELESSBENCH for multi-dimensional evaluation, and its automatically discovered workflows outperform hand-crafted prompting baselines by up to 31 pp and general-purpose workflow optimizers by up to 11.1 pp.

  • Problem

    Existing wireless agents rely on hand-crafted, static workflows, while automated design must handle vast code-structured search spaces, domain tools, and noisy, costly evaluations.

  • Method

    WirelessAgent++ represents workflows as executable programs, jointly searches workflow structure and tool-calling strategy with domain-adapted MCTS, and introduces WIRELESSBENCH for standardized evaluation.

  • Results

    78.37% (WCHW), 90.95% (WCNS), and 97.07% (WCMSA) are achieved, with improvements of up to 31 pp over prompt-based baselines and 11.1 pp over general-purpose workflow optimizers.

  • Takeaways & Limitations

    The framework demonstrates automated discovery of high-performing workflows across knowledge reasoning, code-based network slicing, and mobile service assurance without manual engineering.

  • Takeaways & Limitations

    The framework is limited to a single-agent setting, uses a fixed tool library during MCTS, and evaluates WCMSA with a simplified constant-velocity mobility model.

Abstract

from arXiv · show

The integration of large language models (LLMs) into wireless networks has sparked growing interest in building autonomous AI agents for wireless tasks. However, existing approaches rely heavily on manually crafted prompts and static agentic workflows, a process that is labor-intensive, unscalable, and often suboptimal. In this paper, we propose WirelessAgent++, a framework that automates the design of agentic workflows for various wireless tasks. By treating each workflow as an executable code composed of modular operators, WirelessAgent++ casts agent design as a program search problem and solves it with a domain-adapted Monte Carlo Tree Search (MCTS) algorithm. Moreover, we establish WirelessBench, a standardized multi-dimensional benchmark suite comprising Wireless Communication Homework (WCHW), Network Slicing (WCNS), and Mobile Service Assurance (WCMSA), covering knowledge reasoning, code-augmented tool use, and multi-step decision-making. Experiments demonstrate that \wap{} autonomously discovers superior workflows, achieving test scores of $78.37\%$ (WCHW), $90.95\%$ (WCNS), and $97.07\%$ (WCMSA), with a total search cost below $\$ 5$ per task. Notably, our approach outperforms state-of-the-art prompting baselines by up to $31\%$ and general-purpose workflow optimizers by $11.1\%$, validating its effectiveness in generating robust, self-evolving wireless agents. The code is available at https://github.com/jwentong/WirelessAgent-R2.

I. INTRODUCTION

WirelessAgent++ addresses the labor-intensive and static nature of manually engineered wireless agents by automating workflow design as executable program search. It combines domain-adapted MCTS, closed-loop tool use, and a multi-dimensional benchmark to discover task-adaptive workflows that outperform manual and general-purpose optimization baselines.

  • Manual agent engineering fixes workflow structure, tool invocation, task decomposition, reflection, and output formatting, creating a bottleneck as wireless tasks and models diversify.
  • WirelessAgent++ casts agent design as executable program search over modular workflow operators and jointly optimizes workflow structure and tool-calling strategy with domain-adapted MCTS.The framework is analogous to neural architecture search, but operates over agent workflow topologies.
  • 78.37% (WCHW), 90.95% (WCNS), and 97.07% (WCMSA) are the reported test scores for autonomously discovered workflows.
  • Its ReAct-based ToolAgent enables closed-loop interleaving of LLM reasoning and external tool calls, supported by penalized Boltzmann selection, a maturity-aware critic, and 3-class experience replay.These enhancements target evaluation noise, search cost, and task diversity in wireless settings.
  • WIRELESSBENCH contains 3,392 deterministic problems across WCHW knowledge reasoning, WCNS dynamic network slicing, and WCMSA proactive mobile service assurance.The benchmark covers knowledge reasoning, code-augmented tool use, and multi-step decision-making.
  • Up to 31% over hand-crafted prompting baselines and 11.1% over the best general-purpose workflow optimizer are the reported improvements.

B. Wireless Tasks as Agent Programs

WIRELESSAGENT++ represents wireless agent workflows as executable programs built from modular operators and domain tools, then searches this structured space for high-scoring solutions. Its ToolAgent supports closed-loop reasoning and tool use, while CodeLevel provides deterministic execution.

  • Task and workflow formulation: Wireless tasks are modeled as datasets with evaluation metrics, and workflows are executable programs parameterized by prompts, operators, and control flow.The automated design problem seeks a workflow that maximizes validation performance.
  • Operator library: Operators are typed atomic workflow components, including Custom LLM calls, ReAct-based ToolAgent loops, CodeLevel deterministic tools, and ScEnsemble answer aggregation.ToolAgent interleaves reasoning and acting, whereas CodeLevel executes domain tools without an LLM call.
  • Operator library: ToolAgent repeatedly selects and executes tools, records interaction history, and falls back to summarizing recent steps after errors or reaching its iteration limit.The operator stops when the agent chooses finish or after consecutive failures trigger early termination.
  • Search space: The workflow search space contains syntactically valid programs using only the defined operators and domain tools, returning both an answer and API cost.These workflows are typically short programs of 10–50 lines expressed through a constrained API.
  • Framework architecture: WIRELESSAGENT++ combines this structured search space with MCTS optimization and the WIRELESSBENCH evaluation suite.The framework architecture couples workflow nodes, search, and task-adaptive benchmark workflows.

B. Domain-Specific Tool Library

WIRELESSAGENT++ supplies wireless-specific tools that extend LLM workflows with formula retrieval, numerical computation, ray tracing, and mobility prediction. These tools can be invoked through LLM-driven loops or deterministic code to support domain-aware decisions.

  • Tool library overview: The domain-specific tool library provides wireless capabilities that general-purpose workflow optimizers do not inherently possess.The library is a key differentiator of WIRELESSAGENT++.
  • Telecom formula retriever: The formula retriever searches 31 verified telecom formulas across more than 10 categories using weighted keyword, name, notes, text, and category matches.Returned formulas include variable definitions, LaTeX representations, and usage notes.
  • Telecom calculator: The precision calculator supports 20 wireless-computation operations, including special functions, BER, capacity, fading statistics, and FM coefficients.It uses high-precision numerical routines and fallbacks for the Marcum Q-function to reduce transcendental-function errors.
  • Ray-tracing channel predictor: The ray-tracing engine uses real-world building geometry from three HKUST campus regions to estimate channel quality and support channel-aware resource allocation.It evaluates line-of-sight conditions and derives received SNR, CQI, and throughput from path-loss estimates.
  • Ray-tracing channel predictor: Tool-assisted CQI prediction is critical because direct LLM estimation produces near-random accuracy, while the system supports both LLM-driven and deterministic invocation modes.The deterministic CodeLevel mode avoids LLM variability for supported tools.
  • Mobility prediction: For WCMSA, a Kalman filter predicts the next user position from historical trajectories and is chained with ray tracing to estimate future CQI for proactive resource management.The filter uses a constant-velocity state-space model and processes 4–6 historical positions for one-step-ahead prediction.

C. MCTS-Based Workflow Optimization

The workflow optimizer uses refined MCTS to select, mutate, evaluate, and retain candidate programs while incorporating exploration history and critic feedback. It limits mutations, rejects harmful repeats, and uses repeated median validation scores to reduce noise sensitivity.

  • Optimization procedure: Algorithm 2 initializes a seed workflow, repeatedly selects a parent, applies a critic-guided LLM mutation, evaluates it, and updates stored experience.The procedure returns the best workflow found after the configured search rounds.
  • Selection: Parent workflows are sampled from top-K candidates with a Penalized Boltzmann distribution that incorporates exploration-history penalties.The distribution mixes uniform exploration with score-based selection and uses shifted scores for numerical stability.
  • Selection: Soft pruning makes an all-failure node rarely selected without removing it, preserving recovery opportunities unlike irreversible hard pruning.The penalty factor approaches 0.1 for nodes with all-failure histories.
  • Expansion: The heuristic critic guides an advanced Optimizer LLM to make one focused workflow change, such as revising a prompt, inserting verification, or changing tool use.Each mutation is constrained to modify no more than 5 lines of code.
  • Expansion: The modification checker rejects exact repeats of failed changes and patterns known to degrade performance before evaluation.This prevents blocked mutations from being repeatedly explored.
  • Evaluation: Each candidate is evaluated multiple times, and the median validation score is used to reduce the influence of stochastic LLM outputs and numerical outliers.Evaluation records include per-problem scores, error logs, and API costs for later analysis.

4) Experience Backpropagation:

The framework uses experience feedback to guide workflow mutations, stabilize search under noisy evaluations, and detect convergence while benchmarking wireless capabilities across three complementary dimensions.

  • Experience-guided mutation: A lightweight rule-based critic pre-screens mutations using workflow performance, structural complexity, error patterns, and node saturation.It recommends more conservative changes for high-scoring workflows and structural changes for low-scoring ones.
  • Experience-guided mutation: The critic uses four aggressiveness levels with thresholds τhigh = 0.65 and τmid = 0.50.Workflows with s ≥ 0.65 receive conservative treatment, while those with s < 0.50 may receive aggressive changes.
  • Three-class experience replay: A significance threshold ϵ distinguishes success, neutral, and failure outcomes so the optimizer does not chase noisy score fluctuations.The experiments set ϵ = 0.02, corresponding to 2% absolute accuracy.
  • Three-class experience replay: Successful mutations become positive exemplars, neutral mutations remain unpenalized, and failed mutations are blacklisted while increasing the parent penalty factor ρ.These classes are handled differently during experience replay.
  • Convergence detection: The optimizer monitors the running top-k average and stops after five consecutive rounds without statistically meaningful improvement.The default configuration uses k = 3, patience C = 5, and z = 0.
  • Benchmark scope: WIRELESSBENCH evaluates knowledge reasoning, intent understanding, and multi-step decision making through WCHW, WCNS, and WCMSA.These dimensions target wireless theory, interaction with wireless environments, and proactive resource allocation for mobile users.

B. Data Construction Pipeline

The benchmark suite is built through a standardized pipeline that combines authoritative wireless sources, automated quality control, format-aware scoring, and human validation across three task types.

  • Unified construction pipeline: WIRELESSBENCH problems are sourced from wireless textbooks and 3GPP/IEEE standards, paired with step-by-step chain-of-thought solutions, and processed through psychometric data cleaning.The pipeline uses authoritative sources for WCHW, WCNS, and WCMSA before subsequent validation stages.
  • Unified construction pipeline: Human validation checks every problem’s solution correctness, reasoning, and answer format.Graduate students perform the final verification and standardization step.
  • WCHW: WCHW contains 1,392 university-level wireless communication problems covering modulation, channel capacity, error probability, signal processing, and antenna theory.Problems require domain-specific formulas, unit conversion, and numerical computation with special functions.
  • WCHW: The evaluation system classifies expected answer formats, detects units, and applies multiple scoring strategies to reduce false negatives from format mismatches.It recognizes nine answer formats and 38 unit multipliers across six physical dimensions.
  • WCNS: WCNS combines network state, user position, natural-language service requests, ray-tracing CQI prediction, and proportional-fair resource allocation in a 5G slicing scenario.The benchmark includes eMBB and URLLC slices with capacities of 90 MHz and 30 MHz, respectively.
  • WCMSA: WCMSA extends WCNS with mobility, requiring future-position prediction, channel estimation at predicted locations, service classification, and proactive resource management.Its scoring uses six metrics, including distance-based position prediction and a QoS-verification bonus.
  • Benchmark integrity: All benchmark ground truths are generated from deterministic rules or expert solutions to support reproducibility.The suite statistics are summarized in Table VI.

VI. EXPERIMENTAL RESULTS

WIRELESSAGENT++ is evaluated against prompting and workflow-optimization baselines on general and wireless benchmarks, where it achieves strong performance, low search cost, and progressively improved task-specific workflows.

  • Experimental setup: WIRELESSAGENT++ uses Claude-Opus-4.5 for workflow mutations and Qwen-turbo-latest for workflow execution.Separating optimizer and executor models supports lightweight deployment and cost-performance analysis.
  • Experimental setup: WIRELESSAGENT++ is compared with zero-shot, chain-of-thought, MedPrompt, ADAS, and AFlow baselines on general NLP and wireless benchmarks.AFlow is the MCTS-based optimizer without the proposed domain-specific enhancements.
  • Main results: WIRELESSAGENT++ achieves the highest scores on HotpotQA, DROP, MATH, and WIRELESSBENCH, including a 0.8102 solve rate on WIRELESSBENCH (After).It reaches F1 scores of 0.7273 on HotpotQA and 0.8021 on DROP, plus a 0.8210 solve rate on MATH.
  • Main results: 11.10 pp is the improvement over AFlow on WIRELESSBENCH (After), while the Before comparison shows a 20.12 pp gap.The results attribute these gains to the domain-aware enhancements and ToolAgent.
  • Cost analysis: $4.95 is the total WCHW search cost, compared with $0.99 for WCNS and $1.05 for WCMSA.Optimized-workflow inference costs remain below $0.001 per problem on all three benchmarks.
  • WCHW case study: WIRELESSAGENT++ significantly outperforms all baselines on WCHW, demonstrating the value of domain-specific workflow optimization for wireless knowledge reasoning.The WCHW workflow evolves from a formula-library seed to tool verification and later prompt refinement.
  • WCHW case study: The WCHW workflow rises from 62.44% in the seed phase to 80.86% after ToolAgent discovery and reaches a best validation score of 81.78%.ToolAgent verification produces the largest improvement, while later refinement expands formula coverage and handles formula-type answers.

2) WCNS (Ray-Tracing-Augmented Network Slicing):

WCNS evaluates ray-tracing-augmented network slicing, where WIRELESSAGENT++ improves over prompting without manual workflow engineering. Its search discovers tool use, then compiles the workflow into deterministic execution.

  • Performance: 0.922 composite score: WIRELESSAGENT++ outperforms Prompt-based at 0.613 (+30.9 pp) and approaches WirelessAgent at 0.985.The Rule-based method reaches 1.000 as an idealized performance upper bound.
  • Performance: 96% slice classification, 96% CQI prediction, 91% bandwidth allocation, and 89% throughput calculation show competitiveness across all four sub-metrics.The largest Prompt-based gain is CQI prediction: 96% versus 2%.
  • Workflow evolution: The workflow improves from 61.3% with a bare Custom call to 90.5% after ray-tracing discovery, then 92.18% after deterministic CodeLevel compilation.Tool discovery supplies algorithmic CQI computation instead of unreliable LLM estimation.
  • Cross-task context: WCMSA reaches 96.89% overall, with at least 96% on all six sub-metrics through a Kalman-filter and ray-tracing pipeline.The pipeline predicts position, estimates CQI, and then performs reasoning for proactive service assurance.
  • Ablation evidence: Ablations identify domain tools as the largest contributor, with their removal reducing WCHW accuracy by 19.3 percentage points.The 3-class experience mechanism, penalized Boltzmann selection, and heuristic critic address noisy or inefficient search in distinct ways.
  • Cost: Total MCTS optimization cost stays below $5 across benchmarks, while per-problem inference costs under $0.001.WCNS and WCMSA searches complete in approximately 13–14 minutes.

APPENDIX

The appendix specifies WIRELESSBENCH’s evaluation pipeline and illustrates representative tasks and an optimized WCHW workflow. Its scoring procedure handles heterogeneous answer formats through classification, normalization, and parallel comparison strategies.

  • Evaluation pipeline: Reference answers are classified into nine formats, including numeric, formula, percentage, text, code-sequence, and ratio answers.Regex-based pattern matching performs the classification.
  • Evaluation pipeline: The evaluator recognizes 38 unit multipliers across six physical dimensions and converts extracted values to base SI units.Dimensions include frequency, data rate, power, time, distance, and spectral efficiency.
  • Evaluation pipeline: Four scoring strategies run in parallel, and the maximum score is returned to reduce false negatives from output-reference format mismatches.The strategies include format-aware, numeric, formula, and additional matching procedures.
  • Benchmark tasks: WCHW covers 10 knowledge categories with varying difficulty levels, while WCNS and WCMSA target slicing decisions and proactive resource allocation.The examples span Shannon-capacity and BER reasoning, network slicing allocation, and trajectory-based service assurance.
  • WCHW workflow: The optimal WCHW workflow uses Custom to solve the problem and a two-step ToolAgent ReAct loop to independently verify the answer with Python.The verifier recalculates quantities such as log2() and erfc() before emitting the verified value.

F. WCNS Optimal Workflow

The optimized workflows specialize operator order to each wireless task. WCNS uses deterministic ray tracing before LLM reasoning, whereas WCMSA predicts position and future CQI before final decision-making.

  • WCNS Optimal Workflow: The optimal WCNS workflow scores 92.18% and follows a Tool-then-Reason pattern: CodeLevelRayTracing → Custom.Ray tracing injects exact CQI into the problem before the LLM solves slicing and throughput requirements.
  • WCNS Optimal Workflow: CodeLevelRayTracing extracts coordinates, queries the channel model, returns CQI from 1–15, and injects it into the problem.The operator is deterministic and does not require an LLM call.
  • WCNS Optimal Workflow: The WCNS Custom stage applies bandwidth bounds, CQI-to-spectral-efficiency mapping, and throughput calculation after receiving the enriched problem.The prompt includes worked CQI examples and 3GPP TS 38.214 lookup guidance.
  • WCMSA Optimal Workflow: The optimal WCMSA workflow scores 96.89% and follows Predict-Estimate-then-Reason: Kalman-Predictor → RayTracing → Custom.It outputs position, CQI, slice, bandwidth, throughput, and QoS.
  • WCMSA Optimal Workflow: WCMSA ray tracing runs at the predicted position rather than the current position, injecting future CQI for proactive allocation.The workflow first injects a constant-velocity Kalman prediction, then estimates channel quality at that predicted location.

H. CQI-to-Spectral-Efficiency Mapping

The appendix documents the CQI-to-spectral-efficiency mapping, available workflow operators, benchmark environments, and optimizer configuration. These components define the computational and experimental context for workflow construction.

  • CQI mapping: Table X provides the complete CQI-to-spectral-efficiency mapping derived from 3GPP TS 38.214 Table 5.2.2.1-2.The mapping is used by both the ray-tracing channel predictor and optimized-workflow prompts.
  • Operator repertoire: Table XI lists the typed operators available to MCTS, including flexible LLM calls, ReAct tool use, deterministic code execution, and ensemble operators.CodeLevel operators provide LLM-free execution with zero variance and near-zero cost.
  • Benchmark specification: Table XII defines the 20 WCMSA service types and their associated slice types and minimum throughput requirements.The agent must verify these requirements during mobile service assurance.
  • Experimental environment: The ray-tracing benchmark uses three HKUST campus regions with distinct propagation environments and positions sampled from outdoor areas.Transmitters are placed at the tallest-building centroid in each region, 5.0 m above the rooftop.
  • MCTS configuration: The optimizer prompt permits adding, modifying, or deleting workflow nodes, parameters, and prompts through single focused modifications.It can incorporate reasoning patterns such as review, revision, ensembles, and selfAsk.

O. ReAct Agent Protocol

The ReAct Agent combines optimizable strategy prompts and runtime context with fixed protocol constraints and modular operators. Its prompts enforce structured outputs, tool-use formatting, and iterative execution context.

  • The ToolAgent operator uses a three-layer prompt architecture with a fixed protocol layer enforcing structured XML output.
  • ScEnsemble selects the most frequently occurring answer among generated solutions, while REVIEW_PROMPT returns a boolean correctness judgment with feedback when confidence exceeds 95%.
  • The fixed protocol requires thought and action_type tags, valid JSON tool arguments, pre-computed numeric values, and a nonempty final answer.
  • Each ReAct iteration sends the strategy prompt, available tools, previous steps, and immutable fixed protocol to the Executor LLM.

P. Heuristic Critic Report

The heuristic critic summarizes workflow performance and constrains subsequent modifications according to score and prior experience. Its conservative mode permits local prompt changes while forbidding structural workflow changes.

  • The critic report records operator usage, conditionals, custom calls, attempts, successes, failures, and node saturation.
  • For high scores of at least 0.65, the optimizer is restricted to minor prompt improvements and cannot add operators or alter workflow structure.
  • The experience tree records successful, failed, and neutral modifications, with neutrality defined by an absolute score change below ϵ = 0.02.
  • The illustrated tree reports a ToolAgent addition improving the score by 18.4 percentage points and a ScEnsemble addition reducing it by 19.1 percentage points.
Loading 2603.00501v1…