PaperlayerView exact revision on arXiv

Source-linked AI summary

Integrated Sensing and Edge AI: Realizing Intelligent Perception in 6G

Zhiyan Liu, Xu Chen, Hai Wu, Zhanwei Wang, Xianhao Chen, Dusit Niyato, Kaibin Huang

arXiv:2501.06726v2cs.ITeess.SP

TL;DR

ISEA addresses inefficiencies in connectivity-centric 5G architectures by jointly considering communication, AI computation, and sensing for intelligent perception. This survey synthesizes ISEA techniques, architectures, use cases, and interactions with 6G advances, concluding that ISEA is a promising research direction supported by standardization, industry activity, and benchmarking datasets.

  • Problem

    5G NR’s connectivity-centric design inefficiently separates communication and sensing, while sequential sensing, transmission, and computation creates latency bottlenecks for integrated sensing and AI tasks.

  • Method

    The paper presents a comprehensive survey covering ISEA architectures, digital and AirComp-based air interfaces, advanced signal processing, and their interactions with 6G techniques.

  • Results

    The survey identifies ISEA as a promising 6G research direction, with standardization initiatives, industry prototypes, and datasets supporting real-world evaluation and implementation.

  • Takeaways & Limitations

    ISEA provides a versatile framework for intelligent perception provisioning in edge networks with end-to-end optimality.

  • Takeaways & Limitations

    Conventional 5G NR uses distinct communication and sensing waveforms, causing redundant resource use and sequential-processing delays.

Abstract

from arXiv · show

Sensing and edge artificial intelligence (AI) are envisioned as two essential and interconnected functions in sixth-generation (6G) mobile networks. On the one hand, sensing-empowered applications rely on powerful AI models to extract features and understand semantics from ubiquitous wireless sensors. On the other hand, the massive amount of sensory data serves as the fuel to continuously refine edge AI models. This deep integration of sensing and edge AI has given rise to a new task-oriented paradigm known as integrated sensing and edge AI (ISEA), which features a holistic design approach to communication, AI computation, and sensing for optimal sensing-task performance. In this article, we present a comprehensive survey for ISEA. We first provide technical preliminaries for sensing, edge AI, and new communication paradigms in ISEA. Then, we study several use cases of ISEA to demonstrate its practical relevance and introduce current standardization and industrial progress. Next, the design principles, metrics, tradeoffs, and architectures of ISEA are established, followed by a thorough overview of ISEA techniques, including digital air interface, over-the-air computation, and advanced signal processing. Its interplay with various 6G advancements, e.g., new physical-layer and networking techniques, are presented. Finally, we present future research opportunities in ISEA, including the integration of foundation models, convergence of ISEA and integrated sensing and communications (ISAC), ultra-low-latency ISEA, and practicality issues.

I. INTRODUCTION

6G increasingly treats edge AI and sensing as interdependent capabilities: AI relies on sensory data, while sensing uses edge AI to extract value efficiently. ISEA emerges as a task-oriented response, integrating communications, computation, and sensing for demanding intelligent applications.

  • Edge AI uses mobile edge computing to provide low-latency, bandwidth-efficient, and more private AI applications to resource-limited devices, while relying on massive sensory data for training and inference.
  • Sensing and edge AI are interdependent: edge AI depends on sensory data, whereas sensing depends on advanced edge AI to maximize data value under communication and computing constraints.
  • ISEA addresses the challenges of integrating communications, AI, and sensing for real-time, highly reliable applications, including autonomous driving requiring 30 ms inference latency and near 100% accuracy.
  • The paper develops ISEA through a historical perspective of communications, sensing, and AI, explaining how 6G can support their convergence as a unified communication-computing platform.
  • Mobile networks evolved from voice-centric 1G and digital 2G through 4G sensor connectivity and 5G communication, computing, and sensing integration toward 6G task-oriented AI, communications, and sensing.

3) Milestones of AI:

AI evolved from foundational concepts and symbolic systems to deep learning and foundation models, while remaining constrained by computational efficiency, robustness, and ethics. This progression motivates edge AI and ISEA, which integrate AI with sensing and communication for task-oriented, end-to-end performance.

  • AI emerged as a formal field through machine-intelligence concepts, the Turing Test, the 1956 Dartmouth Conference, and early symbolic programs.
  • Large datasets, GPUs, and improved neural-network architectures drove deep-learning advances, including AlexNet’s 2012 ImageNet success.
  • Foundation models such as GPT-3 and ChatGPT extended pre-training and fine-tuning toward context-aware, human-like text generation.
  • Despite AI’s progress, computational efficiency, robustness, and ethical considerations remain challenges as AI moves toward the network edge for ubiquitous, real-time applications.
  • Sensing and AI are mutually coupled: AI backbones analyze sensing data, while sensor-generated data supports AI-model training and refinement.
  • ISEA integrates sensing-data acquisition, storage, transmission, and mobile-edge AI, jointly optimizing communication, computation, and sensing for end-to-end task performance.
  • Unlike ISAC’s RF sensing-communication integration and SemCom’s semantic transmission focus, ISEA addresses specific AI-based sensing tasks through broader multi-modal, cross-component design.
  • The survey covers ISEA preliminaries, use cases, standardization, principles, metrics, architectures, digital and AirComp interfaces, signal processing, and future research directions.

D. Paper Organization

The paper progresses from ISEA preliminaries and use cases to design principles, techniques, standardization, and future research directions.

  • Paper Organization: It then examines ISEA application scenarios and supporting techniques, alongside standardization efforts, industrial perspectives, and relevant datasets.Use cases include cooperative autonomous driving and other intelligent-perception settings.
  • Paper Organization: The survey begins with sensing data acquisition, edge AI, and new communication technologies as technical preliminaries underpinning ISEA.Edge AI is presented through edge learning and edge inference for low-latency, high-reliability sensing tasks.
  • Paper Organization: Its technical review covers sensing modalities, deep-learning-based sensing, integrated sensing and communications, data fusion, and task-oriented communication methods.The reviewed communication methods include joint source-channel coding and over-the-air computation.
  • Paper Organization: The paper concludes with future outlooks and open research issues for ISEA.The organization figure places future outlooks after the comprehensive review of ISEA techniques.

5) Multi-Modal Sensing:

Multi-modal sensing combines complementary sensor strengths to improve downstream perception, while ISEA coordinates sensing, communication, and computation under task-performance constraints.

  • 5) Multi-Modal Sensing:: LiDAR and RF sensing provide precise 3D spatial information, whereas cameras supply semantic details such as color and texture.Combining these complementary features is expected to boost sensing performance.
  • 5) Multi-Modal Sensing:: Multi-modal fusion supports downstream tasks including object detection, tracking, and semantic segmentation through early, intermediate, or late fusion.Common combinations include LiDAR with camera images and LiDAR with RF sensing.
  • 5) Multi-Modal Sensing:: Centralizing edge-collected sensory data for cloud training is undesirable because it creates privacy constraints and communication bottlenecks.These concerns motivate learning algorithms deployed within the network.
  • 5) Multi-Modal Sensing:: Split inference preserves flexibility and privacy by dividing an AI model between devices and servers, but feature uploading can become a communication bottleneck.Extracted features may exceed the dimensionality of raw data, especially when handling high-dimensional representations.
  • 5) Multi-Modal Sensing:: Task-oriented communication shifts design away from high-rate transmission toward end-to-end sensing performance with low latency.Joint source-channel coding and over-the-air computation are introduced as representative technologies for this shift.
  • 5) Multi-Modal Sensing:: ISEA coordinates AI computation and sensory-data communication for cooperative autonomous driving under stringent latency, reliability, and end-to-end accuracy demands.Vehicle-to-vehicle links exchange shared-space features, while roadside servers assist with localization, perception, and coordinated decisions.

2) Robotics Planning and Control:

ISEA applies intelligent sensing and edge analytics across robotics, digital twins, smart cities, healthcare, positioning, generative AI, and human-machine interaction.

  • 2) Robotics Planning and Control:: Robotics uses AI models to interpret language, perceive environments, make decisions, and interact with users and surroundings.Language and sensing models produce semantic and high-dimensional representations for robotic operation.
  • 2) Robotics Planning and Control:: Digital twins combine real-time IoT sensor data with AI models to create software replicas for testing, monitoring, and behavior prediction.Examples include virtual industrial sensors, digital patients, and digital cities.
  • 2) Robotics Planning and Control:: City Brain uses edge AI to analyze multimodal sensory data from millions of sensors for traffic control, emergency response, and city planning.Inter-domain data sharing and multimodal fusion support its evolution from local intelligence toward a general intelligent engine.
  • 2) Robotics Planning and Control:: Smart healthcare analyzes multimodal data from wearables, patient monitors, and cameras for health monitoring, gesture recognition, emergency response, and epidemic control.The setting is motivated by medical-data privacy, bandwidth costs, and stringent latency requirements.
  • 2) Robotics Planning and Control:: Positioning and object tracking combine satellite and terrestrial technologies to provide accurate, reliable, and low-latency localization and monitoring.Examples include GPS, BeiDou, RFID, Bluetooth Low Energy, and UWB.
  • 2) Robotics Planning and Control:: Generative AI can support versatile sensory-data understanding and automate ISEA operations such as dynamic scheduling and sensor access control.The passage highlights few-shot and zero-shot adaptation, chain-of-thought prompting, and multimodal adaptors.
  • 2) Robotics Planning and Control:: ISEA supports human-machine symbiosis by combining sensor data and AI analytics for responsive interactions and smart-home automation.Applications include recognizing voice or gestures, adapting environments to preferences, and optimizing energy efficiency and comfort.

C. Limitations of 5G Connectivity-Centric Approaches

5G connectivity-centric architectures are inefficient for integrated sensing, AI, and communication because they isolate functions, allocate resources rigidly, and ignore task semantics.

  • C. Limitations of 5G Connectivity-Centric Approaches: 5G NR separates communication and sensing waveforms, creating redundant pilot overhead and sequential-processing latency for integrated applications.These inefficiencies are especially relevant to joint positioning, tracking, and real-time sensor-data fusion.
  • C. Limitations of 5G Connectivity-Centric Approaches: Slot-based 5G scheduling cannot dynamically optimize resources for microsecond-level tracking or computing-aware distributed AI workloads.The cited workloads include split inference across devices and edge servers.
  • C. Limitations of 5G Connectivity-Centric Approaches: Conventional protocols optimize bit-level metrics rather than task-oriented quality, leaving communication capacity and sensing resolution independently optimized.The passage contrasts BER objectives with perceptual quality and sensing accuracy.
  • C. Limitations of 5G Connectivity-Centric Approaches: Emerging 6G applications require millisecond-level communication-and-computation latency, ultra-high end-to-end reliability, and scalability to thousands of devices.Existing 5G networks focus on high-throughput bit pipes and are suboptimal in end-to-end task metrics.
  • C. Limitations of 5G Connectivity-Centric Approaches: ISEA provides a versatile edge-network framework with end-to-end optimality, supported by standardization, industrial prototypes, and datasets for benchmarking.The cited datasets simulate multimodal sensing, mobility, and wireless propagation in realistic environments.

A. The Design Principles

ISEA integrates sensing, AI, and communications around end-to-end sensing-task performance rather than conventional bit-rate maximization. Its design jointly considers task metrics, communication and computation resources, and application-specific tradeoffs.

  • ISEA processes and transmits task-relevant sensory information to improve accuracy, latency, and energy efficiency.
  • Unlike rate-oriented networks, ISEA conveys sensory semantics and data characteristics rather than transmitting every bit without content awareness.
  • ISEA jointly designs communication protocols and resource allocation for AI-empowered sensing models instead of separating communication from computation.
  • End-to-end optimization crosses physical and application layers, while conventional communication metrics must be related מחדש to downstream sensing performance.
  • ISEA evaluation spans task metrics including accuracy, estimation error, coverage, resolution, and timeliness, alongside communication and computation metrics.
  • Timeliness uses metrics such as age of information, false-alarm rate, and track latency for continuous detection and tracking.
  • Communication evaluation includes throughput, latency, energy efficiency, and mobility, with latency covering sensory-data and intermediate-result exchange and aggregation.

3) Computation Metrics:

ISEA computation spans preprocessing, feature operations, fusion, and inference across heterogeneous system components. Its architecture combines sensors, edge and central brains, and actuators, while balancing sensing, communication, and computation efficiency.

  • 3) Computation Metrics:: ISEA computation efficiency is characterized by algorithmic time and space complexity across operations from raw-data preprocessing to feature extraction and fusion.
  • 3) Computation Metrics:: Time complexity, quantified by measures such as FLOPS, contributes to overall latency and informs task offloading and resource management.
  • 3) Computation Metrics:: Space complexity captures memory demands from high-dimensional data operations and motivates low-complexity algorithms for constrained devices.
  • ISEA coordinates sensing, AI, and communication for end-to-end task performance through semantics-aware links and integrated communication-computing design.
  • Its evaluation combines sensing, communication, and computation metrics, including task completion and semantic transmission rates, within resource constraints.
  • The architecture comprises sensors, edge brain, central brain, and actuators, with latency-sensitive data flowing from sensors through edge inference to actuator commands.
  • The central brain handles computation-intensive or time-insensitive tasks, global model updates, and storage of models for downloading to the edge brain.
  • Actuators receive real-time commands from the edge brain and interact with the physical world.

2) Computation Operations and AI Backbones:

ISEA computation follows a pipeline from raw sensory data to task-specific inference, with feature processing and multimodal fusion adapted to communication and computing constraints. Data moves vertically across hierarchy levels or horizontally among peers.

  • 2) Computation Operations and AI Backbones:: A typical ISEA pipeline comprises preprocessing, feature extraction, feature selection and compression, sensory fusion, and task-specific inference.
  • 2) Computation Operations and AI Backbones:: Preprocessing converts raw sensory data into model- or transmission-ready forms through operations such as denoising, filtering, and normalization.
  • 2) Computation Operations and AI Backbones:: Feature extractors generate reusable semantic representations from preprocessed data for downstream task-specific heads.
  • 2) Computation Operations and AI Backbones:: Feature selection and compression reduce communication cost by encoding informative features compactly or removing redundant dimensions.
  • 2) Computation Operations and AI Backbones:: Sensory fusion aggregates abstract features from multiple modalities or views at the edge brain, using methods such as concatenation, view-pooling, and spatial alignment.
  • 2) Computation Operations and AI Backbones:: Task-specific inference applies a specialized head to fused features, with computational overhead ranging from linear classifiers to sophisticated models.
  • 2) Computation Operations and AI Backbones:: These operations can be distributed across ISEA components according to heterogeneous computing capabilities and communication links.
  • 2) Computation Operations and AI Backbones:: ISEA uses vertical transmission for hierarchical analytics and horizontal exchange for nearby peer collaboration, while wireless links require task-specific designs under resource constraints.

B. Paradigms

ISEA offers five adaptable deployment paradigms that place models and computation across sensors, edge servers, and hierarchical or cooperative systems. These choices reflect differing latency, resource, sensing, and coordination requirements.

  • B. Paradigms: The five paradigms are split, AI-on-server, AI-on-sensor, hierarchical, and cooperative ISEA, distinguished by computation workload and communication-link configurations.
  • B. Paradigms: AI-on-sensor avoids transmission latency and suits latency-critical scenarios, but sensor storage and computation constrain model size and complexity.
  • B. Paradigms: AI-on-server offloads computation to resource-rich edge servers and supports cooperative sensing by many low-cost sensors.
  • B. Paradigms: Split ISEA divides an AI model between sensors and servers, using on-sensor feature extraction and server-side inference or training.
  • B. Paradigms: Single-sensor perception is limited by constraints such as camera field of view and LiDAR point-cloud irregularities in complex environments.
  • B. Paradigms: Hierarchical ISEA assigns sensing subtasks and analytics across components according to heterogeneous sensing, computation, and communication resources.
  • B. Paradigms: The paradigms are combinable and switchable, allowing hybrid architectures such as split for latency-sensitive tasks and AI-on-sensor for privacy constraints.
  • B. Paradigms: ISEA architecture includes sensors, edge brains, central brains, and actuators, with computation and communication distributed according to capabilities and collaboration mode.

VI. DIGITAL AIR INTERFACE FOR INTEGRATED SENSING AND EDGE AI

Digital-air-interface design for ISEA shifts from rate-oriented communication toward task-oriented QoS, using joint coding, semantics-aware access control, and resource allocation. The surveyed techniques coordinate sensing relevance, channel conditions, and computation constraints to improve downstream task performance.

  • Design motivation: Existing digital air interfaces cannot guarantee ISEA applications’ latency and reliability requirements under hostile fading and high-dimensional sensory data.ISEA therefore requires physical- and link-layer evolution oriented toward task performance.
  • Joint source-channel coding: Joint source-channel coding unifies separated coding blocks and optimizes them for ISEA QoS requirements such as inference accuracy.Some schemes directly predict labels from received symbols, avoiding signal reconstruction, while others jointly tune source and channel coding rates.
  • Joint source-channel coding: Semantic relevance can guide coding by measuring the importance of sensed features to downstream split-ISEA inference tasks.The resulting importance is incorporated into bit and subcarrier allocation optimization.
  • Access control: ISEA access control retains conventional latency, throughput, and fairness metrics but makes downstream task performance and data semantics central design objectives.Access control and radio resource allocation can overlap because both coordinate devices sharing the wireless medium.
  • Access control: Data-source selection schedules transmissions using sensory semantics and communication metrics, including channel conditions, requester feedback, and predicted sensor-data value.Query-and-feedback schemes select helping sensors with the most semantically relevant information, while mobility-aware scheduling addresses V2V settings.
  • Radio resource allocation: Task-oriented resource allocation considers sensor connectivity, observation relevance, heterogeneous channels, data qualities, and computation capabilities.Related designs extract information, select agents, allocate subchannels, and fuse messages according to expected information gain and message importance.

2) Offloading Scheduling:

ISEA offloading and AirComp designs coordinate workload placement, communication, and computation to reduce resource use and latency. AirComp integrates aggregation into wireless transmission, while its nomographic and non-nomographic variants address different task computations and introduce accuracy–noise tradeoffs.

  • Offloading Scheduling: Heterogeneous offloading requests can be scheduled for global objectives such as utility and throughput when communication and computation resources cannot serve all sensors simultaneously.A two-tier edge-AI autonomous-driving system illustrates this request-aware scheduling setting.
  • Offloading Scheduling: Joint controllers coordinate communication and computation because time-varying channels and limited on-sensor processing can both bottleneck ISEA task performance.One design jointly optimizes raw-data or feature-upload mode, transmit power, and computational speed to minimize total energy consumption.
  • AirComp-based air interface: AirComp reduces aggregation latency by exploiting waveform superposition for simultaneous transmissions, making latency scale as O(1) with the number of devices.This contrasts with orthogonal access, whose communication latency grows with device count.
  • Nomographic Data Aggregation: AirComp aggregates nomographic functions by transmitting pre-processed device outputs simultaneously, then applying post-processing to reconstruct the desired function despite channel distortion.Weighted sums support applications including federated-learning gradient aggregation, model fusion, and multi-view feature pooling.
  • Nomographic Data Aggregation: High-dimensional AirComp supports vector operations such as neural-network vector–matrix multiplication, distributing feature vectors across devices and subcarriers.Multiplexing, diversity, or mixed broadband operation can accommodate high-dimensional gradients and features.
  • Non-nomographic Data Aggregation: Non-nomographic aggregation such as max pooling requires functional approximation, creating a tradeoff between approximation error and noise amplification.The vector p-norm is used to approximate the element-wise maximum in multi-view sensing.
  • AirComp-based air interface: AirComp-based ISEA must combine wireless aggregation with subsequent distributed analytics or estimation, motivating end-to-end accuracy analysis and adaptive transceiver designs.One federated-learning design jointly adapts learning rate and AirComp transmission to time-varying wireless conditions and importance-aware distortion.

B. Radio Resource Management

ISEA radio resource management replaces conventional rate maximization with task-oriented control of power, scheduling, subcarriers, and interference. The survey also emphasizes that practical AirComp must address imperfect channels, mobility, interference, and the gap between theoretical assumptions and deployment conditions.

  • Power Control: AirComp power control must account for non-IID, non-ergodic sensory data and task relevance because classical MSE-minimization assumptions can be suboptimal for ISEA.Gradient statistics, including sensor- and round-dependent means and variances, can determine adaptive power policies.
  • Power Control: Channel noise can sometimes assist optimization by helping aggregated gradients escape saddle points, so suppressing MSE is not always appropriate.A region-adaptive policy suppresses noise in non-stationary regions but reduces power in saddle regions.
  • Sensor Scheduling: Sensor scheduling should jointly consider channel states and observation importance, including deep-fading exclusion and the number of participating sensors.AirComp scheduling has been formulated as joint sensor selection and receive beamforming under MSE and power constraints.
  • Sensor Scheduling: Over-the-air feature fusion can exclude sensors with empty voxel feature vectors using sparsity-pattern feedback before deriving power control.This couples semantic feature structure with transmission participation.
  • Subcarrier Assignment: Broadband AirComp can distribute high-dimensional gradients or features across subcarriers through multiplexing, diversity, or a mixture of both.These strategies address heterogeneous tasks and different channel fades across subcarriers.
  • Practical AirComp: Practical AirComp still faces a gap between established theory and real-world deployment, requiring robustness, compatibility, and prototype validation.The survey identifies implementation issues as a distinct practical concern.
  • Robust AirComp: Perfect synchronization and accurate CSI may be impractical for mission-critical, massive-access ISEA because feedback restrictions and large-scale networking increase overhead.Robust designs therefore model channel-estimation errors stochastically or deterministically and target reliable functional computation.
  • Robust AirComp: Moving vehicles and UAVs can make wireless connections asynchronous, while synchronization time offsets depend inversely on leveraged bandwidth.Mobility therefore adds a deployment-specific synchronization challenge beyond fixed-sensor settings.

3) Demo Development:

On-the-fly ISEA replaces sequential high-dimensional processing with streaming, low-dimensional computation and communication, while FlyCom2 and related methods target latency, device complexity, and energy constraints. The section also considers incremental fine-tuning and online processing for task-specific sensing systems.

  • On-the-Fly Computing and Communication: ISEA sensors face million-point datasets and energy-intensive analytics that challenge lightweight devices and network lifetime.These constraints motivate distributing computation and communication through task-specific on-the-fly designs.
  • On-the-Fly Computing and Communication: FlyCom2 breaks sequential high-dimensional processing into streaming low-dimensional communication and computation for ISEA.It uses random sketching to reduce on-device complexity and supports efficient edge processing.
  • On-the-Fly Computing and Communication: Parallel streaming communication and computation halves latency compared with sequential traditional one-shot algorithms.Accumulated local sketches also let the server infer distributed patterns and provide graceful degradation under link disruptions or packet losses.
  • On-the-Fly Computing and Communication: FlyBoosting progressively transfers and combines lightweight prompts to broaden solvable sensing problems during device-edge cooperative fine-tuning.The approach supports communication-and-computation-efficient adaptation of prompt ensembles to sensing requirements.
  • On-the-Fly Computing and Communication: Online algorithms support real-time ISEA by making immediate decisions from continuously arriving data without requiring the complete dataset.Random sketching is one example for learning principal components from streaming data vectors.

B. ISEA with Multi-Functional Waveforms

ISEA combines multifunctional waveforms, AI-assisted signal processing, privacy-preserving aggregation, and emerging 6G technologies to jointly support sensing and communication. The resulting designs must balance sensing accuracy, communication efficiency, privacy, energy, and deployment complexity.

  • Multi-Functional Waveforms: Multifunctional waveforms integrate sensing, communication, localization, and radar functions into unified signal structures.Recent simultaneous monostatic sensing and communication designs incorporate communication-message uncertainty without compromising coherent sensing, while avoiding cooperation overhead.
  • AI-Assisted Processing: AI models extract information from multifunctional-waveform sensory data and support end-to-end metrics such as accuracy, IoU, and semantic data rate.Examples include intelligent micro-Doppler detection and radio-map construction.
  • Privacy-Preserving Signal Processing: AirComp aggregation conceals individual information through signal superposition and channel noise, making it inherently more privacy-preserving than orthogonal access.AirComp-based federated learning can achieve a certain differential-privacy level without affecting learning performance, while stronger privacy may require added noise or dimension reduction.
  • Advanced 6G Technologies: XL-MIMO can improve ISEA sensing and support massive sensor access through large antenna arrays and near-field beamforming in direction and distance.It supports target detection, localization, tracking, focused imaging, and simultaneous access by densely placed sensors.
  • Reconfigurable Intelligent Surfaces: RIS-enabled ISEA creates additional line-of-sight links, controllable propagation paths, and adaptive beam scanning for wireless sensing.Deployment must jointly consider sensing accuracy, communication efficiency, coverage, power control, and security.

B. Space-Ground Empowered ISEA

Space-ground ISEA extends integrated sensing, communication, and computation across satellites, UAVs, ground devices, edge networks, and foundation-model infrastructure. These architectures address non-terrestrial sensing and resource constraints through onboard or hierarchical processing, while targeting low-latency intelligent operations.

  • Space-Ground Empowered ISEA: Space and ocean ISEA supports global connectivity while accommodating distinct sensing modalities, mobility, propagation, computing, topology, and QoS characteristics.Non-terrestrial sensing can include orbital imagery and underwater sonar.
  • Space-Ground Empowered ISEA: Onboard satellite computation filters and semantically matches sensed data to reduce raw-data transmission over low-rate downlinks.Orbital edge computing uses energy-efficient GPUs, intelligent early discard, and parallel tile processing across satellites.
  • Space-Ground Empowered ISEA: Satellite edge architectures can use ground devices for sensing while satellites act as overarching edge servers.A satellite-UAV network can jointly analyze sensing data and generate control commands for field robots under latency constraints.
  • Edge-Cloud Networking: ISEA may require edge-cloud cooperation or multiple edge networks for hierarchical federated learning, model downloading, and heavy-computation offloading.Best-effort protocols cannot guarantee low latency and high reliability for every such request.
  • Foundation Models: Foundation models can act as intelligent agents that understand sensing data and network conditions to adapt configurations for human commands.ISEA adaptation includes direct prompting for reasoning and planning or task-aware fine-tuning with sensing data, while deployment spans devices, edges, and cloud.

B. Convergence of ISEA and ISAC

ISEA and ISAC have distinct technical motivations but can mutually assist one another within 6G through shared sensing information, AI, spectrum, and hardware. Their convergence creates synchronization, resource-allocation, and sensing–communication tradeoffs that require joint optimization.

  • ISEA optimizes communication and computation for AI-empowered sensing, whereas ISAC jointly designs RF sensing and communication over shared hardware and spectrum.
  • High-resolution ISEA sensing can support ISAC through scene identification, link-blockage prediction, user localization, and improved beamforming and power allocation.It can also reduce pilot overhead, especially in high-mobility scenarios.
  • AI models can translate between sensing domains, enhancing ISAC data through localization refinement, time-domain interpolation, and vision-aided generative reconstruction.Improved ISAC sensing can save spectrum resources without sacrificing sensing performance.
  • Converged ISEA–ISAC systems must integrate heterogeneous sensing data while addressing time synchronization, coverage differences, and relative-location measurement errors.The survey identifies these as practical challenges for multimodal sensing integration.
  • Shared spectrum and hardware create resource-allocation problems spanning ISAC waveforms, user traffic, ISEA feature traffic, and multimodal sensing computation.Allocating more sensing resources can improve resolution but require stronger feature compression, degrading transmission reliability.
  • Ultra-low-latency ISEA tasks require sensing, on-sensor computation, and feature transmission to complete within strict end-to-end latency constraints.Communication latency must remain below several milliseconds for demanding applications.

2) Energy Constraints:

ISEA faces practical energy, interoperability, scalability, and overhead constraints as sensing, AI, and communication are jointly deployed across heterogeneous 6G networks. The survey identifies architectural and algorithmic directions for managing these burdens while preserving end-to-end task performance.

  • 2) Energy Constraints:: ISEA consumes substantial energy because sensing, AI inference and training, and wireless communication operate jointly on resource-constrained edge devices.The concern is especially acute for real-time, mission-critical applications.
  • 2) Energy Constraints:: 10 to 30 watts: a typical LiDAR module’s power consumption creates challenges for battery-powered drones and IoT sensors.
  • 2) Energy Constraints:: Federated learning reduces data transmission but still incurs high energy costs for local model updates.
  • 2) Energy Constraints:: High-dimensional sensory data and model updates make communication energy-intensive, particularly in mmWave or THz bands and during frequent cooperative exchanges.
  • Heterogeneous sensing modalities, devices, protocols, and network dynamics complicate synchronization, interoperability, distributed inference, and cooperative sensing.Examples include RF, LiDAR, and RGB-D data with different formats, rates, scales, and protocols.
  • Standardized data formats, semantic communication, task-adaptive modulation and coding, and LLM-empowered network agents are proposed to improve interoperability and dynamic cooperation.
  • Joint ISEA design adds signaling and computation overhead, especially in ultra-low-latency or massive-access applications.Cross-domain beamforming optimization and AI task-QoS monitoring burden network controllers.
  • The survey establishes ISEA principles and architectures, reviews enabling techniques using end-to-end metrics, and discusses future research directions.
Loading 2501.06726v2…