PaperlayerView exact revision on arXiv

Source-linked AI summary

Intelligent Multi-Modal Sensing-Communication Integration: Synesthesia of Machines

Xiang Cheng, Haotian Zhang, Jianan Zhang, Shijian Gao, Sijiang Li, Ziwei Huang, Lu Bai, Zonghui Yang, Xinhu Zheng, Liuqing Yang

arXiv:2306.14143v3eess.SP

TL;DR

Existing ISAC and its surveys remain centered on RF-based radar sensing, while multi-modal integration lacks a comprehensive review. This paper defines SoM as a unified framework, reviews its paradigm and applications, and reports a high-SNR waveform estimation gain over OFDM.

  • Problem

    Existing ISAC and its surveys focus mainly on RF-based radar sensing, and multi-modal sensing-communication integration still lacks a comprehensive review.

    Source: §I · p. 3, §IX · p. 38

  • Method

    The paper defines SoM as a unified framework with three operational modes and advocates ANN-based processing designed in a model-enhanced, data-driven manner.

    Source: C. SoM Processing · p. 10, §IX · p. 38

  • Results

    At high SNR and the same transmission power, the superimposed waveform gains up to 5 dB estimation gain over OFDM.

    Source: 2) Case Study: A Dual-Function… · p. 24

  • Takeaways & Limitations

    SoM offers a holistic design perspective for multi-modal integration, particularly in dynamic scenarios.

    Source: §IX · p. 38

    SoM-evoke and SoM-enhance research remains at an early stage because aligned multi-modal sensory and wireless channel datasets are lacking.

    Source: E. Current Research Status of SoM · p. 11

Abstract

from arXiv · show

In the era of sixth-generation (6G) wireless communications, integrated sensing and communications (ISAC) is recognized as a promising solution to upgrade the physical system by endowing wireless communications with sensing capability. Existing ISAC is mainly oriented to static scenarios with radio-frequency (RF) sensors being the primary participants, thus lacking a comprehensive environment feature characterization and facing a severe performance bottleneck in dynamic environments. To date, extensive surveys on ISAC have been conducted but are limited to summarizing RF-based radar sensing. Currently, some research efforts have been devoted to exploring multi-modal sensing-communication integration but still lack a comprehensive review. Therefore, we generalize the concept of ISAC inspired by human synesthesia to establish a unified framework of intelligent multi-modal sensing-communication integration and provide a comprehensive review under such a framework in this paper. The so-termed Synesthesia of Machines (SoM) gives the clearest cognition of such intelligent integration and details its paradigm for the first time. We commence by justifying the necessity of the new paradigm. Subsequently, we offer a definition of SoM and zoom into the detailed paradigm, which is summarized as three operation modes. To facilitate SoM research, we overview the prerequisite of SoM research, i.e., mixed multi-modal (MMM) datasets. Then, we introduce the mapping relationships between multi-modal sensing and communications. Afterward, we cover the technological review on SoM-enhance-based and SoM-concert-based applications. To corroborate the superiority of SoM, we also present simulation results related to dual-function waveform and predictive beamforming design. Finally, we propose some potential directions to inspire future research efforts.

I. INTRODUCTION · A. Related Work · B. Contributions and Organization of This Paper

The paper argues that radar-focused RF-ISAC is insufficient for dynamic B5G/6G scenarios and proposes Synesthesia of Machines (SoM) as a unified framework for intelligent multi-modal sensing-communication integration. It surveys the framework, its research foundations and applications, and future directions.

I. INTRODUCTION

  • RF-ISAC is limited to radar sensing and struggles to characterize physical environments or address mobility-driven performance bottlenecks in emerging scenarios.Multi-modal sensing can add diverse communications-related features from non-RF sensors, while data from multiple agents can reinforce sensing through communication networks.Source: §I · p. 3

A. Related Work

  • Existing surveys, including,, and, do not systematically review intelligent multi-modal integration or establish its unified framework and design paradigm.Earlier RF-ISAC surveys largely focus on radar sensing, while and consider multi-modal sensing but lack technical detail or comprehensive review of research progress.Source: A. Related Work · pp. 3–4

B. Contributions and Organization of This Paper

  • The paper proposes SoM by generalizing human synesthesia to machine senses, establishing a systematic framework that expands RF-ISAC to diverse sensing objectives, design methods, and scenarios.The survey presents SoM as the first framework centered on intelligent multi-modal sensing-communication integration.Source: B. Contributions and Organization of… · p. 4, A. Related Work · p. 4
  • SoM’s paradigm defines three operational modes—SoM-evoke, SoM-enhance, and SoM-concert—for mutual enhancement between multi-modal sensing and communications.The paper presents these modes as a systematic approach to understanding intelligent multi-modal integration.Source: B. Contributions and Organization of… · p. 4
  • The survey covers mixed multi-modal datasets, mapping relationships, SoM-enhance and SoM-concert applications, and RF-based radar integration as a special case of SoM.The dataset review is presented as a guide for researchers, and the application areas had not yet been systematically summarized.Source: B. Contributions and Organization of… · p. 4
  • The paper presents simulations on sensing-communication mapping, dual-function waveform design, and SoM-enhance predictive beamforming.It also offers observations and insights on SoM research.Source: B. Contributions and Organization of… · p. 5
  • The proposed research directions target stronger foundations through MMM datasets and mapping studies, alongside future communication and sensing applications of all three SoM modes.These directions are organized around research foundations and future applications.Source: B. Contributions and Organization of… · p. 5
  • The paper proceeds from SoM motivation and framework to datasets, sensing-communication mapping, SoM-enhance and SoM-concert applications, open issues, and conclusions.Sections II–IX cover these topics in sequence.Source: B. Contributions and Organization of… · p. 5

II. INTELLIGENT MULTI-MODAL · A. Emerging Application Scenarios: Opportunities & Challenges · B. Intelligent Multi-Modal Sensing-Communication Integration

High-mobility 6G scenarios and their abundant environmental information motivate a shift beyond RF-centric ISAC toward multi-modal sensing-communication integration. The section outlines emerging application needs, dynamic-system challenges, and why RF-ISAC alone is insufficient.

II. INTELLIGENT MULTI-MODAL

  • Emerging mobility and rich environmental information call for a paradigm shift from extensively studied RF-based integration toward multi-modal sensing-communication integration.The section examines dynamic-scenario challenges and the potential of incorporating multi-modal sensing.Source: §II · p. 5

A. Emerging Application Scenarios: Opportunities & Challenges

  • Intelligent vehicles, UAVs, logistics, and factories require high communication and sensing performance, with localization and sensing supporting machine coordination and motion capture.Self-driving cars and UAVs are examples of intelligent mobile manned platforms; machine actions in logistics and factories can simplify or change system design and reduce signaling overhead.Source: A. Emerging Application Scenarios:… · pp. 5–6
  • SAGSIN targets global space-air-ground-sea coverage, while smart cities and Industry 5.0 require high-performance services, user-centric design, and stringent QoS, privacy, and security.SAGSIN supports terminals moving up to 1000 km/h; smart-city visions also require computing resources and big-data analytics, and Industry 5.0 emphasizes human-centric technologies, including AI and machine-to-machine communications.Source: A. Emerging Application Scenarios:… · p. 6
  • Simultaneous mobility of receivers, transmitters, and scatterers rapidly changes wireless channels, demanding frequent channel updates and real-time resource scheduling.Conventional approaches for low-speed or static scenarios can fail in dynamic settings because of communication overhead and latency.Source: A. Emerging Application Scenarios:… · p. 6
  • Moving sensors produce distorted data, while limited acquisition frequency and sensing range shorten data usefulness and impede timely perception during high-speed movement.LiDAR range measurements can be distorted by changing round-trip distance; larger sensing ranges give agents more time to detect objects and decide how to act.Source: A. Emerging Application Scenarios:… · p. 6

B. Intelligent Multi-Modal Sensing-Communication Integration

  • RF-ISAC’s shared electromagnetic-wave basis yields hardware-level integration gains largely limited to motion measurement, insufficient for global, fine-grained characterization of rapidly changing environments.The paper therefore identifies multi-modal sensing as central to the paradigm shift needed to address emerging dynamic-scenario challenges.Source: B. Intelligent Multi-Modal… · p. 6

1) The Potential of Multi-Modal Sensing-Communication Integration: … 1) Machine Senses:

The paper motivates intelligent multi-modal sensing-communication integration as a way to characterize dynamic environments beyond RF sensing alone, then introduces SoM as a unified framework inspired by human synesthesia and machine senses.

1) The Potential of Multi-Modal Sensing-Communication Integration:

  • Multi-modal sensing captures physical-environment features across domains and has supported communication tasks including link selection, beam alignment, and resource allocation –.The paper describes extracting RF environment-related features in abstract formats to facilitate communication-system applications.Source: 1) The Potential of Multi-Modal… · p. 6
  • B5G/6G networks with native sensing can use environmental features such as scatterer velocity and position to reduce overhead and latency, while communication links support broader, beyond-vision-range sensing,.The paper envisions sensing and communication functionalities operating in symbiosis, with multi-modal sensors and communications equipment helping depict the physical environment.Source: 1) The Potential of Multi-Modal… · pp. 6–7

2) RF-ISAC Versus Intelligent Multi-Modal Sensing-Communication Integration:

  • In dynamic vehicular scenarios, RF-ISAC motion measurements alone may not recover accurate CSI, while camera, LiDAR, and radar features can supply complementary environmental information.The case study notes that frequent conventional channel estimation incurs signaling overhead and latency; multimodal features can help reconstruct surroundings and improve communication functionality.Source: 2) RF-ISAC Versus Intelligent… · p. 7

III. SYNESTHESIA OF MACHINES

  • Prior multi-modal studies span communication tasks and object detection, tracking, and segmentation, but remain fragmented without a cohesive framework.The paper situates these works within intelligent multi-modal sensing-communication integration research and presents SoM as a unified framework for reviewing them.Source: §III · p. 7, 2) RF-ISAC Versus Intelligent… · p. 7

A. Background of Synesthesia

  • Human synesthesia involves one sense being involuntarily evoked by another; researchers proposed that sensory neurons may cross-activate because brain areas processing different senses are interconnected –.Grapheme-color synesthesia is described as a commonly studied form in which seeing letters or numbers evokes corresponding colors.Source: A. Background of Synesthesia · pp. 7–8

1) Machine Senses:

  • The paper analogously calls the diverse environmental information collected by IoT sensors and communication devices “machine senses,” including LiDAR, radar, and RGB-D cameras.Radar collects target information through electromagnetic waves, LiDAR produces dense 3D point clouds, and communication modules estimate the electromagnetic propagation environment; these capture different environmental properties across domains.Source: 1) Machine Senses · p. 8

2) Definition: … D. Objectives and Challenges of SoM

Synesthesia of Machines (SoM) processes information from multiple machine senses to reinforce environmental sensing and enhance communications and networking. Its framework defines three operational modes and positions intelligent, interpretable learning as a foundation for achieving these aims.

2) Definition:

  • SoM is defined as synesthesia processing of environment information collected by machine senses, reinforcing environmental sensing and enhancing communications and networking.Unlike human synesthesia, SoM can potentially involve multiple machine senses.Source: 2) Definition · p. 8, 3) Operational Modes · p. 8

3) Operational Modes:

  • SoM-evoke infers information when its corresponding sense is inactive, SoM-enhance improves communications using sensing data, and SoM-concert fuses information to strengthen sensing tasks.SoM-enhance can reduce communication-module overhead; SoM-concert can combine RGB image semantics with LiDAR structure for object detection and classification.Source: 3) Operational Modes · pp. 8–9
  • RF-ISAC is a special case of SoM, integrating radar sensing and communications through a unified waveform and hardware, but only across two RF senses without guaranteed performance gains.The passage particularly notes the lack of guaranteed gains in dynamic scenarios.Source: 3) Operational Modes · p. 9

C. SoM Processing

  • SoM’s core processing challenge is learning representations across heterogeneous multimodal data and modeling complex nonlinear relationships to enable mutual assistance among senses.RGB images, LiDAR point clouds, and RF propagation data differ in structure, semantics, and operating bands; conventional model-based methods struggle to incorporate and fuse such information.Source: C. SoM Processing · p. 9
  • SoM favors theoretically designed, interpretable ANN architectures over black-box data-driven selection, while complementary learning methods address labeling, generalization, and privacy constraints.Examples include deep reinforcement learning, meta-learning, and federated learning, with applications such as fast beamforming in mMIMO and beam selection –.Source: C. SoM Processing · p. 10

D. Objectives and Challenges of SoM

  • SoM is considered feasible because machine-sense data is interpretable, its basis includes scientific observations, and ANNs have studied architectures and mechanisms.These factors provide SoM with interpretable data, implementation tools, and an objective basis for study, making it a promising B5G/6G candidate.Source: D. Objectives and Challenges of SoM · p. 10
  • SoM aims to infer channel information without communication-module involvement, improve limited-functionality communications without extra resources, and expand sensing through multimodal fusion.Examples include estimating channel characteristics for site placement, using receiver position and motion to reduce mmWave channel-estimation overhead and latency, and sharing vehicle data for beyond-view-range sensing,.Source: D. Objectives and Challenges of SoM · p. 10
  • SoM must establish mapping relationships across senses with different data structures, functions, principles, and frequency bands; aligned multimodal and wireless-channel datasets are scarce.For example, processing radar at 77 GHz with communications at 28 GHz requires mapping between their electromagnetic environments.Source: D. Objectives and Challenges of SoM · pp. 10–11

E. Current Research Status of SoM … A. Existing Datasets for Sensing and Communication Research

SoM broadens RF-ISAC into an intelligent multi-modal sensing-communication framework with three operating modes, while current research and dataset availability remain uneven. The sections summarize its research status and establish mixed multi-modal datasets as a prerequisite for further study.

E. Current Research Status of SoM

  • Research remains mostly limited to RF-ISAC; SoM-evoke is open, SoM-enhance has few studied topics, and RF-ISAC integration does not ensure both functions improve.SoM-evoke and SoM-enhance are still at an early stage, in part because aligned multi-modal sensory and wireless-channel datasets are lacking.Source: E. Current Research Status of SoM · p. 11
  • SoM-concert research has gained attention in sensor fusion and multi-vehicle cooperation, where communication transmits information and multi-modal processing can improve vehicle sensing range, accuracy, and reliability.Source: E. Current Research Status of SoM · p. 11

F. Key Takeaways

  • SoM incorporates multi-modal sensing beyond RF and organizes its processing into three modes: SoM-evoke, SoM-enhance, and SoM-concert.Its objective is to improve communication and sensing systems across more aspects than existing RF-ISAC techniques.Source: F. Key Takeaways · p. 11
  • Unlike RF-ISAC’s emphasis on signal processing and optimization, SoM emphasizes ANNs to uncover complex mappings and support multi-layer, multi-objective performance enhancement.SoM targets generic and challenging dynamic scenarios, seeking to optimize overall communication and sensing systems rather than only trade off radar and communication functions.Source: F. Key Takeaways · p. 11

IV. PREREQUISITE OF SOM RESEARCH: THE MIXED MULTI-MODAL (MMM) DATASETS

  • High-quality multi-modal environment data is a prerequisite for SoM, and simulation datasets offer a complexity-quality trade-off where real-world data cannot flexibly customize scenarios.The paper identifies high-quality data as essential for ANN learning and notes that simulation platforms have been validated for accuracy through measurement campaigns.Source: §IV · p. 12

A. Existing Datasets for Sensing and Communication Research

1) Dataset Classification: · 2) Existing Datasets:

The section classifies sensing-and-communication datasets by their wireless-channel and sensory-data content, then surveys examples across the resulting categories. It also describes dataset limitations and the costs and reliability considerations of measurement and simulation.

1) Dataset Classification:

  • The taxonomy distinguishes COM channel-only, NUM single-modality non-RF, and NMM multi-modal non-RF datasets.COM contains wireless channel information; NUM and NMM contain only non-RF sensory data, respectively uni-modal and multi-modal.Source: 1) Dataset Classification · p. 12
  • It further defines RUB and RMB by whether communication and radar share a band, and MUM and MMM by whether non-RF sensing is uni- or multi-modal.These categories combine wireless channel data with RF or non-RF sensory data.Source: 1) Dataset Classification · p. 12

2) Existing Datasets:

  • DeepMIMO [86] offers parameterized ray-tracing channel data, while 5GMdata combines traffic and ray-tracing simulators for moving transceivers and objects.DeepMIMO parameters can be modified to suit users’ needs.Source: 2) Existing Datasets · p. 12
  • LiDARsim produces realistic LiDAR point clouds using real-world data and a DNN, with intensity simulation and conditional generation for different weather.Its weather-related generation supports simulated data under varying conditions.Source: 2) Existing Datasets · p. 12
  • V2X-Sim supports multi-agent, multi-modal autonomous-driving sensing; OPV2V provides a simulated V2V benchmark with cameras, LiDAR, and GPS/IMU.V2X-Sim includes diverse ground truths and an open-source collaborative-sensing testbed.Source: 2) Existing Datasets · p. 12
  • SHIFT is a multi-task synthetic autonomous-driving dataset that varies time of day and cloud, rain, and fog conditions.It uses realistic discrete and continuous shifts to simulate sensory data under different driving-safety-related conditions.Source: 2) Existing Datasets · pp. 12–13
  • WALDO uses measured IEEE 802.11ay PHY and multistatic-radar data for indoor human localization; Radar-COM simulates FMCW radar and communications on different bands; LiDAR-COM combines LiDAR data with ray-tracing simulations.Source: 2) Existing Datasets · p. 13
  • ViWi combines visual and parameterized wireless-channel data but has RSU-only sensors and no inter-vehicle channels, limiting V2V research; e-FLASH combines camera, GPS, LiDAR, and RF ground truth.e-FLASH includes seven real-world vehicular network scenarios spanning LoS and NLoS conditions.Source: 2) Existing Datasets · p. 13

B. Necessary Properties of the Desirable MMM Datasets · C. M3SC: A Generic Dataset for Intelligent Multi-Modal Sensing-Communication Integration

The paper defines desirable MMM datasets for SoM and presents M3SC, a simulation dataset combining aligned sensing and channel data in a dynamic VCN crossroad scenario with customizable conditions.

B. Necessary Properties of the Desirable MMM Datasets

  • Desirable MMM datasets must align multi-modal sensory data with wireless channel data and collect environment data from different agents.The paper says these properties support mutual promotion between sensing and communications, comprehensive scenario coverage, scalability, and multiple-target research.Source: B. Necessary Properties of the… · pp. 13–14
  • Desirable datasets should model realistic details such as lighting, weather, materials, and complexity, but existing simulations have rough rendering and lack weather or time variation.Real-world datasets may meet these requirements but consume substantial resources and cannot flexibly or promptly construct target scenarios; the paper calls for flexible, high-quality MMM simulation datasets.Source: B. Necessary Properties of the… · p. 14

C. M3SC: A Generic Dataset for Intelligent Multi-Modal Sensing-Communication Integration

  • M3SC combines aligned multi-modal sensing and wireless channel data for a dynamic VCN crossroad, considering mMIMO and mmWave communications.It is constructed using Wireless InSite®, AirSim, and WaveFarer® for channel, multi-modal sensory, and radar simulation, respectively.Source: C. M3SC: A Generic Dataset for… · pp. 14–15
  • M3SC provides RGB images, depth maps, LiDAR point clouds, mmWave radar data, and 1,500 snapshots of V2V and V2I channel data.Its ten radars each use eight Tx and four Rx antennas; channel arrays use 32 antennas on vehicles and 128 on RSUs, at 28 GHz with 2 GHz bandwidth.Source: C. M3SC: A Generic Dataset for… · p. 15
  • AirSim weather and time settings are matched in the wireless and radar instances by manually adding snow or water drops and tuning visual rendering conditions.AirSim can adjust snow, rain, and fog, while time settings affect sky brightness and reflected-light intensity.Source: C. M3SC: A Generic Dataset for… · p. 15
  • M3SC simulation accuracy remains to be validated on an experimental hardware platform, which the authors say is challenging and time-consuming to construct.Their validation approach also relies on software checked by measurement campaigns and simulation parameters configured for realistic propagation and commercial-product settings.Source: C. M3SC: A Generic Dataset for… · p. 15
  • M3SC is presented as the first MMM simulation dataset supporting flexible scenario customization and aligned environment data across adjustable weather and times of day.The examples include vehicle views in sunny and snowy conditions and RSU views during day and night.Source: C. M3SC: A Generic Dataset for… · p. 15

D. Key Takeaways … B. Challenges

MMM datasets support research into mappings between multi-modal sensing and communications, while SoM-customized channel modeling provides a pathway for exploring those mappings. The passages describe dataset needs and challenges in connecting sensory data, fading, and scattering clusters.

D. Key Takeaways

  • Aligned multi-modal sensory and wireless-channel datasets are scarce, and real-world datasets cannot flexibly or promptly meet SoM research needs, motivating customizable simulation platforms.Such platforms can support specific research scenarios and equipment-parameter adjustments.Source: D. Key Takeaways · p. 15
  • MMM datasets enable study of mappings between physical objects and electromagnetic scattering clusters and support subsequent SoM applications in communication and sensing systems.Object-detection ANNs can process sensory data, while clustering algorithms can identify scattering clusters from wireless-channel data.Source: D. Key Takeaways · pp. 15–16

V. CHANNEL MODELING: THE PATHWAY TOWARDS MAPPING RELATIONSHIPS BETWEEN MULTI-MODAL SENSING AND COMMUNICATIONS

  • Channel modeling is proposed as the pathway for exploring complex mappings between multi-modal sensing and wireless communications, which are foundational to SoM.Multi-modal sensors capture physical-environment information, while channel modeling characterizes the RF propagation environment.Source: §V · pp. 16–17, A. Channel Modeling Customized for… · p. 17

A. Channel Modeling Customized for Mapping Relationships

  • SoM-specific channel models must link sensor-observed environments to large-scale fading and realistically model object-based scattering clusters and their evolution for small-scale fading –, –.Large-scale fading includes path loss and shadowing; accurate small-scale fading characterization requires modeling scattering clusters, which correspond to physical objects.Source: A. Channel Modeling Customized for… · p. 17

B. Challenges

  • Differences in propagation across non-RF and RF bands weaken sensory-channel correlation, while frequency-dependent path loss and mmWave channel frequency nonstationarity complicate mapping.The passage attributes mmWave nonstationarity to the failure of the uncorrelated-scattering assumption that holds in sub-6 GHz.Source: B. Challenges · p. 17
  • Multi-modal sensory data are discrete 2D or 3D structures, whereas path loss and shadowing are continuous one-dimensional data, creating a representation mismatch.Source: B. Challenges · p. 17
  • Smooth space-time evolution of scattering clusters is difficult to capture because scatterers appear and disappear, yet it is needed for spatially consistent, smoothly evolving channel impulse responses.RGB-D, radar, and LiDAR can help identify, track, and characterize potential clusters and their motion.Source: B. Challenges · pp. 17–18

C. State-of-the-Art · 1) Mapping Relationship Between Multi-Modal Sensing and Large-Scale Fading: · 2) Mapping Relationship Between Multi-Modal Sensing and Small-Scale Fading:

The section examines how multi-modal sensing maps to large- and small-scale channel fading. It reports an RGB-D path-loss prediction study and reviews channel-consistency modeling methods, their trade-offs, and data limitations.

1) Mapping Relationship Between Multi-Modal Sensing and Large-Scale Fading:

  • The UAV scheme detects buildings in RGB images, matches them to aligned depth maps for density, height, and relative position, then uses a fully connected network to predict path loss.AirSim supplies aligned RGB-D data based on the M3SC dataset, while Wireless InSite® supplies ray-tracing path-loss data.Source: 1) Mapping Relationship Between… · p. 18, Figure 9 · p. 18
  • 5.32 ×10−4 normalized mean square error was obtained for the RGB-D path-loss prediction scheme, supporting accurate prediction from UAV images and depth maps.The result uses Wireless InSite® ray-tracing data as ground truth; predicted path loss can guide UAV networking and help reduce communication energy loss.Source: Figure 10 · p. 18, 1) Mapping Relationship Between… · p. 18

2) Mapping Relationship Between Multi-Modal Sensing and Small-Scale Fading:

  • Channel consistency means channel variation remains smooth as space or time evolves; complex vehicle trajectories lower temporal correlation and make capturing time consistency important.Adjacent antennas or moments share effective scattering clusters, whose evolution is smooth rather than random.Source: 2) Mapping Relationship Between… · pp. 18–19
  • Deterministic models can capture time consistency by smoothly updating scattering clusters and channel parameters, but are complex; the review reports no SoM channel-consistency work based on NGSMs.The review classifies current models as GBDMs, NGSMs, and GBSMs.Source: 2) Mapping Relationship Between… · p. 19
  • Standardized GBSMs mimic time consistency by generating temporally consistent channel parameters, a low-complexity, adaptable approach that neglects propagation geometry and has low accuracy.3GPP TR38.901, METIS, and IMT-2020 use such parameters; 3GPP TR38.901 supports 0.5 GHz to 100 GHz and large-scale antenna arrays.Source: 2) Mapping Relationship Between… · pp. 19–20
  • COST 2100’s visibility-region method captures time consistency through smooth changes in effective clusters, while extensions in, also capture space consistency; realistic real-time tracking is prohibitively complex.The extensions assign each cluster mobile-station and base-station visibility regions, so nearby antenna elements observe similar clusters.Source: 2) Mapping Relationship Between… · p. 20
  • The GBSM in combines visibility regions with temporally consistent parameters and soft cluster-power handover to capture time-space channel consistency.It uses space-VR and time-VR centered on clusters and antenna elements, respectively, with a visible factor for smooth power variation.Source: 2) Mapping Relationship Between… · p. 20
  • The ML-based GBSM in uses spectral clustering of scattering-cluster and transceiver coordinates to capture time-space consistency, but depends on parameter settings and scarce MMM datasets.The review describes this approach as offering a trade-off among accuracy, scalability, and complexity, and as still in its infancy.Source: 2) Mapping Relationship Between… · pp. 20–21

D. Key Takeaways · VI. SOM-ENHANCE: ENHANCED COMMUNICATION FUNCTIONALITY · A. SoM-Enhance-Based Channel Estimation in Dynamic Scenarios

SoM-enhance uses multimodal sensing to infer communication side information and provide concrete CSI, with particular promise for channel estimation in dynamic environments. Realizing this requires addressing complex cross-modal mappings and the limitations of conventional estimation.

D. Key Takeaways

  • Cross-modal mapping is difficult because sensing and communication differ in propagation effects and representations, while object-to-cluster relationships are complex and nonlinear.The paper identifies ANNs as suitable for modeling these nonlinear relationships and emphasizes finding features that connect the different data representations.Source: D. Key Takeaways · p. 21
  • Preliminary work uses YOLO and a customized fully connected network for large-scale fading, while ANNs may map physical objects to electromagnetic clusters.Path-loss prediction is described as decent but needing richer geometric information; cluster evolution can characterize channel consistency.Source: D. Key Takeaways · p. 21

VI. SOM-ENHANCE: ENHANCED COMMUNICATION FUNCTIONALITY

  • SoM-enhance infers communication-related side information from sensing to streamline systems and reduce overhead, then uses concrete CSI to guide transceiver design.Waveform and beamforming design are identified as key factors for improving communication performance.Source: §VI · p. 21

A. SoM-Enhance-Based Channel Estimation in Dynamic Scenarios

  • Channel estimation is critical for precoding, demodulation, and equalization, but hybrid precoding constrains design flexibility and larger antenna arrays require proportionally more pilot sequences.The paper notes that these pilot requirements create excessive communication overhead in non-blind estimation.Source: A. SoM-Enhance-Based Channel… · p. 21
  • Dynamic mmWave MIMO-OFDM channels exhibit greater Doppler and delay spread, making parameter-estimation schemes based on fixed channel models unsuitable as networks or channel characteristics change.Strong propagation-environment dynamics are identified as the fundamental cause of the increased difficulty.Source: A. SoM-Enhance-Based Channel… · p. 21
  • Multimodal sensor information can provide propagation-matched side information to improve dynamic-channel estimation while avoiding excessive signaling overhead.The proposed direction is to process comprehensive environmental information collected by multimodal sensors.Source: A. SoM-Enhance-Based Channel… · p. 21

1) Challenges: … 2) Case Study: A Dual-Function Waveform Design in Dynamic Scenarios:

SoM-enhance-based channel estimation is constrained by scarce aligned multi-modal and channel data and by incomplete mappings from sensed features to propagation. The dual-function waveform case study addresses dynamic sensing and communication needs with a superimposed design whose reported range and velocity estimation results improve on OFDM.

1) Challenges:

  • SoM-enhance-based channel estimation lacks aligned MMM datasets, and scalable, interpretable estimation requires mapping sensed features to electromagnetic environments and propagation channels.Such mappings need multi-scenario, weather, and time-of-day sensory data aligned with channel measures such as DoD, DoA, and CSI; the research remains in its infancy.Source: 1) Challenges · p. 22

2) State-of-the-Art:

  • ANN channel estimators demonstrate feasibility across limited-pilot OFDM, spatial-frequency-temporal mmWave mMIMO, fast-varying channels, and real-time VCN prediction.The earlier methods target static scenarios, while ChanEstNet uses CNN and Bi-LSTM for fast time-varying non-stationary channels and uses Bi-LSTM for real-time prediction.Source: 2) State-of-the-Art · p. 22
  • Beamspace mmWave estimation exploits neural models, location-based partitioning –, and unsupervised learning, while multi-modal images, speed, and radar measurements aid estimation, [47].Existing ANN approaches largely use channel data alone; injecting sensed environment features can address performance degradation in complex, dynamic RF environments.Source: 2) State-of-the-Art · pp. 22–23

1) Dual-Function Waveform Design:

  • At the physical layer, dual-function waveform design shares hardware and spectrum for sensing and communication, making RF-ISAC a special case of SoM,,.The shared resources couple sensing and communication at the signal layer.Source: 1) Dual-Function Waveform Design · p. 23
  • Dual-function waveforms include communication-centric, radar-centric, and jointly optimized designs, with limitations from communication-symbol autocorrelation or radar pulse repetition rate, –.Joint designs balance sensing and communication objectives through beam-pattern shaping and constraints on receiver SNR or sensing performance.Source: 1) Dual-Function Waveform Design · p. 23

2) Case Study: A Dual-Function Waveform Design in Dynamic Scenarios:

  • In dynamic mmWave and sub-terahertz scenarios, attenuation and intermittent LoS loss challenge connectivity, while mobility-driven Doppler and target motion strain waveform design and parameter timeliness.Prior work exploited beamspace sparsity to offset Doppler but did not consider sensing functionality.Source: 2) Case Study: A Dual-Function… · p. 24
  • The proposed waveform overlays index-modulation OFDM with radar signal in frequency, using OFDM-equivalent power to improve sensing without sacrificing communication performance.Proper power allocation balances the two functions for different applications.Source: 2) Case Study: A Dual-Function… · p. 24
  • At the same transmission power, range estimation gains up to 5dB over OFDM at high SNR and approaches the CRB; velocity accuracy improves 15% at low SNR.The range result uses 40% of overall power for the communication-dedicated signal; at high SNR, velocity RMSE is also much closer to the CRB than OFDM.Source: Figure 15 · p. 24, 2) Case Study: A Dual-Function… · p. 24

3) Dual-function Waveform Design Considering MultiModal Sensing: · C. SoM-Enhance-Based Beamforming in Dynamic Scenarios · 1) State-of-the-Art:

These sections frame multi-modal sensing as a source of environmental information for dual-function waveform design and predictive beamforming in dynamic scenarios. They review existing approaches and their limits, including the challenge of moving from discrete beam selection to continuous motion prediction.

3) Dual-function Waveform Design Considering MultiModal Sensing:

  • SoM-enhance-based waveform design uses inputs such as vision and location to represent environmental state and optimize communication-sensing performance in dual-function waveforms.The motivation is that communication and sensing share resources, creating a trade-off, while historical data can serve as a source for physical-layer signal design.Source: 3) Dual-function Waveform Design… · pp. 24–25

C. SoM-Enhance-Based Beamforming in Dynamic Scenarios

  • In dynamic mmWave links, narrow-beam mobility makes frequent alignment necessary, while exhaustive beam training incurs communication overhead and latency.Beam tracking narrows the search using temporal correlation, but still uses bits in each data frame to sound channel changes; predictive beamforming predicts the moving target’s position for the next slot.Source: C. SoM-Enhance-Based Beamforming in… · p. 25
  • SoM-enhanced predictive beamforming extracts a target’s 2D position and motion state from multi-modal sensing, with reasonable feature extraction and cross-scenario generalization remaining challenges.Unlike channel estimation, it need not mine implied electromagnetic-environment features; the section identifies ad-hoc networks and V2V communications as scenarios where generalization matters.Source: C. SoM-Enhance-Based Beamforming in… · p. 25

1) State-of-the-Art:

  • EKF-based schemes use DFRC measurements to track motion,, but sensing errors can impair angle prediction, and existing approaches rely on simplistic motion models.Historical-trajectory prediction and vehicle-RSU cooperation address more complex roadway motion, yet periodic measurements in – accumulate error and add signaling overhead and latency.Source: 1) State-of-the-Art · pp. 25–26
  • Single-modal beam prediction uses ISAC signals, cameras, LiDAR, and radar, but each modality has limitations in feature richness, privacy, storage, or sensing precision.Vision methods, cast codebook beam-pair prediction as image classification; LiDAR supports beam selection,,, while radar-based prediction uses processed maps or radar cubes.Source: 1) State-of-the-Art · p. 26
  • Multi-modal beam prediction offers richer environmental representation through federated learning, distributed inference, and semantic sensing, while discrete codebook classification still limits continuous motion tracking.Federated learning can accommodate missing modalities without uploading raw data, and distributed inference compresses vehicle data into high-level features; low-resolution codebooks can misalign beams, while higher resolution increases classification complexity.Source: 1) State-of-the-Art · pp. 26–27
  • In UAV networks, GPS, IMU, and camera inputs support predictive beam alignment, but SoM-enhanced approaches have not yet been studied systematically,,, [20].Vision-aided prediction demonstrates potential beyond position-only approaches, while [20] uses images from UAV-mounted cameras for beam prediction.Source: 1) State-of-the-Art · p. 27
  • SoM-enhanced predictive beamforming can reduce signaling overhead while maintaining high prediction accuracy, and the paper’s proposed scheme predicts continuous parameters rather than codebook beam pairs.The cited review describes machine-learning methods including reinforcement learning and ANNs as widely used for predictive beamforming.Source: 1) State-of-the-Art · p. 27

2) Case Study: A SoM-Enhance-Based Predictive Beamforming Design in VCN: · D. Key Takeaways

The VCN case study shows how SoM-enhance uses multimodal sensing and wireless-channel data for predictive beamforming, while the takeaways identify opportunities in channel estimation, waveform design, and beam management.

2) Case Study: A SoM-Enhance-Based Predictive Beamforming Design in VCN:

  • RGB images, radar ranging, and wireless-channel CSI provide vehicle position and angular features for an ANN to predict motion and support periodic beamforming [187].The RSU obtains sub-6 GHz CSI from necessary control signaling without extra pilots; the simulation uses Gaussian-noise depth maps to mimic frequent radar sensing.Source: 2) Case Study: A SoM-Enhance-Based… · pp. 27–28
  • SoM-enhance [187] tracks vehicle angle more accurately and robustly under drifting than KF/EKF, avoiding EKF’s accumulating error and extra-spectrum observation requirement.The passage attributes the gain to accurate position features from multimodal sensing and angle-evolution features from wireless-channel data.Source: 2) Case Study: A SoM-Enhance-Based… · pp. 27–28, Figure 18 · p. 29
  • With 8 transmit and 8 receive antennas, SoM-enhance [187] achieves rates closest to the ideal case, while KF rates are lowest at most time slots.As the vehicle approaches, EKF prediction error accumulates and its rate declines; the vehicle’s distance from the RSU first decreases and then increases, with rate varying inversely.Source: 2) Case Study: A SoM-Enhance-Based… · pp. 27–28, Figure 18 · p. 29

D. Key Takeaways

  • For dynamic channel estimation, SoM-enhance can use non-RF sensing to infer scattering geometry, focus limited pilots on main-path subregions, and obtain Doppler, delay, AoA, and AoD indirectly.The passage says conventional methods based on a preassumed channel model are no longer applicable when topology, channel characteristics, and the channel model vary over time.Source: D. Key Takeaways · p. 29
  • The proposed superimposed IM-OFDM waveform addresses time variation, while data-driven design and multimodal sensing can improve its dual communication and sensing functions.Multimodal channel priors can guide signal-processing algorithms and parameters, while waveform echoes can complement existing sensors and improve machine recognition through fused sensing results.Source: D. Key Takeaways · p. 29
  • SoM-enhance predictive beamforming can use fine-grained motion and environmental information for high-angular-resolution beam alignment with low signaling overhead.It may also adapt beamwidth, track complex target behavior, and improve weather robustness; existing codebook beam-pair prediction can misalign users in nearby positions and reduce achievable rates.Source: D. Key Takeaways · p. 29

VII. SOM-CONCERT: REINFORCED ENVIRONMENT SENSING … 1) Multi-Modal Early Fusion:

SoM-concert reinforces environment sensing by integrating information from heterogeneous sensors and, across agents, communication-mediated information exchange. Its single-agent early-fusion approaches unify raw or processed sensor data through grid maps, direct projection, or environment-feature fusion.

A. Necessity of SoM-Concert for Environment Sensing

  • Single-agent sensing can fail in dynamic environments because limited line of sight, occlusion, and harsh weather reduce the range, field of view, and reliability of collected information.SoM-concert integrates environmental information to improve sensing accuracy and robustness.Source: A. Necessity of SoM-Concert for… · p. 30
  • SoM-concert spans single-agent fusion of onboard sensors and multi-agent fusion using communications, with the latter extending sensing range and reliability beyond single-agent sensing.Single-agent sensing has lower deployment complexity; both levels seek to improve sensing capability and can mutually reinforce one another.Source: A. Necessity of SoM-Concert for… · p. 30

B. SoM-Concert for Single-Agent Sensing

  • Cameras provide dense semantics but limited depth, LiDAR provides structured but sparse points, and radar adds robustness in harsh conditions and direct velocity measurement.RGB-D cameras supply dense depth information but have limited measurement range; cameras and LiDAR can be degraded by inadequate light and scattering, respectively.Source: B. SoM-Concert for Single-Agent Sensing · p. 30
  • Single-agent fusion is classified as early, late, or hybrid according to whether ANNs fuse raw or processed raw data, sensing results, or both.Early fusion combines data before separate sensing results are generated; late fusion combines results, while hybrid fusion combines data in some modalities and results in others.Source: B. SoM-Concert for Single-Agent Sensing · pp. 30–31

1) Multi-Modal Early Fusion:

  • Early fusion retains raw-data information redundancy and its theoretical performance upper bound, but heterogeneous data formats or network structures complicate processing.The approaches are categorized as grid map generation, raw data projection, and environment feature fusion.Source: 1) Multi-Modal Early Fusion · p. 31
  • Grid maps unify sensor formats in a bird’s-eye view, using features such as observation counts, reflection intensity, occupancy probability, or velocity estimates,.Ranging-sensor data can be projected by coordinate transformation; for cameras, combines detected objects with LiDAR-derived depth. Grid-map sensing uses clustering or learning-based methods, with learning-based methods adapting vision ANNs but requiring network adjustment and retraining for differing grid assignments.Source: 1) Multi-Modal Early Fusion · p. 31
  • Raw data projection uses calibrated coordinate transformations to align heterogeneous 2D and 3D data directly, as when LiDAR points are projected onto camera pixels.The aligned data can provide both 2D and 3D information for joint ANN processing, which requires specially designed feature-fusion modules.Source: 1) Multi-Modal Early Fusion · p. 31
  • Environment-feature fusion embeds sensor data into CNN feature maps, whose refined semantic representations facilitate fusion across sensor categories.In, radar and multiple fish-eye camera features are projected to a bird’s-eye view, then processed by a deep feature-fusion module and detection head.Source: 1) Multi-Modal Early Fusion · pp. 31–32

2) Multi-Modal Late Fusion: · 3) Multi-Modal Hybrid Fusion: · C. SoM-Concert for Multi-Agent Sensing

Late fusion favors flexibility by combining independently processed sensing results, while hybrid fusion trades off information redundancy, flexibility, performance, and efficiency. For multi-agent sensing, communication lets agents share complementary observations, but interaction strategies must balance sensing information against communication resources.

2) Multi-Modal Late Fusion:

  • Late fusion independently processes each modality before combining sensing results, improving flexibility as sensor types change but leaving association and weighted fusion as key challenges.Examples include combining camera and LiDAR detections, refining 3D boxes using camera proposals, and combining segmentation results.Source: 2) Multi-Modal Late Fusion · p. 32
  • Late-fusion weights can use confidence, distance, and IoU or learned re-scoring, while confidence evaluation can reduce a degraded sensor’s influence in harsh conditions.Zheng et al. quantify environmental sensitivity and sensing confidence using data and network-output probability distributions.Source: 2) Multi-Modal Late Fusion · p. 32

3) Multi-Modal Hybrid Fusion:

  • Hybrid fusion combines early fusion’s information redundancy and performance potential with late fusion’s flexibility, using parallel or cascaded modality pipelines.Its pipeline correlation can be adjusted to balance performance and efficiency; early and late fusion respectively face generalizability and information-loss limitations.Source: 3) Multi-Modal Hybrid Fusion · pp. 32–33
  • Parallel hybrid fusion processes modalities through the sensing pipeline together, including region-based feature fusion guided by 3D proposals and combinations of raw data, features, and proposals.Works also fuse non-image modalities to supplement image-based detection of non-line-of-sight objects.Source: 3) Multi-Modal Hybrid Fusion · pp. 32–33
  • Cascaded fusion uses one modality’s results to guide subsequent processing of another, such as projecting image-derived regions into LiDAR for 3D detection.Image-derived boxes or semantic segmentation can focus LiDAR processing, but single-modal processing risks information loss; using multiple proposals or frustums can improve accuracy and robustness –[197].Source: 3) Multi-Modal Hybrid Fusion · p. 33

C. SoM-Concert for Multi-Agent Sensing

  • Single-agent multimodal sensing may fail to provide enough redundancy when sensors degrade together or dynamic-environment occlusions limit sensing range, motivating information sharing among agents.Communication-enabled multi-agent fusion can add complementary perspectives without increasing the sensor cost of each agent.Source: C. SoM-Concert for Multi-Agent Sensing · p. 33
  • Multi-agent systems share raw data, features, or semantic results, trading communication load and flexibility against retained sensing information.Raw-data sharing provides the most redundancy but requires the most bandwidth; semantic sharing minimizes communication load but loses information during local processing; feature sharing balances volume and redundancy.Source: C. SoM-Concert for Multi-Agent Sensing · pp. 33–34

1) Raw-Data-Level Fusion: … D. Key Takeaways

Multi-agent SoM-concert ranges from raw-data and feature fusion, which preserve more sensing information but require greater communication resources, to semantic fusion, which supports heterogeneous agents with compact shared outputs. The takeaways distinguish these trade-offs from single-agent fusion choices shaped by complementary sensors and computation limits.

1) Raw-Data-Level Fusion:

  • Raw-data-level fusion shares raw measurements or unabstracted local maps across agents, expanding sensing coverage and improving detection, especially under occlusion.Examples include camera sharing for augmented reality, single-line LiDAR and low-resolution image sharing, and fused LiDAR or camera-LiDAR grid maps,.Source: 1) Raw-Data-Level Fusion · p. 34
  • Raw-data fusion preserves sensing information and has high performance potential, but MB-scale point clouds or maps strain communication; selective requests can reduce packets to hundreds of KB at added computation cost.These approaches are consequently more suited to offline tasks such as static mapping than real-time sensing in dynamic scenarios.Source: 1) Raw-Data-Level Fusion · pp. 34–35

2) Feature-Level Fusion:

  • Feature-level fusion shares sparse, compressible ANN feature maps to exploit inter-agent redundancy; feature complementation can make multi-agent detection exceed the sum of single-agent results.Methods weight features by information redundancy or amount, and encoding, irrelevant-feature removal, and channel selection can reduce transmission size; frame sizes range from 50KB to 1MB.Source: 2) Feature-Level Fusion · p. 35

3) Semantic-Level Fusion:

  • Semantic-level fusion shares compact object labels, boxes, or segmentation, enabling sensor-agnostic fusion for heterogeneous agents; weighted fusion can reduce sensing uncertainty.Results are mapped to a unified coordinate system, associated by environmental element, and fused, with lower weights for abnormal data to improve robustness in challenging conditions.Source: 3) Semantic-Level Fusion · p. 35

D. Key Takeaways

  • Single-agent SoM-concert seeks complementary sensor information; late fusion is practical for plug-and-play use, while early and hybrid fusion can learn cross-sensor relationships but involve trade-offs in processing and resources.Early fusion suits RSUs with edge servers [217], [218], whereas camera-guided LiDAR ROI in hybrid fusion can suit resource-limited mobile agents –.Source: D. Key Takeaways · pp. 35–36
  • For multi-agent sensing, raw-data fusion offers high redundancy but burdens communication, feature fusion balances volume and performance but depends on homogeneous configurations, and semantic fusion suits heterogeneous agents with limited bandwidth.Raw-data fusion can support mobile agents through cooperative sensing among RSUs with stable networks; semantic fusion may lose effectiveness in harsh weather or obstructed environments.Source: D. Key Takeaways · p. 36

I. SoM-Evoke-Based Network Scheduling … 2) Channel Modeling for the Exploration of Mapping Relationships:

The future SoM research agenda depends on richer MMM datasets and channel models that clarify how multimodal sensing relates to wireless communications. The supplied passages also describe adaptive fusion for environment sensing and identify these foundations as prerequisites for broader applications.

II. SoM-Enhance-Based Channel Estimation

  • The supplied passage for channel estimation provides only the heading fragment “Enrichment of MMM Datasets for Boosting,” without further claims about that section.No substantive channel-estimation findings are present in the supplied excerpt.Source: §II · p. 36

IV. SoM-Enhance-Based Proactive Beamforming

  • The supplied passage for proactive beamforming provides only the phrase “the Exploration of Mapping Relationships,” without further claims about that section.No substantive proactive-beamforming findings are present in the supplied excerpt.Source: §IV · p. 36

V. SoM-Concert-Based Environment Sensing

  • Environment sensing can adapt fusion to CSI between neighboring agents, sending richer redundant data in favorable conditions and using semantic fusion to reduce communication load in adverse conditions.Feature extraction and transmission can also be jointly designed, allocating communication resources to crucial information needed for sensing in dynamic scenarios.Source: §V · p. 36

VIII. FUTURE RESEARCH FOR SOM

  • MMM datasets and mapping relationships are the data foundation and theoretical basis for subsequent SoM research, while broader applications depend on their availability and deeper understanding.Fig. 22 illustrates the relationship between these foundational research areas and future applications.Source: §VIII · p. 36, Figure 22

2) Channel Modeling for the Exploration of Mapping Relationships:

  • SoM channel models should connect sensing to path loss and shadowing, and relate physical objects and their texture, material, and pose to scattering clusters and their angle, power, and delay.The passage identifies these relationships as needed to model large- and small-scale fading and support subsequent SoM research.Source: 2) Channel Modeling for the… · p. 37

B. Exploration of SoM’s Future Applications … IX. CONCLUSION

The survey presents SoM as a unified framework for intelligent multimodal sensing-communication integration and identifies future applications and research gaps across its operational modes.

1) SoM-Evoke-Based Network Scheduling:

  • Multimodal sensing can estimate CSI and channel statistics to predict link states, enabling scheduling over good-state links to improve wireless-network throughput.The proposed SoM-evoke-based scheduling direction depends on more available MMM datasets.Source: 1) SoM-Evoke-Based Network Scheduling · p. 37

2) SoM-Enhance-Based Channel Estimation:

  • SoM-enhanced channel estimation needs interpretable, scalable mapping of multimodal sensing to propagation, plus diverse training data, to address distribution shift and adapt across environments.Existing ANN schemes estimate mmWave channels from geometry but omit multimodal sensing, and can degrade when actual data statistics differ from training data.Source: 2) SoM-Enhance-Based Channel Estimation · p. 37

3) SoM-Enhance-Based New-Radio Design:

  • At THz and higher frequencies, multimodal environmental cues can inform waveform precoding to compensate for beam squint and beam split caused by transceiver hardware limitations.DL-based methods can infer user locations and scatterer reflection properties to help design phase shifts and time delays.Source: 3) SoM-Enhance-Based New-Radio Design · p. 37

4) SoM-Enhance-Based Proactive Beamforming:

  • Multimodal environment information can help predictive beamforming track complex vehicle behavior and trajectories, addressing instability and outages linked to insufficient motion-state information.Relevant behavior includes emergency braking and sudden lane changes, while environmental context includes pedestrians, nearby vehicles, and obstacles.Source: 4) SoM-Enhance-Based Proactive… · pp. 37–38

5) SoM-Concert-Based Environment Sensing:

  • SoM-concert research can balance sensing performance and communication load through resource planning, while communications can provide physical-layer sensing when sensors fail.Channel-parameter estimation and physical-layer sensing could add information in settings such as dark tunnels or harsh weather.Source: 5) SoM-Concert-Based Environment Sensing · p. 38
  • Global environmental awareness can guide holistic multi-resource management, including routing, scheduling, and task offloading, to improve resource utility and sensing performance.The resources are coupled, and their spatiotemporal distribution depends on user locations and surroundings; existing dynamic-system methods are limited to communication resources.Source: 5) SoM-Concert-Based Environment Sensing · p. 38

IX. CONCLUSION

  • SoM unifies intelligent multimodal sensing and communications through ANN-centered, model-enhanced data-driven processing, with three operational modes suited to dynamic scenarios.The survey introduces M3SC datasets and SoM-customized channel modeling as foundations for research, and distinguishes SoM from RF-ISAC’s focus on RF radar sensing.Source: §IX · p. 38
  • The survey reviews SoM-enhanced transceiver design and SoM-concert environment sensing, identifies research gaps, and positions SoM as a catalyst for B5G/6G connectivity and networked intelligence.Its review covers channel estimation, dual-function waveforms, predictive beamforming, and single-agent and multi-agent environment-sensing work.Source: §IX · p. 38
Loading 2306.14143v3…