SemISAC: Semantic Integrated Sensing and Communications

I. INTRODUCTION

• Conventional ISAC shares spectrum, hardware, signal processing, and waveforms, but physical-layer integration does not capture how information supports downstream inference, sensing, and decisions.Prior work does not provide a general framework jointly modeling and optimizing SemCom and SemS while distinguishing their information-generation mechanisms.Source: §I · pp. 1–2
• SemISAC combines source semantics and sensing task information with side information to design a shared waveform and allocate radio resources for source-meaning transmission and environmental-information acquisition.Its information-theoretic interpretation distinguishes task-relevant from physical information and characterizes their semantic communication–sensing trade-off.Source: §I · p. 2
• The general task-oriented design jointly considers communication and sensing semantic objectives under shared physical-resource constraints, with end-to-end and modular optimization as two realization methods.End-to-end optimization uses final task losses throughout the processing chain; modular optimization targets selected modules using task losses or interfaces approximating task performance.Source: §I · p. 2
• The learning-based OFDM design lowers sensing semantic distortion under a given communication requirement and achieves a more favorable communication–sensing trade-off than baseline designs.Its task-aware semantic proxies track downstream task errors more closely than physical proxies.Source: §I · p. 2

A. SemCom Model for Source Information Transmission

• The SemCom model encodes source information required by a receiver task into a representation, transmits it through a physical channel, and decodes the received signal for that task.The task specification and knowledge base determine relevant source information; in a single-task implementation, the task can be implicit in the model architecture and loss.Source: A. SemCom Model for Source Information… · p. 3

B. SemS Model for Environmental Information Acquisition

• SemS uses task specifications, knowledge bases, and available priors to construct a sensing requirement that guides waveform design toward relevant environmental information before the current state is observed.The requirement can encode relevant parameter ranges, relationships, and relative importance.Source: B. SemS Model for Environmental… · p. 3

A. SemISAC Information Flow and Functional Architecture

• SemCom and SemS retain distinct information-generation mechanisms but share task-dependent representations, physical waveforms, observations, and receiver-side task outputs.The communication decoder recovers source meaning, while sensing processing and decoding infer task-relevant environmental information.Source: B. SemS Model for Environmental… · p. 3, A. SemISAC Information Flow and… · p. 4

III. GENERAL SEMISAC ARCHITECTURE

• SemISAC retains shared physical transmission while using communication and sensing task requirements to configure the shared waveform and radio resources.The framework builds on the branch-specific SemCom and SemS models.Source: A. SemISAC Information Flow and… · p. 3, §III · p. 3

A. SemISAC Information Flow and Functional Architecture

• The joint semantic-to-physical design uses source semantics, sensing requirements, and available channel, resource, and system information to select a shared transmit configuration.Transmit processing applies that configuration to generate the shared signal, which produces communication and sensing observations.Source: A. SemISAC Information Flow and… · p. 4

• For each branch, semantic information equals physical information minus a nonnegative residual, representing information beyond what is relevant to the task.Source: B. Information-Theoretic… · pp. 4–5
• Under the stated observation conditions, achieved communication and sensing task distortions are bounded by the task-relevant information available and retained in their respective branches.Source: B. Information-Theoretic… · p. 5
• Physical and semantic Pareto boundaries characterize physical-information and task-relevant-information trade-offs, respectively, over the same feasible policies.Source: B. Information-Theoretic… · p. 5
• When both residuals vanish for every feasible policy, the semantic and physical information regions and their Pareto boundaries coincide.Source: B. Information-Theoretic… · p. 5
• Outside task sufficiency, residuals can reorder policies in the semantic region, so physical Pareto optimality does not generally imply semantic Pareto optimality.Source: B. Information-Theoretic… · p. 5

• The section formulates the general SemISAC design problem.Source: §IV · p. 6
• It presents E2E and modular SemISAC optimization as two solution approaches under the same semantic objectives.Source: §IV · p. 6
• It describes offline design procedures and a common online execution architecture for the approaches.Source: §IV · p. 6

1) Semantic Task Objectives:

• Branch-specific semantic task losses evaluate task effectiveness, unlike physical metrics such as rate, SINR, estimation error, and CRB.Source: 1) Semantic Task Objectives · p. 6

2) Constrained and Weighted Design Formulations:

• The general formulation jointly optimizes communication and sensing semantic objectives under shared physical resources.Source: 2) Constrained and Weighted Design… · p. 6
• When sensing is primary, the design can impose a required communication semantic-loss level alongside waveform-feasibility and physical-design constraints.Source: 2) Constrained and Weighted Design… · p. 6
• When neither task has a prescribed service constraint, a weighted sum of communication and sensing losses uses β to set task priority.Source: 2) Constrained and Weighted Design… · p. 6
• The two proposed realization methods are E2E SemISAC optimization and modular SemISAC optimization.Source: 2) Constrained and Weighted Design… · p. 6

B. E2E SemISAC Optimization

• E2E optimization evaluates final task losses through the complete processing chain and uses gradients when differentiable, or non-gradient methods otherwise.Source: B. E2E SemISAC Optimization · p. 6
• E2E training updates trainable groups across the processing chain while prescribed parameters remain fixed; representative training conditions and access to the complete chain are generally required.Source: B. E2E SemISAC Optimization · p. 6

C. Modular SemISAC Optimization

• Modular SemISAC optimization updates a selected parameter subset while holding the remaining parameters fixed, preserving the general semantic objectives.Fixed modules may be independently developed, pretrained, or nondifferentiable; their effect on final task losses is represented through inter-module interfaces.Source: C. Modular SemISAC Optimization · pp. 6–7

D. Offline Design and Online Execution

• Offline E2E and modular designs determine deployable parameters from representative source, environmental, channel, and task data, including complete environmental states and sensing targets.Online, the two methods use the same architecture with their respective optimized parameter sets; joint S2P design selects the transmit configuration before shared-signal generation and task decoding.Source: D. Offline Design and Online Execution · p. 7

V. MODULAR SEMISAC REALIZATION VIA LEARNING-BASED OFDM TF PRECODING

• The OFDM realization applies modular SemISAC optimization to task-loss interfaces for TF-precoder design and develops a learning-based S2P policy for low-complexity online execution.The design specifies representative tasks and a shared OFDM signal model before optimizing the precoder.Source: §V · p. 7

A. Communication and Sensing Task Instantiation

• The communication task is four-category AG News topic inference, using a normalized, whitened MiniLM sentence embedding as the transmitted latent representation.A frozen four-class classifier supplies the reference posterior, and the communication receive path produces a topic posterior for the KL sample loss.Source: A. Communication and Sensing Task… · p. 7
• The sensing task decides whether an approaching target reaches a boundary within a safety horizon, with delay–Doppler errors weighted by their relevance to the arrival-risk decision.The task output compares estimated arrival time with Tsafe; the resulting semantic loss is the probability of an incorrect binary decision.Source: A. Communication and Sensing Task… · pp. 7–8

B. Shared OFDM Signal

• Each monostatic OFDM frame has N subcarriers and M symbols, yielding L = NM time-frequency resource elements for probing and communication.The model assumes a sufficient cyclic prefix and negligible intercarrier interference.Source: B. Shared OFDM Signal · p. 8
• Transmit processing maps the normalized communication representation through the TF precoder into a vectorized resource grid, with OFDM modulation and receiver transforms represented by equivalent TF-domain models.The resource-grid operation combines continuous-valued feature-to-symbol mapping, precoding, and resource mapping.Source: B. Shared OFDM Signal · p. 8
• The shared grid yields a communication observation through a diagonal OFDM channel, while the sensing branch reshapes it for a single-target monostatic sensing-channel model.The sensing observation is used by sensing signal processing and the semantic decoder.Source: B. Shared OFDM Signal · p. 8

C. Modular TF-Precoder Optimization

• Only the TF-precoding policy parameters are optimized in this realization; the MiniLM model, classifier, and non-neural processing blocks remain fixed.The learned policy is trained separately for each specified OFDM dimension, power budget, and nonwaveform resource configuration.Source: C. Modular TF-Precoder Optimization · p. 8
• Communication and sensing proxies weight latent-recovery error and delay–Doppler uncertainty by task relevance derived from classifier sensitivity and arrival-risk sensitivity.The physical-quality variables are the LMMSE latent-recovery error covariance and the CRB matrix for normalized delay–Doppler state.Source: C. Modular TF-Precoder Optimization · p. 8
• The S2P policy maps channel and task information with priority β to a TF precoder, designed for a communication requirement expressed in the calibrated proxy domain.Its input includes communication and sensing task information, and its output is the precoder used in shared-signal transmission.Source: C. Modular TF-Precoder Optimization · pp. 8–9

D. Learning-Based S2P Policy

• The S2P policy uses a neural generator and power normalization to produce a precoder from estimated channel features, communication relevance, sensing requirements, and priority β.This avoids repeated LMMSE and Fisher information matrix calculations at each transmission interval.Source: D. Learning-Based S2P Policy · p. 9
• Training samples β across [0, 1] and updates policy parameters by backpropagating the two task-aware semantic proxies through their calculations and the precoder mapping.The simulations train learned methods independently for each power condition for 200 epochs.Source: D. Learning-Based S2P Policy · p. 9, A. Simulation Settings · p. 9
• Online, one policy forward pass maps channel and task information with β to the precoder; numerical optimization, unfolded optimization, and reinforcement learning are also possible design implementations.The selected precoder is applied through transmit processing and OFDM operations.Source: D. Learning-Based S2P Policy · p. 9

VI. SIMULATION RESULTS / A. Simulation Settings

• The benchmarks span uniform precoding, conventional physical ISAC, communication-semantic-aware ISAC, and sensing-semantic-aware ISAC.The communication-semantic-aware benchmark retains a physical sensing objective, while the sensing-semantic-aware benchmark retains a physical communication objective.Source: A. Simulation Settings · p. 9

B. Validation and Performance of Semantic ISAC

• The semantic proxy is more strongly associated with communication task error than the physical proxy, while the sensing semantic proxy preserves arrival-risk decision-error ordering more closely.Physical proxies show weaker associations and greater separation across methods for both tasks.Source: B. Validation and Performance of… · p. 10, Figure 3 · p. 10
• Across most of the power range, Conventional Physical ISAC has lower communication physical distortion, while proposed SemISAC has the lowest sensing semantic distortion.For sensing physical distortion, Conventional Physical ISAC is lower at low power, but proposed SemISAC becomes comparable and then lower toward the high-power endpoint.Source: B. Validation and Performance of… · p. 10, Figure 4 · p. 10
• At Pmax = −10 dB, proposed SemISAC reduces sensing semantic distortion by 1.72 dB versus Semantic Communication Aware ISAC while satisfying the communication proxy requirement.Proposed SemISAC remains below that baseline across the considered power range under the validation-derived requirement ϵc = 0.3798.Source: B. Validation and Performance of… · pp. 10–11, Figure 5 · p. 10
• Proposed SemISAC traces the lowest communication–sensing distortion curve as communication priority varies, and at balanced priority improves both distortions slightly over Semantic Communication Aware ISAC.Increasing communication priority generally lowers communication distortion and raises sensing distortion for each learned design.Source: Figure 6 · p. 11, B. Validation and Performance of… · p. 11
• As communication-prior mismatch grows, proposed SemISAC and Semantic Communication Aware ISAC move from below to above Conventional Physical ISAC in normalized semantic distortion.Under moderate sensing-query mismatch, proposed SemISAC remains close to Perfect Prior SemISAC; full replacement of the arrival-risk query with a delay-estimation query raises normalized distortion to 2.22.Source: B. Validation and Performance of… · p. 11, Figure 7 · p. 11
• Conventional Physical ISAC allocates energy almost uniformly across feature ranks, whereas proposed SemISAC and Semantic Communication Aware ISAC allocate progressively less energy to less classification-relevant features.At balanced task priority, the two semantic communication designs show close agreement in this feature-protection pattern.Source: B. Validation and Performance of… · p. 11, Figure 8 · p. 11

VII. CONCLUSION

• The design reduces sensing semantic distortion under a given communication requirement and improves the communication–sensing trade-off.Simulations also show that the semantic proxies track downstream task errors more closely than physical proxies.Source: §VII · p. 12

OFDM REALIZATION

• The appendix derives the task-loss interfaces used in (34) and (35).Source: OFDM REALIZATION · p. 12

A. Communication Task-Loss Interface from Classifier Sensitivity

• The communication interface characterizes classification-output sensitivity to recovery error in each transmitted latent component.Source: A. Communication Task-Loss Interface… · p. 12
• The classifier maps real latent representations to logits and softmax outputs, whose local KL divergence under recovery perturbation is approximated using classifier sensitivity.Source: A. Communication Task-Loss Interface… · p. 12
• Component relevance combines curvature terms for each latent’s real and imaginary parts to provide the task relevance used by the communication interface.Source: A. Communication Task-Loss Interface… · p. 12
• The task-aware communication proxy combines component relevance with variance-normalized errors, alongside a task-independent physical proxy.Source: A. Communication Task-Loss Interface… · p. 12

B. Sensing Task-Loss Interface from Arrival-Risk Sensitivity

• The sensing requirement evaluates local task sensitivity at the transmitter-available prior because the instantaneous state is unavailable before probing.Source: B. Sensing Task-Loss Interface from… · p. 12
• Final task evaluation estimates delay and Doppler over a bounded search grid, then maps the estimates through the arrival-time decision to the arrival-time output.Source: B. Sensing Task-Loss Interface from… · p. 12
• The sensing proxy uses an equivalent Fisher information matrix and corresponding Cramér–Rao bound for the normalized delay–Doppler state, accounting for the nuisance reflection coefficient.Source: B. Sensing Task-Loss Interface from… · pp. 12–13
• A local approximation of arrival-risk perturbation uses the state covariance proxy to form a waveform-dependent semantic sensing proxy.Source: B. Sensing Task-Loss Interface from… · p. 13