PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Nonlinear Waveform Fusion Across Scales for Integrated Sensing and Communication

Jiayu Liu, Kai Wan, Shuo Shao, Giuseppe Caire

arXiv:2610.32099v1cs.AI

TL;DR

Fixed linear source interactions constrain how image content interacts with a probing reference in ISAC, despite the need for a shared waveform to support both image recovery and localization. The paper learns multiscale nonlinear fusion and receiver processing from the information available at the UE and BS. Across eleven task weights, the proposed waveforms improve localization error and PSNR over two linear controls on independent test images.

  • Problem

    Fixed source-interaction rules do not adapt to current image features, while the shared waveform must support both image reconstruction and target localization.

    Source: §I · p. 1

  • Method

    The paper learns multiscale nonlinear interactions between image features and an LFM reference in a shared complex state, with receiver processing conditioned on available waveform information.

    Source: §I · pp. 1–2, A. Main Contributions · p. 2

  • Results

    Mean normalized localization error fell by 29.0% versus TD superposition and 27.0% versus learned spectral linear fusion, while mean PSNR rose by 1.40 and 0.82 dB, respectively.

    Source: B. Gains from Nonlinear Waveform Fusion · p. 11, §VI · p. 13

  • Takeaways & Limitations

    On independent test images, the proposed synthesis and sequential recovery approach improved both reported metrics at ten of eleven task weights against each linear control.

    Source: §VI · p. 13

Abstract

from arXiv · show

Fixed source superposition and frequency partitioning constrain how communication content interacts with a probing reference in integrated sensing and communication (ISAC). We learn nonlinear waveform fusion across scales for image transmission and localization of multiple targets in range and angle. Reception uses the information available at each end. Adaptive temporal feature weighting and spectral updates combine image features and a deterministic linearly frequency-modulated reference in a shared complex state, generating a waveform with fixed total energy over the array frame. At the user equipment (UE), the reference and channel state information condition a neural demixer for image reconstruction. At the base station, the realized transmit pulse defines a coherent dictionary for sequential localization with learned local correction, physical refinement, and residual updates. We compare the proposed waveforms with superposition in the time domain and learned spectral linear fusion on 512 independent test images, using a shared fixed UE receiver and a baseline sensing receiver using coherent peak search. Averaged across eleven task weights, the proposed waveforms reduce normalized localization error by 29.0\% and 27.0\%, respectively, while increasing mean peak signal-to-noise ratio by 1.40 and 0.82~dB. With the proposed waveforms fixed, learned local correction reduces mean normalized localization error by 12.2\% relative to the same sequential receiver without learned correction.

I. INTRODUCTION

The introduction motivates learning source-feature-dependent fusion because fixed linear interaction rules may not align with image reconstruction and localization objectives. The paper proposes multiscale nonlinear fusion and receiver designs for image transmission and multiple-target localization under a total array-frame energy constraint.

I. INTRODUCTION

  • Fixed linear fusion keeps source-interaction operators unchanged as image features vary, motivating input-dependent fusion to jointly improve image reconstruction and localization.The paper contrasts this objective with prior waveform designs whose signal-level objectives need not coincide with reconstruction and localization error.Source: §I · p. 1

A. Main Contributions

  • The proposed framework learns multiscale temporal and bandwise spectral interactions between image features and a deterministic LFM reference in a shared complex state.Temporal weights use feature magnitude and relative phase; spectral increments enter the shared state before later temporal processing.Source: §I · pp. 1–2, A. Main Contributions · p. 2
  • The system jointly supports image transmission and localization of multiple targets in range and angle, with total energy constrained over the array frame.The transmitter fuses the communication representation with the same deterministic LFM chirp repeated across pulses, then beamforms and normalizes the frame.Source: A. Main Contributions · p. 2, A. Transmit Signal Model · p. 3
  • At the UE, the LFM reference and CSI condition neural image demixing; at the BS, the realized pulse defines the coherent dictionary for sequential multiple-target localization.The sensing receiver uses local correction, physical refinement, amplitude refitting, and residual updates.Source: A. Main Contributions · p. 2, §I · p. 2

A. Transmit Signal Model

  • The BS constructs communication and sensing beams from CSI and a prescribed sensing covariance, while the UE receives the channel-aligned mode synthesized from image and LFM features.The sensing covariance is formed independently of the realized target scene.Source: A. Transmit Signal Model · p. 3

B. Communication Model

The UE receives the channel-aligned communication mode and uses the known LFM reference and CSI to reconstruct the image.

B. Communication Model

  • The UE channel is flat across frequency and constant over each frame, with independent complex Gaussian additive noise samples.The effective communication gain is defined by the channel and the first beamforming column.Source: B. Communication Model · p. 3
  • The effective channel gain is gc = h_c^T b0, with b0 the first column of Beff; the UE observes a mode aligned with the communication channel, jointly synthesized from the image representation and LFM reference.Source: B. Communication Model · p. 3
  • The UE reconstructs the image from the full received frame using the known LFM reference and CSI.The observed mode is jointly synthesized from the image representation and LFM reference.Source: B. Communication Model · p. 3

C. Sensing Model

The BS estimates target range and angle from each echo using the corresponding realized transmit pulse. Waveform learning jointly minimizes normalized image-reconstruction and localization losses under a total-energy constraint.

C. Sensing Model

  • The target model uses static targets with fixed range, angle, and echo coefficient over the frame, without Doppler progression.The echo model uses circular blocks and periodic extension for fractional delays.Source: C. Sensing Model · p. 3
  • For each sensing decision, the BS estimates K target range-angle pairs from the received echo and the corresponding realized transmit pulse.The transmit waveform in each direction is determined by the array steering response and pulse waveform.Source: C. Sensing Model · p. 3

D. Problem Formulation

  • Waveform learning minimizes expected normalized communication and sensing losses over image, channel, scene, and noise realizations, subject to the total-energy constraint.The sensing loss aggregates range and angle localization errors, while fixed scaling constants keep task losses on shared scales.Source: D. Problem Formulation · p. 4
  • A separate waveform model is trained for each task weight with fixed receive processing, after which the local sensing network is adapted with the transmitter fixed.The image and sensing objectives are evaluated through separate receiver calls for individual pulses.Source: D. Problem Formulation · p. 4

III. PRELIMINARIES

The preliminaries motivate making source interactions depend on current features rather than fixed operators. The framework processes complex features and their magnitude and relative-phase relationships in a shared state.

A. From Linear Superposition to Fusion That Depends on the Inputs

  • With fixed-context linear fusion, a perturbation to the communication input produces a response independent of current source features, motivating feature-dependent interaction weights.The operators may implement learned bandwise or mode mixing but remain independent of the current source features.Source: A. From Linear Superposition to Fusion… · p. 4

B. Processing of Complex Features

  • Complex features are indexed by channel, spatial mode, pulse, and fast-time sample, with normalization applied independently to each frame and spatial mode.The feature channels are divided into four equal contiguous groups.Source: B. Processing of Complex Features · p. 4
  • Dilated complex convolutions aggregate neighboring pulse and fast-time samples at varying scales without adding coefficients or downsampling.The compact kernels couple in-phase and quadrature components, and circular padding preserves the pulse-by-sample dimensions.Source: B. Processing of Complex Features · p. 4

C. Statistics of Magnitude and Relative Phase

  • Magnitude descriptors represent individual feature strengths, while bounded descriptors of the conjugate product capture pairwise relationships and relative phase.The conjugate product is invariant to a common phase rotation of the two features.Source: C. Statistics of Magnitude and… · p. 4

IV. PROPOSED FRAMEWORK

  • The framework combines separate image and LFM branches with a shared complex state, then beamforms and normalizes outputs to the prescribed total array-frame energy.Temporal processing combines sample neighborhoods while spectral updates couple sources within frequency bands; the UE and BS use the information available at their respective receivers.Source: §IV · p. 4

1) Image representation and feature initialization:

Image features and a sensing reference are initialized in a context-conditioned shared representation. A separate spectral branch derives frequency-selective increments from the initial source features for later interaction with that state.

1) Image representation and feature initialization:

  • The image encoder maps each image to 2Cz real feature channels at 8 × 8 resolution, normalized before complex pairing.The resulting representation provides Z for mode expansion and source-feature generation.Source: 1) Image representation and feature… · pp. 4–5
  • CSI, prescribed sensing covariance, and beamforming descriptors form a context vector used to condition the initial joint state.The context summarizes communication and sensing roles across beamforming modes.Source: 1) Image representation and feature… · p. 5

2) Bandwise spectral interaction:

  • Bandwise spectral updates adapt source contributions using current source spectra, with both sources participating in every processing band.Four increments are computed once per frame from initial features and subsequently incorporated through temporal updates.Source: 2) Bandwise spectral interaction · pp. 5–6, Figure 3 · p. 6

3) Adaptive temporal interaction:

Adaptive temporal interaction updates a joint state from image, reference, and prior joint features. Its candidate features and position-varying weights depend on current feature relationships, while dilated convolutions provide multiple fast-time neighborhoods.

3) Adaptive temporal interaction:

  • At each of eight stages, source branches update their features before temporal interaction with the joint state.Joint-branch dilation repeats (1, 2, 4, 8), while the sensing branch uses progressively wider sample spacings and the communication branch uses neighboring samples.Source: 3) Adaptive temporal interaction · p. 6
  • The temporal block combines normalized joint and updated source features, generating complex candidates and real weights from magnitudes and pairwise phase relationships.The weights incorporate source-to-source and source-to-joint relationships, so they depend on current synthesis state as well as source content.Source: 3) Adaptive temporal interaction · pp. 6–7
  • At fixed physical context, changing source features can change the position-varying weights applied to the joint candidate.Context modulates candidate amplitude and phase, while the adaptive weights vary with feature relationships at each position.Source: 3) Adaptive temporal interaction · p. 7

4) Assembly across scales and waveform generation:

The generator assembles source updates, spectral injections, and temporal interactions across eight stages before producing the transmitted array frame. The UE uses the reference and CSI to condition neural demixing for image reconstruction.

4) Assembly across scales and waveform generation:

1) UE: Demixing Using the Reference and CSI:

  • At the UE, the LFM reference and CSI condition a neural demixer whose output is refined and mapped to an image representation.The receiver features are trained through the image reconstruction objective.Source: 1) UE: Demixing Using the Reference… · p. 7

2) BS: Coherent Sensing Using the Realized Waveform:

The BS constructs coherent dictionaries from the realized pulse and localizes targets sequentially. The procedure combines learned local correction with physical refinement and residual updates.

2) BS: Coherent Sensing Using the Realized Waveform:

  • The baseline uses coherent matched filtering and peak search, with hard nonmaximum suppression and parabolic interpolation between bins.Both baseline and proposed receivers construct dictionaries from the realized pulse.Source: 2) BS: Coherent Sensing Using the… · p. 7
  • Candidate detections are excluded when their residual energy is too small, their atom response is nearly invisible, or their coordinates are physically invalid.With zero ridge and a full-column-rank active dictionary, the candidate score is energy orthogonal to the active span.Source: 2) BS: Coherent Sensing Using the… · p. 7
  • Physical refinement uses an approximate projected Jacobian that omits the full derivative of amplitude refitting, and its regularized projection is not orthogonal.Source: 2) BS: Coherent Sensing Using the… · p. 8
  • The proposed sequential receiver selects targets from coherent scores, corrects each grid initializer with a CNN, then refines coordinates and updates the residual.The CNN uses local residual correlations and waveform point-spread-function features; physical refinement accepts a proposed update when it reduces the residual objective.Source: 2) BS: Coherent Sensing Using the… · pp. 7–8

C. Training for Image Reconstruction and Localization

Waveform learning keeps UE and baseline sensing processing fixed, then adapts the sensing receiver with transmitters held fixed. The training losses combine image reconstruction with normalized, pulse-based localization errors.

C. Training for Image Reconstruction and Localization

  • Waveform learning updates the image encoder and fusion operator while keeping the UE receiver and baseline sensing DSP fixed; afterward, transmitters are held fixed during local-correction adaptation.The communication loss combines MSE and SSIM.Source: C. Training for Image Reconstruction… · p. 8
  • Localization loss pairs unordered estimates and targets one-to-one using range and angle errors normalized by fixed reference scales.The assignment cost uses |Δr|/(5 m) + |Δθ|/(10°).Source: C. Training for Image Reconstruction… · p. 8
  • Sensing loss averages receiver losses from uniformly sampled distinct pulse positions to estimate the one-pulse loss across the frame.Each receiver call uses Pobs = 1.Source: C. Training for Image Reconstruction… · p. 8

1) Waveform optimization:

Waveform optimization uses a soft surrogate for discrete sensing selections while preserving gradients through the physical channel and sensing reference. Receiver adaptation updates the local network, and evaluation reports image and localization metrics under specified channel, target, and noise conditions.

1) Waveform optimization:

  • The forward sensing loss uses hard DSP selections, while a sequential decoder supplies a soft-peak straight-through backward surrogate.Masked softmax probabilities form weighted-mean coordinates; detached hard maxima determine exclusion regions, and discrete selections and assignments are detached.Source: 1) Waveform optimization · p. 8

2) Adaptation of the Sensing Receiver:

  • For fixed transmit waveforms, receiver adaptation minimizes normalized localization loss on cached labeled pulses by updating only the local network parameters.Candidate choices and assignments are detached, while gradients pass through continuous correction and physical refinement.Source: 2) Adaptation of the Sensing Receiver · p. 8

V. EXPERIMENTAL RESULTS

  • The experiments first compare waveform constructions and noise sensitivity, then assess sequential sensing recovery and learned local correction with waveforms fixed.The system uses three static targets sampled independently, with no minimum separation.Source: §V · p. 9, 1) System, data, and metrics · p. 9

1) System, data, and metrics:

  • Communication noise follows received-frame power, whereas sensing noise at a given reference SNR is shared across waveforms because reference power is fixed.Reported measures include PSNR and SSIM for reconstruction and range and angle MAE for localization; normalized errors use training-calibrated scales.Source: 1) System, data, and metrics · p. 9

2) Comparison design:

The comparison evaluates nonlinear and linear waveform constructions under defined UE and sensing-receiver configurations. Training starts from a shared reference initialization, and evaluation uses held-out images with paired scenes, channels, and noise.

3) Training and evaluation:

  • The common encoder and UE initialization comes from a four-mode reference transceiver trained for 60 epochs with LFM conditioning and image, ambiguity, and auxiliary sensing losses.Waveform training uses a 10,000-image COCO stream; training schedules and spectral scales are specified separately.Source: 3) Training and evaluation · pp. 9–10

2) Comparison design:

  • The controls include time-domain superposition, learned bandwise spectral fusion, and FDM; main waveform comparisons use baseline sensing DSP with each configuration’s complete realized waveform.FDM splits frequency support and frame energy between sources, while the spectral control applies learned source transforms.Source: 2) Comparison design · p. 9, Table II · p. 10

3) Training and evaluation:

  • Figure 5 plots PSNR against range MAE, angle MAE, and normalized localization error across eleven task weights; Figure 6 plots PSNR and SSIM against receive SNR at w = 0.5.Figure 5 uses baseline DSP; Figure 6 uses the common UE receiver and shows conditional 95% bootstrap intervals.Source: Figure 5 · p. 10, Figure 6 · p. 10
  • Figure 7 shows original images and proposed reconstructions at 0 and 20 dB, while Figure 8 plots localization errors against reference sensing SNR at w = 0.5 and K = 3.Figure 8 distinguishes baseline coherent peak search from sequential recovery with local correction.Source: Figure 7 · p. 10, Figure 8 · p. 11
  • Evaluation uses 512 development and 512 test images across eleven task weights, with scenes, channels, and noise paired across methods.Test images were unused in training or method selection; bootstrap intervals resample images as clusters, conditional on trained models.Source: 3) Training and evaluation · pp. 10–11

B. Gains from Nonlinear Waveform Fusion

The proposed waveform improves image and localization metrics over linear controls across most matched task weights. Sequential recovery with local correction further reduces localization error, while the realized pulse exhibits altered correlation structure.

B. Gains from Nonlinear Waveform Fusion

  • On the development set, the proposed construction achieves higher PSNR and lower normalized localization error than each linear control at most matched weights; at w = 1, localization error rises sharply when its loss term vanishes.This underscores the role of sensing supervision in waveform learning.Source: B. Gains from Nonlinear Waveform Fusion · p. 11

C. Noise Sensitivity of the Learned Waveforms

  • Across tested receive SNRs, the proposed waveform maintains higher mean PSNR/SSIM than both linear controls; with baseline DSP, its localization errors stay lower, while neural assistance reduces the high-SNR plateau.The reconstruction sweep holds the w = 0.5 models fixed.Source: C. Noise Sensitivity of the Learned… · pp. 11–12

D. Contribution of Sequential Recovery and Local Learning

  • On independent tests, mean DRA is 1.0899 for baseline DSP, 0.1185 for sequential recovery without correction, and 0.1041 with correction; correction further reduces error by 12.2%.Amplitude refitting, physical refinement, and residual updates account for most of the reduction. On development data, correction lowers all three errors at every tested weight, with shared initialization and physical refinement.Source: Figure 9 · p. 11, D. Contribution of Sequential Recovery… · p. 12

E. Waveform Structure and Correlation Diagnostics

  • The realized pulse retains a distinct zero-delay autocorrelation peak but has higher sidelobes over much of the delay range and a different ambiguity pattern; its structure enters an image-content-dependent coherent dictionary.The waveform objective does not explicitly penalize similarity to the reference or sidelobes.Source: E. Waveform Structure and Correlation… · pp. 12–13

VI. CONCLUSION

  • At ten of eleven weights, both metrics improve; mean localization error falls 29.0% versus TD and 27.0% versus learned spectral linear fusion, with PSNR gains of 1.40 and 0.82 dB.These comparisons use independent test images with a shared fixed UE receiver and baseline DSP receiver.Source: §VI · p. 13

Related papers

Loading 2610.32099v1…