PaperlayerView exact revision on arXiv

Source-linked AI summary

Personalized Saliency in Task-Oriented Semantic Communications: Image Transmission and Performance Analysis

Jiawen Kang, Hongyang Du, Zonghang Li, Zehui Xiong, Shiyao Ma, Dusit Niyato, Yuan Li

arXiv:2209.12274v1eess.IV

TL;DR

The paper addresses inefficient UAV image retrieval, missing personalization in semantic encoding, and limited mathematical understanding of fading effects in task-oriented semantic communication. It proposes a triple-based scene-graph framework, personalized attention-weighted encoding, fading analysis, and game-based multi-user resource allocation. Numerical evaluations report improved personalization, interference performance, and communication quality, including a 64% communication-cost reduction.

Abstract

from arXiv · show

Semantic communication, as a promising technology, has emerged to break through the Shannon limit, which is envisioned as the key enabler and fundamental paradigm for future 6G networks and applications, e.g., smart healthcare. In this paper, we focus on UAV image-sensing-driven task-oriented semantic communications scenarios. The majority of existing work has focused on designing advanced algorithms for high-performance semantic communication. However, the challenges, such as energy-hungry and efficiency-limited image retrieval manner, and semantic encoding without considering user personality, have not been explored yet. These challenges have hindered the widespread adoption of semantic communication. To address the above challenges, at the semantic level, we first design an energy-efficient task-oriented semantic communication framework with a triple-based {\color{black}scene graph} for image information. We then design a new personalized semantic encoder based on user interests to meet the requirements of personalized saliency. Moreover, at the communication level, we study the effects of dynamic wireless fading channels on semantic transmission mathematically and thus design an optimal multi-user resource allocation scheme by using game theory. Numerical results based on real-world datasets clearly indicate that the proposed framework and schemes significantly enhance the personalization and anti-interference performance of semantic communication, and are also efficient to improve the communication quality of semantic communication services.

I. INTRODUCTION

The paper targets UAV image-sensing-driven task-oriented semantic communication by addressing inefficient image retrieval, non-personalized semantic encoding, and limited analysis of wireless fading. It proposes a scene-graph framework, personalized encoding, mathematical channel analysis, and game-based multi-user resource allocation.

I. INTRODUCTION

  • Traditional UAV communication transmits all captured images, wasting communication resources and UAV energy on images users do not need.Keyword-based retrieval can still suffer packet drops and retransmissions under limited energy or poor wireless channels.Source: A. Background and Motivations · p. 2
  • Existing semantic encoding methods do not account for individual user preferences, risking loss of information important to particular users during fading transmission.The paper motivates personalized coding weights and user-specific resource allocation.Source: A. Background and Motivations · p. 2
  • The proposed framework models image information as a triple-based scene graph to retrieve selected images matching users’ query text.This replaces transmission of all images with semantic matching of user queries to image information.Source: B. Solutions and Contributions · p. 2
  • A personalized semantic encoder assigns higher weights to triplets more relevant to each user, supporting differential transmission of important information.The weighting mechanism is based on users’ subjective interests.Source: B. Solutions and Contributions · p. 2
  • The paper mathematically analyzes semantic triplet transmission under generalized fading channels and derives a game-theory-based multi-user resource allocation scheme.The scheme uses a retrieval-task utility function to improve utilization of limited UAV resources.Source: B. Solutions and Contributions · p. 2

II. RELATED WORK

Prior work covers semantic encoding, task-oriented communication, privacy, and fading-channel performance, but task-oriented systems generally ignore differences among users. The proposed system combines personalized saliency, semantic triplet retrieval, and power allocation for UAV image services.

II. RELATED WORK

  • Semantic communication transmits extracted semantic information rather than all original data, reducing unnecessary communication overhead.The receiver reconstructs the original information from the transmitted semantic representation.Source: A. Semantic Communication · p. 3
  • Prior task-oriented communication systems adapt transmitted semantics to goals but ignore individual differences in users’ needs.This motivates incorporating personalized saliency into the proposed framework.Source: A. Semantic Communication · pp. 3–4
  • Wireless semantic communication performance is negatively affected by multipath fading, while prior studies examined selected channel models such as Rayleigh and Rician fading.The paper positions its mathematical fading analysis against this prior performance-analysis literature.Source: A. Semantic Communication · p. 4
  • The proposed system predicts user saliency, extracts image triplets, fuses objective and subjective attention, and uses the result for personalized semantic communication.This pipeline connects customized user information with UAV-captured imagery.Source: B. Personalized Saliency · p. 4
  • For each user, the UAV extracts image triplets and transmits them using TDMA, while power is allocated across users and individual triplets under an energy constraint.The two optimization problems separately address inter-user allocation and within-user triplet allocation.Source: B. Optimization Problems Analysis · p. 5

C. SINR Analysis

The section models a multi-user UAV downlink in which beamforming and transmit powers determine each user's SINR under interference and noise. Users are positioned relative to a fixed-altitude UAV, with channel quality affected by distance and path loss.

System Model

  • The UAV serves K users from NT antennas while transmitting semantic features from aerial images.Each user has an individual preference, and the UAV uses its antenna array to obtain array gains and improve channel quality.Source: C. SINR Analysis · p. 5

Geometry

  • User positions are represented by uk, the UAV position by uu, and their separation determines the UAV-to-user link distance.The kth user's horizontal coordinate is uk=(xk,yk,0), while the UAV hovers at fixed altitude zu.Source: C. SINR Analysis · p. 6

Signal Model

  • Linear beamforming multiplies user k's data symbol by beamformer wk, while interfering paths contribute interference power at the receiver.The model includes NIk interfering paths, average interfering transmit power, and interference symbols for each user.Source: C. SINR Analysis · p. 6

SINR

  • The kth user's SINR depends on desired transmit power, channel gain, interference, and noise.The section introduces Pk as the kth user's transmit power and σ2 as the noise term in the SINR expression.Source: C. SINR Analysis · p. 6

Beamforming

  • Maximum ratio transmission provides the optimal beamforming vector for the modeled user channel.The beamformer is expressed using the channel vector hk.Source: C. SINR Analysis · p. 6

D. Channel Model

The channel model uses Fisher-Snedecor F fading for UAV links and Rayleigh fading for interference, then derives distributions for channel gains and SINR. Because exact multi-antenna expressions are analytically difficult, the summed desired-channel gain is approximated by a single F distribution.

Desired Link

  • The UAV-to-user link follows Fisher-Snedecor F composite fading, combining Nakagami-m small-scale variation with inverse-Nakagami-m shadowing.Measurements at 5.8 GHz are cited as supporting this model in both line-of-sight and non-line-of-sight scenarios.Source: D. Channel Model · p. 6
  • The desired beamformed gain is a sum of NT Fisher-Snedecor F random variables, which is approximated by a single Fisher-Snedecor F distribution.The approximation avoids the less interpretable multivariate Fox’s H-function representation.Source: D. Channel Model · p. 6

Distributional Model

  • The approximated gain Z is modeled as F(mfk, msk, z̄k), whose PDF and CDF are used in subsequent analysis.The notation identifies the multipath fading parameter, shadowing parameter, and average gain.Source: D. Channel Model · p. 6

Interference

  • Interference signals are modeled with Rayleigh fading, and the large-scale interference fading is incorporated into the mean channel value.The sum of NIk independent Rayleigh-fading signals has a Nakagami-m amplitude with m=NIk.Source: D. Channel Model · p. 6

SINR Distribution

  • Theorem 1 provides closed-form PDF and CDF expressions for the SINR using the desired-channel and interference distributions.The derivation uses the modeled channel gain, interference statistics, and transmit-power terms.Source: D. Channel Model · p. 6

E. Approximation Analysis

The section develops an accurate approximation for a difficult integral appearing in the SINR CDF, then derives high-transmit-power asymptotics and verifies them through outage probability comparisons.

Motivation

  • Closed-form PDF and CDF expressions are difficult to interpret directly, motivating an approximate CDF and high-SNDR analysis.The paper states that numerical analysis is used to verify the resulting expressions.Source: E. Approximation Analysis · p. 6

Accurate Approximation

  • The integral IA lacks an established closed solution, so the paper derives an accurate approximation for it.The authors characterize IA as absent from the cited mathematical integral references and difficult to solve exactly.Source: 1) Accurate Approximation · p. 6
  • Lemma 1 gives an accurate approximation of IA when ρ is small.The approximation is then substituted into the SINR CDF derivation.Source: 1) Accurate Approximation · p. 6

Asymptotic Analysis

  • Theorem 3 approximates the SINR CDF in the high-transmit-power regime.This asymptotic expression is developed after the approximate CDF is obtained.Source: 2) High-SNDR approximation · p. 7

Verification and Insights

  • The accurate approximation nearly matches the closed-form expression, while the high-SINR approximation is close when Pk exceeds 25 dBW.The outage probability comparison also shows that larger mfk produces a faster decrease in outage probability as transmit power increases.Source: 3) Verification and Insights · p. 7

A. Bit Error Probability

The section formulates bit error probability for semantic triplet transmission under modulation-specific conditional error behavior. It connects triplet encoding and transmission conditions to the probability that semantic information is dropped.

Bit Error Probability

  • Under different modulation formats, the kth user's bit error probability is expressed using modulation-specific parameters λ1 and λ2.The conditional bit error probability is represented through the incomplete-gamma expression in the model.Source: A. Bit Error Probability · p. 7
  • Theorem 4 derives the bit error probability Ek for user k.The theorem provides the analytical expression used for subsequent triplet-drop calculations.Source: A. Bit Error Probability · p. 7

Triplet Drop Probability

  • The triplet drop probability Pk is calculated from the kth user's bit error probability Ek and the derived expression.The drop probability is evaluated with the help of the preceding bit-error result.Source: B. Triplet Drop Probability · p. 7

Application Context

  • In UAV aerial photography, transmitting every aerial image to every user is impractical in fading wireless environments.The application instead motivates selective delivery based on users' image needs.Source: §V · p. 7

A. OA-SemCom: A Fully Objective Approach

PERSF-SEMCOM detects image triplets, combines objective visual attention with user-specific saliency, prioritizes triplets, and allocates transmission power according to each user’s preferences.

Triplet Detection

  • Triplet Detection extracts subject-relation-object triplets, attention heatmaps, and subject/object bounding boxes from each captured image.These outputs provide the semantic representation and spatial information used by later personalization and priority modules.Source: A. OA-SemCom: A Fully Objective Approach · p. 8

Personalized Saliency Fusion

  • PERSF-SEMCOM fuses RelTR’s objective attention with a personalized saliency heatmap to estimate user-specific triplet priorities.The personalized saliency predictor uses user identity and the captured image; attention maps are normalized and combined by weighted sum.Source: B. PERSF-SEMCOM: Achieve… · pp. 8–9

Triplet Priority Estimation

Power Allocation and Retrieval

  • The power-allocation module assigns more transmission power to higher-priority triplets, reducing their likelihood of being dropped over the wireless channel.The main algorithm uses RCGA for allocation, while the receiver matches received triplets against the user’s query through accurate or fuzzy search.Source: B. PERSF-SEMCOM: Achieve… · pp. 9–10
  • At the receiver, Triplet Search computes a query-match score and downloads the current image when the score exceeds a user-specified threshold.TS-AM supports exact triplet-format queries, whereas TS-FM supports free-text queries.Source: B. PERSF-SEMCOM: Achieve… · pp. 9–10

VI. NUMERICAL RESULTS

The numerical evaluation uses pretrained scene-graph and saliency models, real-world visual data, channel settings from prior work, and Naive-SemCom and OA-SemCom as benchmarks.

Experimental Platform

  • The platform uses Ubuntu 18.04, an Intel Xeon E5-2678 CPU, and four GeForce RTX 2080 Ti GPUs.RelTR is pretrained on Visual Genome, which contains 108k images, 150 object categories, and 50 relationship categories.Source: A. Environment Setup · p. 10

Models and Data

  • RelTR serves as the Triplet Detection model, while a pretrained saliency-prediction model supports the personalized saliency module.RelTR’s reported top-50 scene-graph recall is 25.2, with a corresponding mean value of 8.52.Source: A. Environment Setup · p. 10
  • Unless otherwise specified, small-scale fading and system parameters such as transmission power and distance follow common settings from prior literature.The study also references the STREET dataset and channel-configuration parameters for three users.Source: A. Environment Setup · p. 10, Figure 4 · p. 10, Table II · p. 10

Baselines

  • Naive-SemCom allocates equal power without semantic-triplet importance, whereas OA-SemCom quantifies triplet priorities for scheduling.These two approaches provide the performance-comparison baselines.Source: A. Environment Setup · p. 10

B. Results and Analysis

Experiments show that personalized saliency, priority-based allocation, and channel-aware analysis improve utility, reduce optimality gaps and communication overhead, and reveal sensitivity to fading, interference, and distance.

Effectiveness of PERSF-SEMCOM

  • The optimal utility typically occurs at α between 0.1 and 0.2, where subjective attention contributes 80%-90% of fused attention.PERSF-SEMCOM outperforms Naive-SemCom in most tested cases, while α = 1.0 degenerates to OA-SemCom and performs worse than Naive-SemCom.Source: B. Results and Analysis · pp. 10–11

Transmit Power

  • At P = 3kW, PERSF-SEMCOM reduces the optimality gap by 54%, 57%, and 37% for the three users, respectively.OA-SemCom instead widens the gaps by 26%, 57%, and 86%.Source: B. Results and Analysis · p. 11

Channel Conditions

  • When multipath and shadow fading weaken, utility increases; utility responds faster to mfk than to an equivalent increase in msk.The analysis attributes the stronger sensitivity to BER increases caused by multipath effects rather than shadowing.Source: B. Results and Analysis · p. 12
  • When interference exceeds 10 W, every additional 5 m of transmission distance causes an approximately 58% utility decrease.The paper recommends reducing transmission distance by adjusting the UAV trajectory under substantial interference.Source: B. Results and Analysis · p. 12

Communication Overhead

  • PERSF-SEMCOM reduces transferred image data from 224.8MB to 81.29MB, a 64% communication-cost reduction, after sending 873 triplets with 10.24KB overhead.The selective process sends 64 images to specified subscribers instead of transmitting all captured images.Source: B. Results and Analysis · p. 12

Overall Findings

  • The conclusion reports that the framework and schemes achieve personalized semantic communication and significantly enhance UAV resource utilization on real-world datasets.The approach combines a triple-based scene graph, personalized attention-based encoding, fading analysis, and game-based multi-user allocation.Source: §VII · pp. 12–13

APPENDIX A PROOF OF LEMMA 1

The appendix derives the closed-form expression in Lemma 1 by substituting intermediate expressions, evaluating an integral, and applying the bivariate Meijer’s G-function definition.

Derivation

  • The proof substitutes equation (7) and a cited equation into (A-1), then rewrites the exponential function using a Mellin-Barnes integral.These substitutions transform the expression into a form suitable for evaluating the integral term I1.Source: A. Proof of PDF · p. 13

Conclusion

  • Substituting I1 into (A-2) and applying the bivariate Meijer’s G-function definition yields equation (9) and completes the proof.The special-function representation provides the final closed-form result.Source: A. Proof of PDF · p. 13

B. Proof of CDF

The proof derives the CDF by expressing intermediate integrals, applying hyper-geometric and Meijer’s G-function identities, and substituting the resulting forms into earlier expressions.

APPENDIX C PROOF OF THEOREM 2

Theorem 2 is proved by deriving the CDF of γ_k and evaluating E_k through successive substitutions and special-function identities.

  • The proof starts from the definition of the CDF and reduces the integration in (C-3) using Lemma 1.Source: Appendix C · p. 14
  • E_k is first expressed using the definition of the Gamma function.Source: Appendix D · p. 14
  • Expression (C-4) is substituted into (D-1), after which integral I4 is solved using a cited identity.Source: Appendix D · p. 14
  • Combining I4 with (D-2) produces E_k as (16), completing the proof.Source: Appendix D · p. 14

Related papers

Loading 2209.12274v1…