Performance Optimization for Semantic Communications: An Attention-based Reinforcement Learning Approach

Contribution

• Prior work includes semantic communication systems and evaluation metrics, while this paper jointly considers semantic extraction, transmission, recovery, and performance evaluation.The paper positions this joint framework as its main contribution.Source: A. Related Works · pp. 1–2, B. Contributions · p. 2

Framework

• The framework extracts text meaning into a knowledge graph of entity-relation-entity semantic triples, which users convert back into text with graph-to-text generation.The semantic pipeline jointly covers extraction, transmission, and recovery.Source: B. Contributions · p. 2

Performance metric

• MSS measures whether recovered information is correct and whether it contains all information from the original text.The metric jointly captures semantic accuracy and semantic completeness.Source: B. Contributions · p. 2

Problem formulation

• The BS jointly optimizes resource allocation and the semantic information selected for transmission to maximize total MSS under delay constraints.The optimization addresses partial semantic-information transmission when wireless resources are limited.Source: B. Contributions · p. 2

Proposed algorithm

• The attention PPO algorithm evaluates triple importance and relates importance distributions to total MSS when selecting semantic information and allocating resources.This relationship supports effective policies for resource allocation and semantic-information transmission.Source: B. Contributions · p. 2

Reported results

• Compared with a standard communication network, APPO reduces transmitted words by up to 41.3% while achieving a 2-fold improvement in total MSS.The paper also analyzes convergence and reports a locally optimal solution guarantee.Source: B. Contributions · p. 2

Resource-block transmission

• OFDMA provides Q downlink orthogonal resource blocks for semantic-information transmission.The model allocates these blocks to serve the users.Source: B. Transmission Model · p. 3

Resource allocation

• Each user's allocation is a binary vector, where α_i,q = 1 means resource block q is assigned to user i.The model assumes each resource block is allocated to only one user and each user occupies only one resource block.Source: B. Transmission Model · p. 4

Channel model

• The downlink capacity depends on each resource block's bandwidth, BS transmit power, interference, noise, and the user channel gain.The channel gain incorporates Rayleigh fading and BS-user distance.Source: B. Transmission Model · p. 4

Delay constraint

• A transmission-delay limit D determines the maximum number of semantic-information tokens that can be sent to each user.The model assumes constant transmit power and allows transmit-power optimization as an extension.Source: B. Transmission Model · p. 4

Partial semantic information

• The BS transmits a selected subset of semantic triples as the partial semantic information for user i.The selected graph must satisfy the transmission-delay constraint.Source: C. Text Recovery · p. 4

Text recovery

• After receiving the partial graph, each user applies a graph-to-text generation model to recover coherent multi-sentence text.The model is assumed to be well-trained and shared among all users.Source: C. Text Recovery · p. 4

Recovery quantities

• The recovered text length is represented by M_i, while H_i denotes the number of selected semantic triples.These quantities describe the partial semantic information and its recovered output.Source: C. Text Recovery · p. 4

• MSS jointly captures whether recovered information is correct and whether the recovered text contains the original information.Source: D. Semantic Similarity Model · p. 4
• Semantic accuracy is the ratio of correct token occurrences in recovered text to all token occurrences in recovered text.Source: D. Semantic Similarity Model · p. 5
• Semantic completeness is the ratio of correct token occurrences in recovered text to token occurrences in the original text.Source: D. Semantic Similarity Model · p. 5
• The MSS combines semantic accuracy and completeness using weight parameter ϕ.The parameter ϕ adjusts the contributions of the two components.Source: D. Semantic Similarity Model · p. 5
• An additional penalty ξ_i accounts for short recovered text, while token matching determines correct occurrences.If a recovered token appears more often than in the original, only the matching occurrences count as correct.Source: D. Semantic Similarity Model · p. 5
• Changing ξ_i allows users to use received partial semantic information for text recovery while controlling the accuracy–completeness tradeoff.Source: D. Semantic Similarity Model · p. 5

• The optimization objective is to maximize total MSS across all users while satisfying the transmission delay requirement.Source: E. Problem Formulation · p. 5
• The problem jointly optimizes each user’s resource-block allocation and the partial semantic information transmitted.Source: E. Problem Formulation · p. 5
• Each user can occupy only one resource block, and each resource block can be allocated to only one user.Source: E. Problem Formulation · p. 5
• The total MSS depends on the selected semantic information, resource-block allocation, and a neural-network text-generation model.Source: E. Problem Formulation · p. 5
• The optimization is difficult because its non-convex objective relationship cannot be accurately characterized and semantic selection is coupled with resource-block allocation.Source: E. Problem Formulation · p. 5

• APPO combines a proximal policy optimization-based reinforcement-learning algorithm with an attention network.Source: §III · p. 5
• The attention network calculates an importance value for each semantic triple.Source: §III · p. 5
• APPO relates the importance distribution of semantic triples to total MSS when optimizing resource-block allocation and semantic information selection.Source: §III · p. 6
• The trained text-recovery model is shared by users and the base station, enabling the base station to obtain total MSS after allocation and partial selection are determined.Source: §III · p. 6

• A softmax layer normalizes triple importance values into an importance vector whose length equals the number of semantic triples.Source: A. Attention Network for Importance… · p. 6
• The attention network receives token-vector sequences representing a semantic triple and the original text.Source: A. Attention Network for Importance… · p. 6
• Its hidden layer finds correlations between tokens in each semantic triple and tokens in the original text.Source: A. Attention Network for Importance… · p. 6
• The network uses parameter matrices W_tri and W_tex, each with dimensions D_a × D_x, to compute token representations.Source: A. Attention Network for Importance… · p. 6
• The output layer produces an importance value for each semantic triple.Source: A. Attention Network for Importance… · p. 6
• Each normalized importance value y_i lies in [0, 1] and represents the importance of its corresponding semantic triple.Source: A. Attention Network for Importance… · p. 6
• Because MSS depends on received semantic information and resource-block allocation, the importance distribution alone cannot directly yield the optimal allocation and selection.Source: A. Attention Network for Importance… · p. 6

• The BS acts as the agent, determining resource-block allocation and the partial semantic information transmitted to each user.Source: B. Components of the APPO Algorithm · p. 6
• Each action is a resource-block allocation vector satisfying the optimization constraints; allocated blocks determine which important triples can be transmitted before the delay threshold.Source: B. Components of the APPO Algorithm · pp. 6–7
• For each user, the transmitted partial semantic information contains the most important triples in its semantic information subject to the delay constraint.Source: B. Components of the APPO Algorithm · pp. 6–7
• The state is the importance distribution of all users' semantic information, represented as s = [f(G1), . . . , f(GU)] in a continuous state space.Source: B. Components of the APPO Algorithm · p. 7
• A DNN parameterized by θ maps each state to an action probability πθ(s, a) = P(a|s), with the objective of maximizing total MSS.Source: B. Components of the APPO Algorithm · p. 7
• The reward is the total MSS resulting from an action at a state, so maximizing the APPO reward solves the total-MSS maximization problem.Source: B. Components of the APPO Algorithm · p. 7

• APPO is trained offline using historical semantic-communication data for resource-block allocation and semantic-information selection.Source: C. APPO for Total MSS Maximization · p. 7
• The stored policy samples K resource-block allocation actions, which form the training set for the policy that maximizes total MSS.Source: C. APPO for Total MSS Maximization · p. 7
• The policy objective combines expected total-MSS reward with a penalty for the Kullback–Leibler divergence from the stored policy.Source: C. APPO for Total MSS Maximization · p. 7
• Policy parameters are updated by gradient ascent using a learning rate δ to improve total MSS.Source: C. APPO for Total MSS Maximization · p. 7
• The penalty coefficient λ is adjusted according to the divergence between stored and updated policies, controlling learning-rate changes.Source: C. APPO for Total MSS Maximization · p. 8
• Iterative policy updates learn the relationship between semantic-information importance distributions and total MSS, yielding a policy for resource allocation and semantic-information selection.Source: C. APPO for Total MSS Maximization · p. 8

Complexity

• APPO complexity includes attention-network computation for semantic-triple importance distributions and trained-policy computation for resource allocation.Source: D. Complexity and Convergence of APPO · p. 8
• The attention-network parameter matrices have size Da × Dx, with the stated assumption Da ≪ Dx, including Da = 64 and Dx = 500 as an example.Source: D. Complexity and Convergence of APPO · p. 8
• The policy action space contains all possible resource allocations, with size |A| = U! (U−Q)!, while the state depends on semantic-information importance distributions.Source: D. Complexity and Convergence of APPO · p. 8
• Given the APPO structure, computational complexity depends on the number of users, resource blocks, and semantic triples in each extracted semantic information.Source: D. Complexity and Convergence of APPO · p. 8

Convergence

• APPO convergence requires a lower-bound objective, evaluable actions sampled from a stored policy, and monotonic increases in the objective during updates.Source: D. Complexity and Convergence of APPO · p. 8
• The KL-divergence penalty controls policy differences, while learning-rate adjustment supports convergence to a locally optimal solution.Source: D. Complexity and Convergence of APPO · pp. 8–9

Semantic information extraction

• The attention network estimates semantic-triple importance from context-dependent token representations, enabling the BS to identify which triples matter most.Correlations between triples and text tokens are computed from contextual representations rather than word-frequency statistics.Source: §IV · p. 9

Data reduction

• In one example, transmitting 86 semantic-information tokens instead of 178 original-text tokens reduces the data size by 51.7%.The semantic triples are assigned an importance distribution summing to 1; the example contrasts a triple importance of 0.1923 with another of 0.0385.Source: §IV · p. 9

Partial semantic transmission

• The APPO algorithm selects important triples and allocates resource blocks, improving total MSS by 36.7% over baseline a) and 2-fold over baseline b) in the illustrated example.The recovered text based on selected important triples contains the original text’s main meaning.Source: §IV · pp. 9–10

Data reduction

• Compared with baselines c) and e), the framework reduces transmitted data size by up to 41.3% and 84%, respectively.The reduction becomes larger for longer texts because multiple sentences can be represented by a single semantic triple.Source: §IV · pp. 10, 12

Algorithm performance

• APPO converges after 100 iterations, versus about 300 for APG and 250 for ADQN, while improving MSS by 11.2% and 18.5%, respectively.The reported convergence comparison uses a semantic communication network with 10 RBs and averages results over 10 independent runs.Source: §IV · p. 12
• On DocRED, APPO improves total MSS by 17.8%, 26.8%, and 2-fold over baselines a), b), and c), respectively; on AGENDA, the gains are 16.9%, 22.3%, and 1-fold.The paper attributes these gains to dynamic learning-rate adjustment and transmitting important semantic triples under limited wireless resources.Source: §IV · p. 12

• The framework models textual semantic information with a knowledge graph and introduces MSS to capture semantic accuracy and completeness.Source: §V · p. 13
• Total MSS is evaluated as the number of resource blocks, users, and masked tokens varies.Source: Figure 10 · p. 13, Figure 11 · p. 14, Figure 12 · p. 14
• The optimization jointly selects resource blocks and partial semantic information under wireless resource and transmission-delay constraints.Source: §V · p. 13
• APPO estimates triple importance with attention and links importance distributions to total MSS for resource allocation and semantic-information selection.Source: §V · p. 13
• The proposed framework significantly reduces the data size required for transmission while increasing total MSS.Source: §V · p. 14