PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking Communication Metrics: How Should We Measure Meaning?

Niloofar Tavakolian, Hakimeh Purmehdi, Jungyeon Baek

arXiv:2608.21626v1cs.LG

TL;DR

Semantic communication lacks standardized ways to measure meaning across modalities, tasks, and operating conditions. This survey organizes text- and image-based KPIs by communication goal, modality, receiver output, reference availability, evaluation level, and constraints, then compares their roles and limitations. It concludes that evaluation should remain modality-aware, while interpretable reference-free and multimodal KPIs remain major unresolved gaps.

  • Problem

    Semantic communication lacks standardized semantic ground truth, universal success criteria, and a single KPI that covers differing goals, modalities, tasks, and channel conditions.

  • Method

    The survey develops an evaluation-centered taxonomy of text- and image-based KPIs across communication goals, modalities, receiver outputs, reference availability, evaluation levels, and communication constraints.

  • Results

    The survey finds a layered evaluation logic spanning reconstruction, perceptual quality, representation-level similarity, downstream task utility, reference-free monitoring, and communication-aware reliability, with modality-dependent semantic correctness.

  • Takeaways & Limitations

    A unified semantic-KPI framework should combine multiple evaluation levels while remaining modality-aware in how text and image correctness are assessed.

Abstract

from arXiv · show

Semantic communication shifts the objective of communication systems from accurate symbol reconstruction toward meaning preservation, task accomplishment, and efficient information exchange. However, its evaluation remains fragmented across telecommunications, natural language processing, computer vision, and machine learning, and no single metric can characterize semantic quality across modalities, tasks, and channel conditions. This article surveys key performance indicators (KPIs) for text- and image-based semantic communication systems from a unified, evaluation-centered perspective. Unlike prior surveys primarily organized around architectures, applications, or transmission strategies, this work focuses on how semantic success should be defined and measured. Existing KPIs are classified according to communication goal, source modality, receiver output, reference availability, evaluation level, and channel or resource constraints. The survey reviews reconstruction-based, task-oriented, reference-free, representation-level, perceptual, and channel-aware metrics, and presents a cross-modality comparison of their roles, strengths, and limitations. It further analyzes how unresolved semantic-KPI challenges affect monitoring, quality assurance, resource optimization, fault diagnosis, and standardization. Key open problems include the absence of universal semantic success criteria and standardized semantic ground truth, semantic drift, limited reference-free evaluation, weak integration of machine-learning metrics with communication constraints, and the lack of relation-level and multimodal KPIs. Finally, future research directions are outlined toward standardized, interpretable, adaptive, task-aware, and communication-aware evaluation frameworks.

I. INTRODUCTION

Semantic communication evaluates whether meaning, semantic information, or receiver objectives are preserved rather than only whether transmitted symbols are accurately reconstructed. This survey organizes KPI selection around communication goals, modalities, receiver outputs, evaluation levels, and communication constraints.

  • Semantic communication shifts evaluation from symbol-level accuracy toward semantic preservation and successful receiver-side task accomplishment.
  • Communication success varies with the receiver objective, including source reconstruction, semantic-state recovery, classification, retrieval, segmentation, and question answering.
  • The survey reviews text- and image-based KPIs spanning reconstruction fidelity, semantic similarity, task utility, perceptual quality, reference-free evaluation, and telecom-oriented reliability and efficiency.
  • KPI selection is structured by communication goal, source modality, receiver output, reference availability, evaluation level, and operating constraints.
  • Text-based semantic communication: Text-based KPIs include surface-overlap, embedding-based semantic similarity, learned evaluation, task-oriented, reference-free, and telecom-oriented semantic-state metrics.
  • Semantic reconstruction metrics: Surface-overlap metrics are simple and widely used for lexical reconstruction quality but may penalize valid paraphrases because they emphasize visible token overlap.

3) METEOR:

Text semantic reconstruction metrics differ in what they compare and how directly they capture meaning. METEOR accommodates some paraphrase variation, while character-, embedding-, and learned metrics introduce distinct robustness and dependency trade-offs.

  • METEOR: METEOR uses stemming and synonym-based matching, making it more tolerant of meaning-preserving paraphrases than BLEU and ROUGE.
  • METEOR: METEOR remains reference-based and depends on lexical matching resources despite its greater paraphrase tolerance.
  • chrF: chrF compares character-level n-grams and can better handle morphological variation, spelling differences, and tokenization mismatches.
  • chrF: chrF does not directly measure semantic equivalence because it evaluates character-level surface patterns.
  • Embedding-based metrics: BERT-based sentence similarity compares learned sentence representations, but its score depends on the encoder, pooling method, layer choice, and normalization.
  • Embedding-based metrics: BERTScore compares contextual token embeddings and is more robust to meaning-preserving paraphrases than overlap metrics, but requires a reference and pretrained encoder.
  • Learned evaluation metrics: Learned evaluators such as BLEURT and COMET can model complex meaning and fluency dimensions but are model-dependent and sensitive to domain shift.

B. Goal-Oriented Text Semantic Communication KPIs

Goal-oriented text semantic KPIs evaluate whether transmitted meaning supports the receiver’s intended task, rather than whether reconstructed wording matches the source. They span intermediate outputs such as intents, slots, frames, answers, and rankings, through final task completion.

  • Goal-oriented evaluation prioritizes correct task decisions or actions over lexical similarity between source and reconstructed text.The receiver may preserve intended meaning even when the exact sentence wording differs.
  • 1) Intent Classification Accuracy: Intent accuracy measures whether the receiver predicts the correct high-level action or semantic category.It is useful when recovering the overall intent is sufficient, but may be too coarse when specific arguments are required.
  • 2) Slot Filling Precision, Recall, and F1: Slot filling evaluates whether task-relevant entities and arguments are preserved, using precision, recall, and F1 over predicted slots.Slot F1 can expose critical detail loss even when a reconstructed sentence remains grammatical.
  • 3) Frame Accuracy / Exact Match: Frame accuracy, or exact match, requires the complete semantic frame—including intent and required slots—to match the ground truth.It is stricter than intent accuracy or slot F1 because partial meaning recovery is insufficient.
  • 4) Question Answering Metrics: Question-answering KPIs use exact match and token-level F1 to evaluate whether the receiver produces the correct answer, regardless of source wording.Exact match requires normalized answer identity, whereas token-level F1 gives partial credit for overlapping answer tokens.
  • 5) Retrieval Metrics: Recall@K, MRR, and NDCG: Retrieval metrics assess whether the correct content appears near the top of a ranked candidate list, using Recall@K, MRR, and NDCG.Recall@K checks top-K inclusion, MRR rewards earlier correct results, and NDCG accommodates graded relevance.

6) Task Success Rate:

Task success rate evaluates the final objective of semantic communication, while reference-free KPIs support deployment when source references are unavailable. These metrics improve practical monitoring but involve distinct dependencies and trade-offs.

  • 6) Task Success Rate: Task success rate measures whether the receiver completes the communication’s final goal, such as making a booking, executing an API call, or selecting an action.It evaluates the end result rather than intermediate metrics such as intent accuracy or slot F1.
  • 6) Task Success Rate: A system can achieve high task success despite imperfect sentence reconstruction when essential semantic information is preserved.Conversely, fluent generated text can still fail if it loses a critical intent or slot.
  • Reference-Free Evaluation: Reference-free evaluation estimates semantic acceptability without requiring the original source message or a gold reference at the receiver.It is closer to deployment scenarios and supports monitoring of semantic reliability.
  • 1) Quality Estimation Metrics: Quality-estimation metrics use learned models to predict whether received text is meaningful, fluent, or useful without a human reference.They are model-dependent and may become unreliable under domain shift.
  • 2) Anchor-Embedding Similarity: Anchor-embedding similarity compares a reconstructed sentence with a transmitted semantic anchor, avoiding the need to retain the original sentence at evaluation time.The anchor adds communication overhead and may create privacy or security concerns because embeddings can leak source information.
  • 3) Semantic Success and Outage: Thresholded semantic similarity yields binary success or failure and semantic outage measures below-threshold meaning preservation.Results depend on the selected threshold, embedding model, and similarity function.
  • 4) Perplexity: Perplexity provides a reference-free proxy for fluency or grammatical naturalness, but fluent text does not guarantee semantic correctness.It should therefore support rather than replace primary semantic KPIs.
  • Reference-free metrics trade off deployment practicality against metric-specific weaknesses: training-data dependence, anchor overhead, threshold sensitivity, or fluency-only measurement.These trade-offs motivate telecom-style semantic state recovery metrics.

D. Telecom-Style Semantic State Recovery KPIs

Telecom-style KPIs evaluate recovery of task-relevant semantic states rather than exact source reconstruction, linking semantic quality to reliability and resource cost. The main families measure semantic distortion, error, outage, and quality–cost tradeoffs under communication constraints.

  • Semantic State Recovery: Telecom-style evaluation represents a source message as a semantic state W and assesses recovery of its estimate Ŵ rather than exact sentence reconstruction.Semantic states may be intent labels, knowledge-graph triplets, embeddings, or distributions over meanings.
  • Evaluation Implications: These KPI families shift evaluation toward semantic reliability and efficiency when systems recover task-relevant states under channel and resource constraints.Their usefulness depends on explicitly defining and justifying semantic quality, distortion, and resource measures.
  • Semantic Distortion: Semantic distortion applies a representation-dependent distance between the original and recovered semantic states.Possible distances include mismatch, Euclidean or cosine distance, mean squared error, missing entities, wrong relations, or distributional divergence.
  • Semantic Error Probability: Semantic error probability counts incorrect recovery of discrete semantic states, such as predicting CancelBooking instead of BookFlight.It is interpretable for discrete states but less straightforward for continuous embeddings and structured semantic graphs.
  • Semantic Outage Probability: Semantic outage probability measures the frequency with which recovered meaning exceeds an unacceptable quality threshold, allowing near-matches to remain acceptable.It can be plotted against SNR, bandwidth, rate, or fading, but results depend on the chosen threshold and quality measure.
  • Rate–Semantic Tradeoff: Rate–semantic evaluation studies semantic quality or distortion against transmitted symbols, bits, bandwidth, latency, energy, or other resource costs.Semantic efficiency scores and constrained optimization compare quality under resource budgets, SNRs, bandwidth limits, or compression rates.

IV. IMAGE-BASED SEMANTIC COMMUNICATION KPIS

Image-based semantic communication requires multiple KPI families because visual quality spans pixel fidelity, structure, perception, representation similarity, task utility, realism, and channel-aware reliability. Pixel and perceptual metrics capture different aspects of reconstruction but do not by themselves guarantee preservation of task-relevant semantics.

  • KPI Framework: Image-based KPIs evaluate whether visual information is preserved, recovered, or useful after transmission, spanning reconstruction, perception, representation, task utility, realism, and channel reliability.The appropriate metric depends on the visual content and receiver objective.
  • Pixel Fidelity: Pixel-fidelity metrics measure numerical closeness between original and reconstructed images but may miss loss of task-relevant visual details.MSE weights all pixels equally, so errors in object boundaries or other important regions may matter more than larger background errors.
  • Pixel Fidelity: PSNR derives from MSE and provides a logarithmic measure commonly compared across SNR, bandwidth ratio, and compression rate.Despite its communication-oriented comparability, PSNR remains pixel-level and may correlate poorly with perception or task success.
  • Structural and Perceptual Quality: SSIM compares local luminance, contrast, and structure, making it more sensitive than PSNR to blurred boundaries, smeared edges, and contrast loss.It still does not identify whether degraded structure belongs to an important object or task-critical cue.
  • Structural and Perceptual Quality: MS-SSIM evaluates structural similarity across resolutions, capturing coarse scene layout alongside fine edges, textures, and small objects.It remains a structural or perceptual measure rather than a direct test of task-level semantic correctness.
  • Structural and Perceptual Quality: LPIPS measures perceptual distance using weighted deep-network feature activations across multiple layers.It is useful for learned or generative transmission when outputs look realistic despite lacking pixel alignment with the original.

2) CLIP-Based Image Similarity:

CLIP-based and downstream task metrics evaluate whether transmitted images preserve semantic content, task utility, or distribution-level realism rather than pixel fidelity alone. Each metric captures a different receiver objective and has corresponding reference, model, or dataset limitations.

  • CLIP-Based Image Similarity: CLIP similarity evaluates image meaning in a multimodal feature space using image-image or image-text alignment.It is useful when the receiver need not reproduce the exact image but must preserve a caption, prompt, or task instruction.
  • CLIP-Based Image Similarity: CLIP similarity may miss fine-grained spatial details, small objects, medical abnormalities, or domain-specific semantics absent from its training data.This limitation makes it unsuitable as a standalone guarantee of detailed semantic preservation.
  • Downstream Visual Task Utility Metrics: Downstream task metrics compare receiver predictions with ground-truth outputs for classification, detection, segmentation, or retrieval.These metrics measure whether transmitted visual information is sufficient for the intended application rather than directly comparing the source and reconstructed images.
  • Downstream Visual Task Utility Metrics: Task metrics can disagree with image fidelity: low-PSNR images may support correct classification, while visually acceptable images may fail detection or segmentation.Accordingly, task-oriented systems should report task metrics together with reconstruction or perceptual metrics.
  • Image Retrieval Metrics: Retrieval metrics assess ranked-output quality, but depend on the candidate set, relevance labels, and embedding model.Recall@K checks whether the correct item appears among the top K candidates, whereas mean reciprocal rank measures how early the first correct item appears.
  • Distribution-Level Realism Metrics: FID, KID, and Inception Score evaluate distribution-level realism or diversity across image sets, not source-image semantic preservation.FID and KID compare real and generated distributions, while Inception Score uses classifier confidence and class diversity; these metrics depend on datasets, feature extractors, or classifiers.

F. Channel-Aware Semantic Reliability Metrics

Channel-aware semantic reliability metrics evaluate whether task-relevant meaning remains useful as channel and resource conditions change. They connect semantic quality to distortion, outage, and quality–resource trade-offs rather than treating image evaluation independently from communication.

  • Channel-Aware Semantic Reliability Metrics: Channel-aware evaluation should report semantic quality against SNR, bandwidth ratio, transmitted symbols, latency, or energy.This connects image semantic communication to classical distortion, outage, and rate-reliability concepts.
  • Semantic Distortion: Visual semantic distortion measures differences between original and recovered semantic representations using representation- or task-specific distance functions.Distances may use cosine or Euclidean embedding distance, missing objects or wrong relations, 1−Dice, or 1−mAP.
  • Semantic Outage: Semantic outage probability measures how often semantic quality falls below an acceptable similarity or distortion threshold.It can be plotted against SNR, compression ratio, bandwidth, latency, or transmitted semantic symbols, but the threshold must be reported clearly.
  • Task Outage: Task outage defines failure directly through downstream requirements, such as Dice below a threshold, mAP below a required level, or an incorrect classification.This can capture application failure even when a reconstructed medical image appears visually smooth.
  • Semantic Efficiency: Semantic efficiency relates a semantic quality score to resource cost, including transmitted symbols, bandwidth, latency, or energy.Quality–resource curves make evaluation communication-aware by showing how meaning changes with SNR, bandwidth ratio, compression rate, or latency.
  • Cross-Modality Comparison: The common evaluation logic across text and images selects KPIs according to the receiver objective rather than modality alone.Reconstruction, representation, task, reference-free, and telecom-style metrics serve different evaluation goals.

B. Open Challenges and Gaps in Semantic Communication KPI Design

Semantic KPI design lacks the stable, reproducible, and universally applicable measurement basis available to conventional communication systems. The resulting gaps affect benchmarking, monitoring, resource optimization, real-time evaluation, and scheduler fairness.

  • Universal Semantic Success Criteria: No single semantic success criterion generalizes across applications because success may mean preserved meaning, completed tasks, or matched sender intent.This prevents meaningful comparison across systems, vendors, and research groups without agreed application-specific criteria or a common semantic reference.
  • Standardization and Reproducibility: Semantic metrics lack standardized methodologies, may conflict for the same output, and can change when embedding models are updated or fine-tuned.This fragmentation challenges the standardization and benchmarking role traditionally served by conventional KPIs.
  • Semantic Drift: Semantic drift arises from independently evolving knowledge bases, asynchronous model updates, and changes in operating environments or user behavior.Such non-stationarity makes it difficult to distinguish genuine semantic failures from expected effects of deployment drift.
  • Cross-Layer Entanglement: Cross-layer entanglement means throughput or bit-error improvements do not necessarily maximize semantic utility, while normal physical KPIs may miss semantic failures.Semantic communication couples physical and MAC layers with application semantics, AI inference, knowledge bases, and edge/cloud intelligence.
  • Evaluation Cost: Semantic KPI computation can require inference, embeddings, multimodal processing, language-model reasoning, or knowledge-graph analysis, adding latency, energy, and processing costs.A multi-second KPI is unsuitable for millisecond-scale URLLC monitoring, and evaluation overhead can exceed the monitored application's latency budget.
  • Semantic Fairness: Semantic resource allocation lacks an established fairness criterion comparable to proportional fairness or max-min fairness.Open alternatives include equal semantic accuracy, equal task-success probability, or priority based on semantic criticality.

6) Multi-User Semantic Fairness:

Semantic KPI evaluation lacks universal success criteria, standardized references, and consistent benchmarks across modalities, objectives, and operating conditions. The survey organizes these gaps and motivates adaptive, interpretable, communication-aware evaluation.

  • Evaluation gaps: No standardized semantic reference makes performance claims across systems, vendors, and research groups difficult to compare reliably.A KPI-selection framework can organize evaluation procedurally but cannot establish universal semantic ground truth.
  • Evaluation gaps: No single KPI fits every system because meaningful evaluation depends on the receiver’s reconstruction, inference, or task objective.Faithful image reconstruction may use MSE or PSNR, whereas task-oriented systems may require accuracy, F1, mAP, Dice, or Recall@K.
  • Communication constraints: Machine-learning metrics often omit SNR, bandwidth, rate, latency, energy, transmitted symbols, and computational complexity.High BERTScore, CLIP similarity, or task accuracy may therefore coexist with impractical communication or computational resource requirements.
  • Reference-free evaluation: Reference-free metrics remain limited because deployment receivers often lack the original source or a gold-standard reference.Learned quality estimators, semantic anchors, and fluency proxies can support evaluation but may misjudge semantically critical errors.
  • Task awareness: Task-critical content must be distinguished from non-critical information to define service-quality thresholds and detect failures reliably.The priority differs by task, such as weather understanding versus medical diagnosis.
  • Multimodal evaluation: Object- and pixel-level image metrics can overlook relationships among objects, while multimodal settings require context consistency beyond global alignment.Scene graphs and subject-predicate-object representations address relation-level information, but structured reference-free multimodal quality remains unresolved.
  • Adaptive evaluation: Static KPI-selection procedures cannot adapt during operation to changing channel conditions, task distributions, or receiver states.Learned selection or weighting could use channel state, task confidence, or semantic-anchor similarity to support runtime adaptation.
  • Unified framework: The survey compares layered KPI families from reconstruction and perception through representation similarity, task utility, reference-free monitoring, and communication-aware reliability.The framework remains modality-aware because text is linguistic and intent-based, whereas images are spatial, perceptual, object-based, and task-dependent.

A. BLEU Example

The examples contrast surface-overlap metrics with learned and embedding-based semantic metrics using meaning-preserving and unrelated reconstructions. They show why semantic communication evaluation must distinguish lexical recovery from meaning preservation.

  • BLEU Example: ROUGE records lexical overlap but may underestimate meaning preservation when equivalent content uses different words.“Cold” and “freezing” express similar weather conditions despite limited exact overlap.
  • BLEU Example: METEOR can increase matching through stemming and synonym-based matching, unlike exact word-overlap measures.The metric is illustrated on “cold” versus “freezing,” where semantically related terms can count as matched.
  • BLEU Example: chrF compares character n-grams and helps with spelling, morphology, or tokenization differences, but does not directly measure semantic equivalence.chrF++ additionally includes word n-grams.
  • BLEU Example: Embedding-based sentence similarity gives meaning-preserving reconstructions higher scores than unrelated reconstructions.It computes cosine similarity between sentence embeddings rather than relying only on surface word overlap.
  • BLEU Example: BERTScore can reward contextual substitutions such as “freezing” for “cold” while penalizing unrelated tokens such as “dog,” “sleeping,” and “sofa.”It compares contextual token embeddings and assigns partial semantic credit despite low exact n-gram overlap.
  • BLEU Example: BLEURT uses a learned reference-candidate evaluator that is expected to score meaning-preserving reconstructions higher than topic-changing outputs.Its exact score depends on the checkpoint, training data, and domain, so the model version should be reported.

B. Slot Filling Precision, Recall, and F1 Example

The examples show how task-oriented text metrics measure progressively different levels of semantic recovery, from partial slot preservation to complete frames, answers, and retrieval outcomes.

  • B. Slot Filling Precision, Recall, and F1 Example: Slot filling evaluates whether task-relevant entities or arguments survive transmission, rather than only whether the sentence remains grammatical.Recovering the destination while losing the date yields partial semantic recovery.
  • B. Slot Filling Precision, Recall, and F1 Example: Frame accuracy is stricter than intent accuracy or slot F1 because correctness requires the complete intent and all required slots.A missing date makes an otherwise correct BookFlight intent and destination frame incorrect.
  • B. Slot Filling Precision, Recall, and F1 Example: Exact Match is strict, whereas token-level F1 gives partial credit when an answer contains the correct information with extra words.The example contrasts an exact answer with “to Montreal city,” which still contains the correct token.
  • B. Slot Filling Precision, Recall, and F1 Example: Recall@K tests whether the correct retrieved item appears within the top K results, while MRR uses the reciprocal rank of the first correct item.These metrics evaluate retrieval sufficiency without requiring textual identity with the original sentence.

F. Task Success Rate Example

Task success rate evaluates whether communication preserves enough information to complete the receiver’s intended goal. Reference-free and threshold-based KPIs extend evaluation to deployment, but introduce model, overhead, and threshold considerations.

  • F. Task Success Rate Example: Task success rate is an end-to-end KPI that counts whether the intended task is completed, not whether reconstructed text resembles the source.A flight booking fails when the destination is correct but the required date is missing.
  • F. Task Success Rate Example: Reference-free evaluation estimates semantic acceptability without giving the receiver the original sentence or a gold reference.Quality estimation, semantic anchors, similarity thresholds, and fluency proxies provide deployment-oriented alternatives.
  • F. Task Success Rate Example: Quality estimation can score fluent, context-compatible reconstructions higher than outputs that change topic, but it is model-dependent and may fail under domain shift.Exact scores depend on the quality-estimation model, checkpoint, training data, and input format.
  • F. Task Success Rate Example: Anchor-embedding similarity enables receiver-side semantic checking without the full source, but transmitting the anchor or tag adds overhead.The receiver compares an embedding of the reconstruction with sender-provided semantic side information.
  • F. Task Success Rate Example: Semantic success rate converts similarity into binary success using threshold τ, while semantic outage is the complementary failure probability.With τ = 0.80, illustrative similarities of 0.995 and 0.49 produce success indicators 1 and 0, respectively.

D. Perplexity Example

The examples illustrate reference-free fluency assessment, weighted semantic-state distortion, semantic error and outage probabilities, and how these measures support reliability-oriented evaluation.

  • Perplexity Example: Perplexity provides a reference-free fluency proxy, with lower values generally indicating more fluent or natural sentences.It is computed from a language model’s token probabilities and average negative log-likelihood.
  • Perplexity Example: Perplexity alone cannot guarantee semantic correctness, because a fluent unrelated sentence may receive a low score.It can detect corrupted or ungrammatical outputs but should not be used alone as a semantic communication KPI.
  • Semantic-State Metrics: Weighted semantic distortion measures mismatches between recovered and original semantic states according to component-specific importance weights.An indicator marks incorrect or missing components, and the weighted sum reflects their relative contribution.
  • Semantic-State Metrics: Semantic distortion represents partial failure by distinguishing a small missing detail from a major semantic error, unlike binary success measures.This makes the score useful when semantic states contain multiple structured components.
  • Semantic-State Metrics: Semantic error probability counts how often the recovered semantic state differs from the original state, extending classical error probability from symbols to semantic states.For discrete states, the error event is defined as 𝑊̂≠𝑊.
  • Semantic Reliability: Semantic outage probability converts a continuous quality score into a reliability measure by declaring outage when quality falls below threshold 𝜏.The resulting measure can be plotted against SNR, bandwidth, rate, or latency.

D. Rate-Semantic Tradeoff Example

The rate-semantic example shows that higher semantic quality can require more transmitted symbols and therefore greater communication cost. The controlled image experiment complements this tradeoff by examining how pixel, perceptual, semantic, and reliability-oriented KPIs respond to degradation.

  • Rate-Semantic Tradeoff Example: Rate-semantic tradeoff evaluates how semantic quality changes as the communication resource budget changes.The objective is to preserve sufficient meaning while transmitting fewer symbols, bits, or features.
  • Rate-Semantic Tradeoff Example: Configuration C uses twice as many semantic symbols as B while improving similarity from 0.86 to 0.93, illustrating a quality-cost tradeoff.With threshold 𝜏=0.80, configuration A fails while B and C succeed.
  • Rate-Semantic Tradeoff Example: A practical system should report semantic quality together with resource usage, such as similarity, distortion, or outage versus rate or SNR.This pairing exposes the cost of improving semantic quality rather than reporting quality in isolation.
  • Controlled Image KPI Experiment: The image experiment uses one original street-scene image and controlled receiver-side degradations rather than a learned semantic communication model.The setup is intended to demonstrate KPI behavior reproducibly, not benchmark a complete system.
  • Controlled Image KPI Experiment: Each sample-level result compares the original image independently with one receiver-side output using pixel, structural, perceptual, and CLIP image-image metrics.Task-level metrics require labels or a downstream model, so the controlled experiment focuses on image-pair KPIs and threshold-based outage.
  • Interpreting the Sample-Level Results: As degradation increases, pixel fidelity, structural quality, perceptual similarity, and CLIP similarity are expected to worsen, while these metrics may miss task-critical details.MSE should increase, PSNR decrease, SSIM and MS-SSIM decrease, and LPIPS increase; CLIP similarity can miss small domain-specific details.
Loading 2608.21626v1…