PaperlayerView exact revision on arXiv

Source-linked AI summary

HintKD: Hint Knowledge Distillation for Bandwidth-Constrained Cloud-Edge Inference

Wanling Luo, Jiayi Zhang, Bokai Xu, Ziheng Liu

arXiv:2610.08412v1cs.ITmath.IT

TL;DR

Cloud-edge inference can reduce UE computation, but exchanging information during inference makes it sensitive to latency, channel quality, and uplink bandwidth. HintKD compresses teacher guidance into discrete hints and uses them to adapt UE-side student features. Experiments report competitive accuracy with only a few bits of inference payload per sample.

  • Problem

    Cloud-edge collaboration relies on exchanging information during inference, making system performance sensitive to round-trip latency, channel quality, and uplink bandwidth.

    Source: §I · p. 1

  • Method

    HintKD trains a cloud teacher with MCMI, quantizes its guidance into discrete hints, and uses a UE-side FiLM adapter to modulate student features.

    Source: §I · pp. 1–2, C. Edge-Side: Hint Adaptation via FiLM · p. 3

  • Results

    Experiments on DeepSense 6G and MNIST report competitive accuracy while reducing inference-side payload to only a few bits per sample.

    Source: §IV · p. 6

  • Takeaways & Limitations

    For bandwidth-constrained edge intelligence, transmitting structured guidance can be more efficient than transmitting raw features or full logit vectors.

    Source: §IV · p. 6

Abstract

from arXiv · show

Collaborative inference between the edge cloud and user equipment (UE) is a promising paradigm for deploying large deep neural networks (DNNs) in 6G networks. However, existing cloud-edge inference and distillation schemes often require the real-time transmission of high-dimensional intermediate features or soft probability vectors, which imposes a substantial communication burden on bandwidth-limited wireless links. To address this challenge, we propose HintKD, a bandwidth-constrained distillation framework for cloud-edge inference. The core idea is to compress the teacher's guidance into compact discrete hints rather than transmit raw features directly. Specifically, a cloud-side teacher is first trained with a maximum conditional mutual information (MCMI) objective to preserve informative intra-class variations. Its latent representation is then mapped to a compact codebook through differentiable vector quantization. During deployment, the cloud transmits only a hint index, and the UE uses a lightweight FiLM-based adapter to modulate student features according to the received codeword. Experiments on the DeepSense 6G beam prediction task show that HintKD preserves competitive accuracy while reducing the inference-side communication payload from O(D_t) for feature transmission or O(C) for logit transmission to O(1) for discrete hint transmission. In addition, experiments on the MNIST dataset further verify the generality of the proposed distillation method across different datasets and tasks. These results show that HintKD achieves a favorable accuracy-bandwidth trade-off by replacing high-dimensional feature or logit transmission with a compact discrete hint index.

I. INTRODUCTION

Cloud-edge inference can ease UE resource limits, but wireless exchange makes collaborative inference sensitive to link conditions. HintKD addresses bandwidth constraints by sending compact discrete hints and reports a favorable accuracy-bandwidth trade-off.

I. INTRODUCTION

  • Cloud-edge collaboration shifts resource-intensive processing away from constrained UEs, but exchanged inference information makes performance sensitive to latency, channel quality, and uplink bandwidth.Poor channel conditions or coverage holes can sharply degrade quality of service.Source: §I · p. 1
  • Existing KD transfers teacher knowledge through logits or richer intermediate representations, but feature- and logits-based transmission can be prohibitive on bandwidth-constrained wireless links.Feature payload scales with teacher feature dimension, while logits payload scales with the number of classes.Source: §I · p. 1
  • HintKD compresses cloud-side teacher guidance into a discrete hint, using MCMI teacher training, differentiable codebook design, and FiLM-based student adaptation.The UE receives a hint index and uses its codeword to modulate local student features.Source: §I · pp. 1–2
  • DeepSense 6G experiments report competitive predictive accuracy and a more favorable accuracy-bandwidth trade-off than Feature KD and Logits KD, while MNIST experiments support generality across datasets and tasks.The framework replaces feature- or logit-level real-valued transmission with a low-bit discrete index.Source: §I · p. 2

II. HINTKD CLOUD-EDGE INFERENCE FRAMEWORK

The section introduces the HintKD framework through its overall system architecture.

II. HINTKD CLOUD-EDGE INFERENCE FRAMEWORK

  • The proposed HintKD framework is presented through an overall system architecture figure.The passage identifies the figure but does not describe its components.Source: §II · p. 2

A. Training Formulation

The training formulation first trains a cloud teacher to retain informative input variation within classes, then trains the student conditioned on quantized teacher features. The student and hint mechanism are optimized jointly after the teacher is frozen.

A. Training Formulation

  • The cloud teacher maps inputs to latent features and class probabilities, and its MCMI objective encourages predictions to retain input information given the class.This is intended to preserve intra-class environmental features for later hint generation.Source: A. Training Formulation · p. 2
  • The empirical CMI estimate uses the average KL divergence between each teacher output and its class-conditional centroid to encourage sample-specific environmental features.The passage describes this divergence as providing richer guidance for subsequent hint generation.Source: A. Training Formulation · p. 2
  • After teacher training, the teacher is frozen and the student and hint mechanism are jointly optimized to minimize student prediction error conditioned on quantized teacher features.The formulation bridges the fixed teacher’s knowledge with the student’s inference logic.Source: A. Training Formulation · pp. 2–3

B. Cloud-Side: Contextual Vector Quantization

The cloud-side quantizer maps continuous teacher features to discrete codebook indices, with relaxed assignments used during training and hard indices transmitted at deployment. This yields constant-order inference payload rather than feature- or class-scaled transmission.

B. Cloud-Side: Contextual Vector Quantization

  • The quantizer maps continuous teacher features to discrete indices represented by codebook prototypes, whose size is K.The codebook vectors are latent prototypes.Source: B. Cloud-Side: Contextual Vector… · p. 3
  • During training, Gumbel-Softmax produces relaxed codebook assignments to support gradient flow.The encoder projection aligns the teacher feature dimension with the codebook, and the relaxation uses Gumbel noise and temperature.Source: B. Cloud-Side: Contextual Vector… · p. 3
  • ⌈log2 K⌉ bits per hint index gives O(1) deployment payload, versus O(Dt) for Feature KD and O(C) for Logits KD.The passage states that O(1) < O(C) < O(Dt) in the described setting.Source: B. Cloud-Side: Contextual Vector… · p. 3
  • Offline joint training transmits a soft probability vector for differentiability, whereas online deployment sends only the hard hint index.The deployment procedure retrieves the codeword corresponding to that index at the edge.Source: B. Cloud-Side: Contextual Vector… · p. 3

C. Edge-Side: Hint Adaptation via FiLM

The UE retrieves the codeword indexed by the cloud hint and uses adapter projections to modulate its local student feature before classification.

C. Edge-Side: Hint Adaptation via FiLM

  • The UE retrieves the codeword for the received hint index and uses adapter projections to conditionally modulate its local student feature.The student backbone maps the raw input to a local feature, while the projections map the hint codeword to scale and shift parameters.Source: C. Edge-Side: Hint Adaptation via FiLM · p. 3
  • The modulation preserves the student feature dimension, allowing the modulated representation to feed the task-specific classifier.The classifier produces the student prediction probability vector.Source: C. Edge-Side: Hint Adaptation via FiLM · p. 3
  • The student prediction is used to compute cross-entropy loss against the ground-truth labels.The passage describes the prediction as part of the complete inference chain from transmitted hint to student decision.Source: C. Edge-Side: Hint Adaptation via FiLM · pp. 3–4

D. Theoretical Analysis

The analysis characterizes HintKD’s communication costs and gives a hint-conditioned generalization bound. It relates the bound to simpler per-hint learning problems and states when training communication is more efficient.

D. Theoretical Analysis

  • Conditioning on K discrete hints partitions the student hypothesis space into K subspaces, yielding a tighter bound through lower-complexity per-hint problems.The bound also includes the teacher-student alignment error and the sample counts assigned to each hint.Source: D. Theoretical Analysis · p. 4
  • Inference communication falls from real-valued payloads scaling linearly with Dt or C to an integer payload scaling logarithmically with K.The paper contrasts Feature KD and Logits KD with HintKD’s discrete index transmission.Source: D. Theoretical Analysis · p. 4
  • During training, HintKD is more communication-efficient than Feature KD when K < Dt and than Logits KD when K < C.The stated typical Edge AI dimensions are K < C ≪ Dt.Source: D. Theoretical Analysis · p. 4

III. SIMULATION RESULTS

Experiments evaluate HintKD on DeepSense 6G Scenario 9 beam prediction and MNIST digit classification. Results show a trade-off between hint bandwidth and accuracy on Scenario 9, while maintaining competitive accuracy across both tasks.

DeepSense 6G Scenario 9

  • 49.53% Top-1 accuracy with 4-bit MCMI-HintKD exceeds the offline student’s 47.12% by 2.41 percentage points on Scenario 9.The comparison is reported in Table II.Source: Table II · p. 5, §III · p. 5

MNIST

  • On MNIST, 4-bit HintKD reaches 99.17% Top-1 with an MLL-trained teacher and 99.06% with an MCMI-trained teacher, versus 99.05% for the offline student.Top-5 accuracy remains 99.97% under the reported setting.Source: Table II · p. 5, §III · p. 5

Hint bandwidth

  • 45.44% Top-1 accuracy at 2 bits and 47.07% at 6 bits both fall below the 4-bit setting on Scenario 9.The authors identify 4 bits as the best trade-off between compact representation and predictive performance.Source: §III · p. 5

Teacher objectives

  • MCMI-trained teachers achieve higher CMI, whereas MLL-trained teachers achieve lower LL; these objective differences affect student distillation performance.Fig. 2 compares accuracy, CMI, and LL across methods.Source: §III · p. 4, Figure 2 · p. 6

Overall results

  • Across Scenario 9 and MNIST, the results support HintKD’s accuracy-bandwidth trade-off and its applicability to beam prediction and image classification.The evaluation uses Scenario 9 as the main beam-prediction task and MNIST to test generality.Source: §III · pp. 4, 6

IV. CONCLUSION

HintKD replaces high-dimensional teacher transmissions with compact discrete hints for cloud-edge distillation. Experiments on DeepSense 6G and MNIST show competitive accuracy with only a few bits per sample at inference.

IV. CONCLUSION

  • HintKD replaces high-dimensional teacher transmissions with compact discrete hints.The framework combines an MCMI-trained teacher, differentiable codebook learning, and FiLM-based adaptation on the UE.Source: §IV · p. 6
  • Experiments on DeepSense 6G and MNIST show competitive accuracy while reducing inference-side payload to a few bits per sample.Source: §IV · p. 6
  • Future work will investigate adaptive codebooks for time-varying channels and extend the framework to broader multimodal sensing scenarios.Source: §IV · p. 6

Related papers

Loading 2610.08412v1…