PaperlayerView exact revision on arXiv

Source-linked AI summary

Freshness-Aware Constrained Sensing-Aided Beam Prediction with Knowledge Distillation

Abolfazl Zakeri, Nhan Thanh Nguyen, Ahmed Alkhateeb, Markku Juntti

arXiv:2609.01225v1eess.SP

TL;DR

The paper addresses beam prediction when fresh sensory data cannot be continuously acquired under sensing budgets or failures. It incorporates AoI into a gated visual predictor, compares fixed sampling policies, and distills an unconstrained teacher into a constrained student. Experiments on DeepSense 6G show near-doubling of top-1 accuracy at strict budgets and identify teacher training regime as more consequential than distillation loss.

  • Problem

    Existing sensing-aided beam prediction frameworks generally assume continuously available fresh sensory data, despite practical sensing budgets, failures, and intermittent connectivity.

  • Method

    The framework imposes an average sensing-rate constraint, fuses encoded AoI with visual features through gating, evaluates three fixed sampling policies, and distills an unconstrained teacher into a constrained student.

  • Results

    AoI fusion nearly doubles top-1 beam prediction accuracy at αmax = 0.1 relative to an image-only baseline, while age-aware KD achieves the best overall accuracy-complexity trade-off.

  • Takeaways & Limitations

    AoI can serve as a zero-cost contextual signal for input reliability, and KD can regularize constrained inference rather than serving only model compression.

Abstract

from arXiv · show

Beam prediction leveraging environmental data reduces over-the-air beam training overhead. Existing frameworks, however, assume continuous access to fresh sensory data, an assumption that breaks down under sensing budget constraints or sensor failures. To make this more practical, this paper proposes a sensing-aided beam prediction framework that operates under an average sensing rate constraint. We incorporate the age of information (AoI) directly into the beam prediction pipeline as a synthetic input modality: the age of the most recently captured data is encoded and fused with visual features through a gating mechanism. This provides the predictor with explicit context information about input sensory data reliability. We formalize and examine three fixed sampling policies, accumulated, uniform, and randomized, under the average sensing budget. We further develop a knowledge distillation (KD) framework that operates as a robustness regularizer rather than a pure model compression method. In particular, the high-capacity teacher is trained unconstrained on fully sampled data, and its representational knowledge is transferred to a compact student deployed under the sensing budget. We conduct numerical experiments on the DeepSense 6G data set. The results show that AoI fusion nearly doubles top-1 accuracy at strict sensing budgets, and age-aware models achieve near-optimal top-3 accuracy with only 20% of the data. Furthermore, we find that the teacher training regime is a more consequential design choice than the distillation loss function.

I. Introduction

The paper addresses beam prediction when fresh sensory data is unavailable under sensing constraints. It introduces age-aware fusion, fixed sampling policies, and knowledge distillation for constrained inference, with experiments showing strong low-budget gains.

  • Motivation: Fresh sensory data is often unavailable because of power, processing, communication, sensor-failure, and connectivity constraints.Existing frameworks commonly assume fresh data at every scheduling slot, making realistic performance assessments overly optimistic.
  • Problem formulation: The problem is formulated under an average sensing rate constraint capturing the trade-off between sensing cost and beam prediction quality.The formulation can be extended to per-modality sensing costs for modality selection.
  • Age-aware fusion: AoI is encoded as a synthetic modality and fused with CNN image features through a gating mechanism to provide input-reliability context.The age signal is intended to inform the predictor about the freshness of the retrieved image.
  • Knowledge distillation: Knowledge distillation transfers knowledge from an unconstrained teacher to a budget-constrained student as a bridge between training and inference conditions.The study finds teacher training regime more consequential than the specific distillation loss function.
  • Sampling policies: Three fixed sampling policies—accumulated, uniform, and randomized—are formulated to satisfy the sensing budget.Their frame-selection behavior is characterized as a function of αmax.
  • Evaluation: At αmax = 0.1, AoI fusion nearly doubles top-1 accuracy relative to an image-only baseline, while age-aware KD offers the best accuracy-complexity trade-off.The evaluation uses the DeepSense 6G dataset.

II. Related Work

Prior work reduces beam-management overhead with environmental sensing and knowledge distillation but generally assumes fresh sensing data. This paper distinguishes itself by modeling freshness explicitly, using AoI as a learnable predictor input, and studying constrained teacher–student deployment.

  • Sensing-aided beam prediction: Position, vision, and multimodal sensing methods have demonstrated accurate beam prediction while reducing codebook search or training overhead.RGB cameras are described as low-cost and information-rich sensing sources.
  • Freshness gap: Most prior sensing-aided approaches assume sensing data is always available and fresh at inference, without modeling sensing constraints or freshness variability.The paper identifies this as a gap in existing evaluations.
  • Knowledge distillation: Prior knowledge-distillation work compresses sensing-aided beam predictors, including reductions exceeding 27 times while retaining 96% top-5 accuracy in one reported study.Other work transfers multimodal knowledge to radar-only students or operates without original training data.
  • Age of information: AoI is treated here as a per-sample learning signal rather than solely as a system-level scheduling objective.The model conditions predictions on freshness to cope with high AoI from packet losses or sensor failure.
  • Distinction from prior work: Compared with the authors’ earlier work, the present framework uses architectural age modulation, finite non-consecutive histories, and an age-aware KD predictor with a 180-fold parameter reduction.Earlier work used dataset augmentation or only the most recent sample and did not address model complexity.
  • Distillation role: KD is framed as a regularizer against distribution shift between unconstrained teacher training and budget-constrained student inference, rather than compression alone.The paper systematically studies both distillation loss and teacher training regime.

A. Problem Formulation

The formulation maps stale retrieved observations and their ages to beam selections while constraining the average sensing rate. Sensing and prediction are coupled, but the paper fixes the sampling policy to enable predictor design under three budget-satisfying families.

  • Beam prediction: The beam predictor maps the most recently retrieved observation X(t−δ(t)) and its age δ(t) to a codebook-based beam index m(t).The predicted beam depends on age both directly and through the retrieved sample.
  • Objective: The per-slot loss compares the predicted beam with the offline ground-truth optimal beam label using a loss such as cross-entropy.The loss acts as a proxy for communication performance because beam misalignment degrades SNR and achievable rate.
  • Time model: A time slot denotes a scheduling opportunity for sensing and beam decisions, distinct from the numerology-fixed 3GPP NR physical-layer slot.The paper relates accumulated and uniform sampling to periodic beam-management cadences.
  • Sensing constraint: The finite-horizon optimization constrains the fraction of slots in which sensing occurs through αmax ∈(0, 1].The normalized budget can represent aggregate power, bandwidth, or processing limitations, with expectations covering randomized sensing decisions.
  • Coupled problem: Sensing decisions and beam prediction are coupled because sampling determines the observations available to future predictions.This coupling creates a difficult sequential decision-making problem.
  • Policy simplification: The paper fixes sampling to accumulated, uniform, or randomized budget-satisfying policies, thereby breaking the coupling and focusing on predictor design.Accumulated and uniform policies represent periodic cadences, whereas randomized sampling reflects irregular availability.

IV. Age-Aware Beam Prediction Framework

The framework uses a finite history of previously acquired, potentially non-consecutive sensory samples because fresh data is unavailable at every slot. It explicitly encodes each sample’s age and fuses that context with visual features for beam prediction.

  • Framework: Fresh sensory data is not available at every slot under the sensing constraint, so prediction uses a finite history of acquired samples.The history can contain heterogeneous modalities, although this work instantiates it with RGB images.
  • Age-aware processing: Buffered samples are ordered from most recent to oldest with non-decreasing age, and W is the GRU sequence length.The ordering organizes the acquired observations for temporal processing.
  • Framework: The history buffer contains the W most recent sensing slots, which are generally non-consecutive.The samples are ordered by acquisition time, with t1 > t2 > · · · > tW and tw ≤ t.
  • Age-aware processing: Each retrieved sample’s age is encoded and fused with its visual features as a synthetic modality.This gives the predictor direct context about the reliability of its inputs rather than treating staleness only as degradation.

A. Sensing Data Sampling Algorithms

The paper evaluates three fixed, signal-agnostic sampling policies that satisfy an average sensing budget by construction. They differ in whether sensing is accumulated deterministically, placed on a fixed grid, or randomized independently across slots.

  • Policy design: Three fixed, signal-agnostic policies—accumulated, uniform, and random sampling—are used to satisfy the average sensing budget.Their sensing decisions do not depend on sensed-data content or modality, isolating the learning benefit of age awareness.
  • Accumulated Sampling: Accumulated sampling uses a deterministic fractional accumulator that meets αmax exactly over any finite horizon T.The construction updates an accumulator and resets it whenever sensing occurs.
  • Uniform Sampling: Uniform sampling senses once every N := ⌈1/αmax⌉ slots under the average budget.For αmax ∈(0.5, 1), N = 2, so extra budget produces no additional observations or performance gain.
  • Random Sampling: Random sampling independently draws each sensing decision using a Bernoulli scheme with mean αmax.Its time-average rate converges to αmax as T grows large, while short-horizon realizations may deviate from the budget.

B. Data Preprocessing, Age Encoding, and Fusion

The predictor preprocesses images, embeds age into the visual feature dimension, and combines both embeddings through adaptive gating before temporal aggregation and classification. A compact CNN replaces the heavier ResNet18 extractor, with knowledge distillation used to address the resulting capacity risk.

  • Data Preprocessing: RGB images are resized, converted to tensors, scaled to [0, 1], and normalized with ImageNet statistics before feature extraction.This aligns inputs with pretrained backbone distributions and stabilizes optimization.
  • Age Encoding: The scalar AoI is locally available at zero sensing cost and acts as a contextual filter for visual-input reliability.It carries no environmental information of its own but informs the predictor about image freshness.
  • Age Encoding: The age encoder projects scalar δ(t) into a D-dimensional embedding matching the image representation.Trainable parameters and element-wise sigmoid activation form the age embedding, while independent normalization mitigates scale disparity.
  • Age Fusion: A learnable gating mechanism adaptively combines image and age embeddings before temporal aggregation.The gate produces dimension-wise weights from both embeddings, allowing reliance on visual content versus age context to vary by feature dimension.
  • Model Architecture: The gating formulation is agnostic to the specific modality and can extend to LiDAR or radar feature representations.The instantiated system uses RGB image features.
  • Model Architecture: The compact CNN student uses around 60K trainable parameters and replaces the heavy ResNet feature-extraction block.The compact design addresses deployment complexity, but reduced capacity risks accuracy loss under sparse and stale inputs; KD uses the ResNet18 model as teacher.
  • Temporal Aggregation: The fused history is processed by a GRU and classifier, while W = 1 removes sequential aggregation and the GRU parameters.The final GRU state is mapped to beam logits and the predicted beam distribution.

V. Knowledge Distillation Framework

The KD framework transfers knowledge from a teacher trained on fully sampled, temporally dense data to a student that trains and operates under the sensing budget. This addresses sparse, stale student inputs while distinguishing the teacher’s training regime from ordinary compression-oriented KD.

  • KD Setting: Standard KD compresses a larger teacher into a compact student, but this framework uses KD under different teacher and student sensing conditions.The teacher has fully sampled training data, whereas the student trains and operates under the sensing budget.
  • Motivation: Without distillation, low-budget student training is dominated by high-AoI samples, limiting the inter-class structure needed to generalize across budgets.Training on fully sampled data followed by constrained deployment instead creates a training–inference mismatch and degrades performance.
  • Teacher Training: The teacher is evaluated under full-sampling and limited-sampling training regimes, pairing labels with either current or most-recent available samples.Full sampling uses α(t) = 1, while limited sampling follows the sensing budget and policy.
  • Teacher Training: The optimal teacher training regime differs from that of a standalone predictor.Limited-sampling training aligns a standalone predictor with constrained inference, whereas a distillation teacher benefits from the distinct fully sampled regime described in the paper.
  • KD Objective: Student training combines the task loss against ground-truth beam labels with a distillation loss against teacher outputs.These terms form a single KD loss for the student.

B. Task Loss

The beam predictor uses cross-entropy as its base task objective, while student training adds a distillation term weighted against task loss. KL and MSE provide alternative ways to align student and teacher outputs.

  • Task loss: Cross-entropy trains the teacher and non-distilled student to predict the optimal beam index from model logits.The task loss is averaged over minibatches of consecutive slots.
  • Distillation objective: The KD objective combines task loss with a distillation loss controlled by γ, balancing ground-truth supervision and teacher guidance.The same formulation supports the student’s combined training objective.
  • KL divergence: KL distillation matches temperature-softened teacher and student distributions, using τ > 1 to expose relative class similarities.A τ^2 factor compensates for the reduced gradients caused by distribution softening, while KL training requires tuning τ and γ.
  • Logit matching: MSE distillation matches the teacher’s and student’s raw logits without requiring a temperature parameter.This reduces the distillation hyperparameter search to γ alone and captures inter-class structure through logit magnitudes.

VI. Training Algorithms

The training procedure preserves temporal ordering, applies budgeted sampling across all data splits, and uses early stopping based on budget-consistent validation. Training mode determines whether the sensing budget is enforced during model training.

  • Dataset construction: Chronological partitioning assigns earlier samples to training, later samples to validation, and the remainder to testing.This keeps each history buffer dependent only on past observations, preserving the meaning of the sampling constraint.
  • Age-tagged sequences: Budgeted sampling retrieves the most recent available image and pairs it with its AoI at every unsampled slot across training, validation, and testing.The same inference-style construction is applied throughout evaluation splits.
  • Training modes: FS training always uses fresh data, whereas LS training draws samples from the budgeted policy and computes their AoI values.The two modes therefore differ in whether the sensing budget is enforced during training.
  • Optimization and validation: The optimizer minimizes task loss with Adam and retains the checkpoint with the lowest validation loss.Validation always uses the sensing budget, with early stopping after K epochs without sufficient improvement.

B. Training With Distillation

Knowledge distillation trains a compact student under the sensing budget using a frozen teacher’s outputs as additional targets. The procedure supports KL or MSE distillation while retaining validation and early stopping.

  • Student training: The student always trains under the sensing budget, regardless of whether its teacher was trained with fresh or budgeted sensing.This separates the student’s constrained training regime from the teacher’s regime.
  • Teacher-student processing: A frozen teacher and the student process the same age-tagged input tuple, and teacher logits provide the distillation target.The student is updated using the combined KD objective.
  • Distillation variants: The KD algorithm supports KL or MSE distillation, with temperature τ used only for the KL variant.The loss type and associated hyperparameters are configured before training.
  • Validation and stopping: Student training evaluates validation loss under the sensing budget and retains the best checkpoint with early stopping.The procedure mirrors the budget-consistent validation logic used for non-distilled models.

VII. Numerical Results and Discussions

Experiments on DeepSense 6G Scenario 9 evaluate fusion, sampling, and history-window choices under constrained sensing. Age-aware fusion is strongest at low budgets, accumulated sampling performs best, and shorter history is favored.

  • Experimental setup: 20 Monte Carlo runs evaluate Scenario 9 with a 32-beam codebook, using common random numbers for fair budget comparisons.The experiments examine age fusion, sampling algorithms, and history-window size.
  • Age fusion: At αmax = 0.1, multiply and gating fusion reach approximately 47% Top-1 accuracy versus 22% without age.This is nearly a twofold improvement at the most restricted sensing budget.
  • Age fusion: At αmax = 0.2, age-aware models reach approximately 95% near-upper-bound Top-3 accuracy, while the baseline reaches that level only at αmax = 0.5.Age-aware models retain their advantage mainly in the low-budget regime.
  • Sampling policies: Accumulated sampling consistently outperforms uniform and randomized sampling, with the largest gap at low αmax.The policies converge as αmax approaches one, while uniform sampling gains nothing beyond αmax > 0.5 because it selects the same frame set.
  • Training–inference mismatch: FS-trained models reach approximately 12% Top-1 accuracy at αmax = 0.1, reflecting their mismatch with sparse, stale inference inputs.Age-fused models instead gain approximately 15–20 percentage points over image-only configurations at low budgets.
  • History window size: For age-fused models, W = 1 outperforms W = 2 and W = 3 across budgets, especially when αmax is low.Longer histories can add stale frames whose gaps are too large or variable to provide useful temporal context.

B. Impact of Knowledge Distillation

Knowledge distillation improves compact-student beam prediction under sensing constraints, with age-aware distillation performing best and the teacher’s training regime mattering more than the loss choice.

  • At low budgets, KL loss with γ = 0.1 outperforms γ = 0.9, but the ordering reverses at high budgets.The result indicates a larger task-loss weight is favored under severe data scarcity, while richer teacher soft-label information becomes more useful at higher budgets.
  • The teacher’s training regime dominates the distillation-loss choice: full-sampling teachers produce stronger students than limited-sampling teachers across budgets and Top-1 and Top-3 metrics.The gap widens at low αmax, despite limited-sampling teachers performing better under constrained standalone inference.
  • KD consistently improves accuracy over no-KD baselines across the full sensing-budget range.The comparison includes distilled and non-distilled students with and without age fusion.
  • Age-KD achieves the highest Top-1 accuracy throughout, while FS-Img-KD trails marginally and both outperform LS-Img-KD.These configurations differ in age fusion and teacher training regime.
  • Without KD, the age-fused student stalls near 0.47 Top-1 at αmax = 1, whereas KD enables the model to exploit age and image inputs across budgets.The no-KD age student is competitive at αmax ≤ 0.3 but plateaus as more visual data becomes available.
  • Training the no-age student exclusively on fully sampled data yields approximately 0.12 Top-1 at αmax = 0.1, while distilled variants avoid this low-budget collapse.The distilled teacher transfers abstract beam-class relationships that improve robustness to sparse inference inputs.
Loading 2609.01225v1…