PaperlayerView exact revision on arXiv

Source-linked AI summary

RAST: Resolution-Aware Privileged Structure Transfer for Low-Resolution Audio Activity Recognition

Ji Hwan Park, Gautham Krishna Gudur, Yufei Shen, Dawei Liang, Edison Thomaz

arXiv:2609.38780v1cs.SDcs.AI

TL;DR

Audio HAR faces sensing-cost and privacy constraints, while reducing sampling rate can remove activity-relevant cues. RAST transfers compact HR teacher structure during training to an LR student; it outperforms LR-only and direct-transfer baselines across SAMoSA and AudioIMU while using only LR audio at inference.

  • Problem

    Audio sensing entails resource costs and privacy concerns, while lower sampling rates can remove cues needed to distinguish activities.

    Source: §1 · pp. 1–2

  • Method

    RAST distills HR teacher representations into tokens and neighborhood structure, then uses local HR–LR alignment to train a student that deploys with LR audio only.

    Source: §1 · p. 2, §3.2 · p. 3

  • Results

    Across SAMoSA and AudioIMU, RAST consistently outperforms LR-only and direct teacher-transfer baselines; AudioIMU macro-F1 gains 7.8% over LR-only training.

    Source: §5.1 · p. 5

  • Takeaways & Limitations

    RAST improves LR activity recognition while preserving an exclusively low-resolution sensing path at inference.

    Source: §1 · p. 2, §6 · p. 5

    Broader evaluation of privacy and real-world deployment tradeoffs remains future work.

    Source: §6 · p. 5

Abstract

from arXiv · show

Audio is increasingly used for human activity recognition (HAR) because it captures object interactions, environmental events, and contextual cues in everyday environments. High-resolution (HR) audio provides rich acoustic information for model development but incurs substantial energy and storage costs and may expose sensitive speech content. Low-resolution (LR) audio offers a more privacy-preserving and resource-efficient alternative for deployment, but reduced sampling rates can remove acoustic cues essential for activity recognition, leading to significant performance degradation. We formulate this training-deployment mismatch as sensor-resolution privileged learning, in which HR audio is available during training, while inference relies exclusively on LR audio. We propose RAST, a resolution-aware transfer framework that compresses HR teacher representations by preserving token-level information and neighborhood structure before performing localized HR-LR alignment. Experiments on the SAMoSA and AudioIMU datasets show that RAST consistently outperforms LR-only training and direct teacher-transfer baselines, improving LR-only recognition by up to approximately 7.8% while requiring only LR audio at inference.

1 Introduction

RAST addresses the mismatch between information-rich HR training audio and resource- and privacy-aware LR deployment by transferring compact HR structure to an LR student. It improves LR-only recognition on both evaluated datasets while keeping inference LR-only.

1 Introduction

  • Audio sensing can capture behavioral context, but continuous sensing entails bandwidth, storage, computation, and energy costs and raises privacy concerns about unintended speech capture.These constraints motivate the search for sensing approaches that balance recognition with resource use and sensitive-data exposure.Source: §1 · p. 1
  • Lower sampling rates can remove activity-relevant acoustic cues, while held-out recognition accuracy falls as the rate diverges from the 16-kHz HR reference.HR audio preserves high-frequency and transient structure associated with activities including chopping, brushing, clapping, and pouring.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • The paper frames the setting as privileged learning: HR audio is available during training, but inference uses only LR audio.The stated objective is to transfer discriminative HR information without requiring HR sensing at deployment.Source: §1 · p. 2
  • RAST transfers HR-derived tokens and teacher-neighborhood structure, then uses local hard-negative alignment rather than requiring the LR student to reproduce the full HR embedding space.The transferred structure is intended to avoid fine-grained information that may be unobservable after resolution reduction.Source: §1 · p. 2
  • RAST improves balanced accuracy and macro-F1 over LR-only training by 6.7% and 7.5% on SAMoSA, and 6.1% and 7.8% on AudioIMU.The reported gains are relative improvements.Source: §1 · p. 2

2 Related Work

Related work spans distillation, privileged-information learning, cross-modal transfer, compact representations, and deployment-aware sensing. RAST distinguishes its setting by addressing the limited observability of HR evidence in LR audio.

2 Related Work

  • HR-to-LR transfer differs from conventional teacher-student transfer because the LR student sees a degraded view of the same event, making some HR evidence absent or unrecoverable.Directly matching HR predictions or representations may therefore impose targets that LR input cannot support.Source: §2 · p. 2
  • Prior work addresses deployment constraints and privacy through mobile acoustic sensing, low-resolution or bandwidth-restricted audio, and self-distillation for 1–2 kHz microphone inputs [15] [3] [17].RAST uses HR audio as training-only supervision and requires LR audio at inference.Source: §2 · p. 2
  • RAST adapts compact-representation and intermediate-structure transfer to resolution mismatch by using HR-derived tokens and local teacher-neighborhood structure instead of reproducing the full HR embedding space.Related approaches include information bottlenecks, discrete latent codes, and hint-based, relational, and contrastive distillation [1] [30] [31] [22] [25] [29].Source: §2 · p. 3

3 Methodology

RAST frames low-resolution audio activity recognition as privileged learning: paired HR and LR views support training, but deployment uses only LR. It transfers compact token and neighborhood structure before local HR–LR alignment.

3.1 Problem Formulation

  • The formulation pairs HR and LR views of each labeled audio segment, with HR available during training and LR alone observed at inference.The deployed model must predict the activity label from the LR input.Source: §3.1 · p. 3

3.2 Framework Overview

  • RAST trains an HR teacher, transfers compact structure to an LR student, refines it through local alignment, then discards the teacher and auxiliary targets for LR-only deployment.The framework transfers bottlenecked HR-derived structure rather than requiring the student to reproduce the full HR embedding space.Source: §3.2 · p. 3

3.3 Privileged Structure Extraction

  • HR teacher representations are clustered into discrete tokens, while mini-batch similarities define teacher-neighborhood distributions that preserve local geometry.The tokens are auxiliary training targets; neighborhood structure adds local geometry that tokens alone do not preserve.Source: §3.3 · p. 3

3.4 Student Optimization and Alignment

  • The LR student is warm-started with label, token, and teacher-neighborhood supervision, then locally aligned to HR embeddings using hard-negative mining.HNM focuses alignment on locally confusable examples rather than matching the full HR manifold uniformly.Source: §3.4 · p. 4

4 Experiments

The experiments evaluate low-resolution audio recognition on two wearable HAR datasets using subject-independent protocols. They compare RAST with LR-only and HR-to-LR transfer baselines under shared encoder and held-out-fold conditions.

4.1 Datasets and Sensing Setup

  • AudioIMU includes 15 participants and 23 activities, while SAMoSA includes 20 participants; both use wearable audio and IMU recordings, with LR input simulated at 1.25 kHz from 16 kHz HR audio.Audio was segmented into aligned five-second windows, and only LR audio was available at test time.Source: §4.1 · p. 4

4.2 Evaluation Protocol

  • Evaluation uses subject-independent held-out participants or groups and reports mean ± standard deviation over five seeds for balanced accuracy and macro-F1.SAMoSA uses leave-one-group-out validation over six groups; AudioIMU uses leave-one-participant-out validation for participants 01–15.Source: §4.2 · p. 4

4.3 Methods and Implementation

  • All methods share a PANNs-based encoder and identical held-out folds; baselines test logit distillation, instance- or class-level alignment, and direct hard-negative alignment against RAST.RAST adds its token and teacher-neighborhood warm start to hard-negative mining.Source: §4.3 · p. 4

5 Results

RAST outperforms LR-only and direct-transfer baselines on both datasets; ablations support token-and-neighborhood warm starts, while embedding analysis shows improved separation among locally confusable activities.

5.1 Discussion on Overall Results

  • On SAMoSA, RAST raises BA 0.6123→0.6532 and macro-F1 0.603→0.6482; on AudioIMU, BA rises 0.5157→0.5474 and macro-F1 0.4933→0.532, beating direct-transfer baselines.These are the reported LR inference results against LR-only training; RAST is best on both datasets.Source: §5.1 · p. 5

5.2 Ablation Study

  • With CE-only initialization, HNM lifts BA 0.6123→0.6218; HR-token supervision gives warm-start BA 0.6218 and post-HNM BA 0.6443.Table 2 reports component-level SAMoSA performance before and after HNM.Source: §5.2 · p. 5, Table 2 · p. 5
  • Token-plus-neighborhood supervision gives the strongest warm start (0.6308 BA, 0.6182 macro-F1) and final result (0.6532 BA, 0.6482 macro-F1).Neighborhood supervision alone yields a moderate warm-start score, with post-HNM performance comparable to token supervision; tokens provide anchors and neighborhood regularization structures local geometry for HNM.Source: §5.2 · p. 5, Table 2 · p. 5

5.3 Embedding Structure Analysis

  • RAST embeddings separate locally confusable activities better than LR-only representations, which are more entangled for activities relying on fine cues attenuated by downsampling.The figure visualizes intermediate embeddings; the reported interpretation links the improved separation to the structured warm start.Source: §5.3 · p. 5, Figure 3 · p. 5

6 Conclusion

RAST transfers HR-derived token and neighborhood structure into an LR model before local HR–LR alignment, and consistently outperforms LR-only and direct transfer on SAMoSA and AudioIMU under subject-independent evaluation. The authors identify broader evaluation of privacy and real-world deployment tradeoffs as future work.

  • RAST compresses HR teacher representations into token- and neighborhood-level supervision before local HR–LR alignment.Source: §6 · p. 5
  • Across SAMoSA and AudioIMU, RAST consistently outperforms LR-only training and direct teacher-student transfer under subject-independent evaluation.Source: §6 · p. 5
  • Future work includes broader evaluation of privacy and real-world deployment tradeoffs, as well as student-aware teachers, adaptive bottlenecks, and LR-recoverable structure-preserving objectives.Source: §6 · p. 5
Loading 2609.38780v1…