PaperlayerView exact revision on arXiv

Source-linked AI summary

Exemplar-Free Analytic Learning for Multi-Label Audio Class-Incremental Learning

Siyuan Luo, Yang Xiao, Ting Dang

arXiv:2609.29777v1eess.AS

TL;DR

Multi-label audio class-incremental learning must accommodate new sound classes even though each phase lacks labels for previously learned classes. The paper evaluates closed-form analytic learning and ALMA’s continuous pseudo-labeling and frequency weighting; analytic methods outperform gradient baselines while old-class local mAP stays nearly unchanged.

  • Problem

    Multi-label audio class-incremental learning must add sound classes despite current-phase annotations omitting old classes that may still be present.

    Source: §1 · p. 1, §2.1 · p. 2

  • Method

    The paper evaluates a closed-form analytic learner that preserves old-class weights and ALMA, which adds continuous old-class pseudo-labels and frequency-based sample weighting.

    Source: §1 · p. 1

  • Results

    Analytic learning substantially outperforms gradient-based methods across evaluated setups, and previously learned class groups’ local mAP remains nearly unchanged as classes are added.

    Source: §5 · p. 4

  • Takeaways & Limitations

    The results support analytic learning as an effective, computationally simple direction for multi-label audio class-incremental learning.

    Source: §5 · p. 4

    Training annotations cover only newly introduced classes, with no labels for previously learned classes.

    Source: §2.1 · p. 2

Abstract

from arXiv · show

Audio classification is inherently a multi-label task, as real-world acoustic environments contain multiple simultaneous sound events. When new sound classes emerge, models must incorporate them without forgetting previously learned ones: a challenge known as class-incremental learning. Existing methods rely on storing past data and iterative gradient updates, which struggle under incomplete multi-label supervision because only the newly introduced classes are annotated at each phase, leaving old-class labels unavailable. We investigate exemplar-free analytic continual learning as a principled alternative, in which a linear classifier is updated in closed form without storing historical recordings or performing incremental back-propagation, and previously learned weights remain intact by construction. Building on analytic learning, we further propose ALMA, which addresses incomplete supervision and class imbalance through continuous old-class score estimates and frequency-based sample weighting. Experiments on a 50-class AudioSet-R benchmark across three incremental setups show that the analytic learner substantially outperforms gradient-based methods, and previously learned classes retain nearly unchanged detection performance as new classes are added. This study shows that ALMA is a simple yet effective solution to multi-label audio class-incremental learning.

1. INTRODUCTION

Real-world sound co-occurrence motivates multi-label audio class-incremental learning, but phase-wise incomplete labels make retaining old classes difficult. The paper investigates analytic closed-form learning and introduces ALMA’s continuous pseudo-labels and frequency weighting, reporting improvements over gradient-based baselines.

  • Real-world sound co-occurrence requires multi-label class-incremental learning, but phase-wise labels omit old classes even when those sounds are present.Source: §1 · p. 1
  • Under missing-label supervision, iterative gradients can treat unannotated old events as absent and overwrite prior knowledge; distillation targets may also be unreliable.Source: §1 · p. 1
  • Closed-form analytic updates leave old-class weights unchanged by construction; ALMA adds continuous prior-model scores and rarity-based sample weighting.Source: §1 · p. 1
  • Analytic learning consistently improves on gradient-based baselines, while ALMA’s soft pseudo-labels and frequency weighting further address unannotated old classes.Source: §1 · p. 1

2. ALMA FOR AUDIO MULTI-LABEL CIL

Multi-label audio CIL provides labels only for newly introduced classes, leaving annotations for overlapping old sounds unavailable. ALMA combines closed-form analytic updates with soft old-class pseudo-labels and frequency-based sample weighting.

2.1. Problem Setting and Notation

  • Each phase labels only its newly introduced classes, although recordings may also contain previously learned sounds.The completed target matrix combines current binary labels with estimates for old classes.Source: §2.1 · p. 2

2.2. Analytic Learning

  • A frozen encoder supplies features to a linear classifier fitted by closed-form ridge regression, avoiding incremental back-propagation and historical recordings.The recursive solution is mathematically equivalent to joint retraining across seen phases.Source: §2.2 · p. 2

2.3. Proposed ALMA

  • ALMA addresses multi-label missing annotations with continuous old-class pseudo-labels and frequency-based sample weighting.The previous-phase classifier estimates old-class scores for current recordings rather than treating unobserved labels as zeros.Source: §2.3 · p. 2, §2.3.1 · p. 3

2.3.1. Continuous Pseudo-Labels

  • Old-class scores from the preceding classifier are clipped to [0, 1] and retained as continuous targets alongside current binary labels.Keeping scores continuous preserves uncertainty about which old sound classes are present.Source: §2.3.1 · p. 3

2.3.2. Frequency-Based Sample Weighting

  • Each recording is weighted by the average rarity of its positive classes, including supplied labels and thresholded old-class pseudo-labels.Less frequent classes receive larger rarity factors, normalized to mean one across the seen classes.Source: §2.3.2 · p. 3

2.3.3. Recursive Analytic Update

  • The recursive analytic update retains prior target-feature statistics and adds current-phase contributions, recovering the classifier without storing earlier feature and target matrices.Completed targets and per-recording weights enter the accumulated weighted ridge-regression objective.Source: §2.3.3 · p. 3

3. EVALUATION PROTOCOL

The evaluation uses a 50-class AudioSet-R subset under phase-specific incomplete labels, with a frozen CNN14 encoder and gradient-based comparison methods. Performance is measured using class-wise and cumulative mAP.

3.1. Data and baselines

  • The benchmark uses the 50 most frequent AudioSet-R classes; clips can have multiple positive labels and appear in multiple phase sets.Training exposes only current-group labels, while evaluation covers all classes seen so far.Source: §3.1 · p. 3
  • A CNN14 encoder trained at the base phase is then frozen, providing fixed 4096-dimensional embeddings from 64-bin log-mel spectrograms.The protocol evaluates classifiers on fixed audio features after the base phase.Source: §3.1 · p. 3
  • Comparisons include FT, EWC, SI, LwF, Joint training, and PPR; AP ranks positive clips per class, while mAP averages AP across classes.Cumulative mAP covers classes seen at each phase, Mean cum. averages phase scores, and Final mAP is the last-phase score.Source: §3.1 · p. 3

4. RESULTS

Analytic methods outperform gradient-based baselines, while ALMA retains old-class performance as new classes are added and remains strong across increasingly challenging protocols. Ablations attribute additional gains mainly to old-class target construction, not sample weighting.

4.1. Overall Incremental Performance

  • Analytic learning accumulates prior phases in one closed-form solution; ALMA remains 0.80 and 1.82 points below PPR and overtakes FT and LwF after the first update.ALMA starts below FT and LwF at the base phase; its advantage emerges after the first incremental update.Source: §4.1 · p. 3, Figure 2 · p. 4
  • 43.32% mean cumulative mAP and 39.08% final mAP for ALMA exceed Analytic CIL’s 42.73% and 38.43%, while LwF reaches 37.95% and 26.98%.The comparison is reported in Table 1.Source: §4.1 · p. 3, Table 1 · p. 4

4.2. New-Class Learning and Old-Class Retention

  • P0 maintains approximately 47.8% local mAP and P1 approximately 33.3% through subsequent phases, showing old-group performance remains stable.Figure 3 tracks each class group across evaluation phases on a fixed test pool.Source: Figure 3 · p. 4, §4.2 · p. 4

4.3. Performance Across Incremental Protocols

  • Across Setups A–C, ALMA final mAP is 39.08%, 37.47%, and 34.50%, beating LwF by 12.10, 17.89, and 18.13 points.The setups use progressively longer incremental sequences.Source: Figure 4 · p. 4, §4.3 · p. 4

4.4. Ablation Study

  • Hard pseudo-labels add 0.29 mean cumulative and 0.28 final mAP points over zero targets; continuous scores add 0.27 and 0.40 over hard labels; weighting changes them +0.03 and −0.03.The sample-weighting changes are for mean cumulative and final mAP, respectively.Source: Table 2 · p. 4, §4.4 · p. 4

5. CONCLUSION

The analytic learner improves multi-label audio CIL without historical recordings or incremental back-propagation. ALMA uses continuous old-class score estimates, and previously learned groups retain nearly unchanged local mAP as new classes are added.

5. CONCLUSION

  • The analytic learner operates without storing historical recordings or performing incremental back-propagation.The conclusion presents analytic continual learning as a computationally simple direction for multi-label audio CIL.Source: §5 · p. 4
  • Across all evaluated setups, analytic learning substantially outperforms gradient-based methods, with the closed-form recursive update identified as the main source of improvement.ALMA further improves performance by replacing zero-filled targets with continuous old-class score estimates.Source: §5 · p. 4
  • Previously learned class groups retain nearly unchanged local mAP as new classes are added, indicating strong resistance to catastrophic forgetting.The conclusion describes analytic learning as an effective direction for multi-label audio CIL.Source: §5 · p. 4
Loading 2609.29777v1…