Source-linked AI summary
Sample-Conditioned Representation Selection for Audio Few-Shot Learning
Fengrui Liu, Ningxin Shen, Yi Li, Yiwei Fu, Feng Liu, Jiangmeng Li
TL;DR
Few-shot audio classifiers can exploit foreground–background co-occurrences, leaving feature reliability uncertain when those associations shift. SAMPLESELECT selects a fixed-budget mask per input with the source encoder and classifier frozen. On SpurAudio, it leads the compared methods in OOD accuracy across four settings and improves on matched FULLREP.
Problem
When foreground–background associations shift, few-shot classifiers may lose reliable features, and existing adaptation methods do not directly target input-varying background sensitivity.
Source: §1 · p. 1Method
SAMPLESELECT learns a fixed-budget mask independently for each input while keeping the source encoder and classifier frozen, using classification and cross-background contrastive supervision.
Source: §1 · p. 1, §2.2 · p. 2Results
Across ResNet12 and Conv64 in 5-way 1-shot and 5-shot settings, SAMPLESELECT has highest OOD accuracy and improves FULLREP by 5.982, 4.902, 6.603, and 8.375 percentage points.
Source: §3.2 · p. 3, §4 · p. 4Takeaways & Limitations
Sample-conditioned selection improves the use of frozen representations under class–background co-occurrence shift.
Source: §3.2 · p. 3The shift diagnosis is not causal because IID and OOD routes contain different foreground recordings; test-time masking and adaptation use no query labels or background labels.
Source: §2.3 · p. 3, §3.4.1 · p. 4
Abstract
from arXiv · showhide
Few-shot audio classifiers may rely on foreground-background co-occurrences and fail when those correlations shift. On SpurAudio, the resulting representation shift is concentrated and class dependent: for ResNet12, the top 10 percent of channels explain 82.80 percent of the null-corrected shift contribution. We propose SAMPLESELECT, which predicts a fixed-budget feature mask independently for each input while keeping the encoder and source classifier frozen. Training uses differentiable Gumbel Top-k selection with foreground classification and cross-background contrastive losses; inference uses deterministic Top-k masks and support-only linear adaptation. Across ResNet12 and Conv64 in 5-way 1-shot and 5-shot evaluation, SAMPLESELECT gives the best OOD accuracy among the compared methods and improves the matched full-representation control by 4.90-8.38 percentage points. Ablations and representation analyses further support the learned selection mechanism. Code is available at https://github.com/Cross-Innovation-Lab/SAMPLESELECT/
1. INTRODUCTION
Background shifts make few-shot classifiers’ reliance on foreground–background co-occurrences risky, while limited support data leaves unclear which representation coordinates remain reliable. SAMPLESELECT selects features per example and improves OOD accuracy across four SpurAudio settings.
1. INTRODUCTION
- Foreground–background co-occurrences can act as shortcuts, and existing adaptation or reweighting methods do not directly target input-varying background sensitivity.The small support set offers little evidence for identifying which coordinates remain trustworthy when those associations change.Source: §1 · p. 1
- For frozen ResNet12, the top 10% of 640 channels account for 82.80% of corrected shift contribution, and affected channels vary substantially across foreground classes.This pattern motivates input-dependent rather than globally fixed feature selection.Source: §1 · p. 1
- SAMPLESELECT predicts fixed-budget masks independently for each input while freezing the encoder and source classifier, then uses deterministic Top-k masks and a fresh support-only episodic head.Unlike FULLREP’s all-coordinate input, it masks support and query examples independently.Source: §1 · p. 1, Figure 1 · p. 1, Figure 2 · p. 2
- SAMPLESELECT achieves the highest OOD accuracy in all four SpurAudio backbone/shot settings, improving matched FULLREP by 4.90–8.38 percentage points.Ablations and representation analyses support learned selection and cross-background supervision.Source: §1 · pp. 1–2
2. METHODOLOGY
The pipeline trains a source representation, freezes the encoder and source head, learns per-example masks, then evaluates with deterministic masks and a support-only episodic head.
2.1. Source Representation Learning
- The encoder and source classifier are trained on source foreground classes and then frozen; ResNet12 uses 640-dimensional descriptors and 12,800-dimensional classifier representations, while Conv64 uses 1,600 dimensions for both.The selectable descriptor and classifier representation are defined separately for ResNet12.Source: §2.1 · p. 2
2.2. Sample-Conditioned Selector Learning
- A scorer assigns each coordinate a per-input score and forms a retention-ratio Top-k mask using differentiable sequential Gumbel-Softmax draws during training.The selector objective combines classification with cross-background contrastive supervision, favoring same-class consistency across observed backgrounds; only the selector is updated.Source: §2.2 · p. 2
2.3. Inductive Few-Shot Evaluation
- At inference, each example receives an independent deterministic mask, and a fresh linear head is fitted only on masked support representations before predicting masked queries.No query label, query-set statistic, or test-time background label is used for masking or adaptation; FULLREP uses the same support-only procedure without selection.Source: §2.3 · pp. 2–3
- Table 1 reports percentage accuracy on SpurAudio for ResNet12 and Conv64, with Δ defined as IID minus OOD and FULLREP as the matched no-selection control.The table caption identifies bold values as the best IID or OOD accuracy among compared methods.Source: Table 1 · p. 3
3. EXPERIMENTS
SAMPLESELECT is evaluated on SpurAudio under 5-way 1-shot and 5-shot protocols across two backbones. The experiments compare its OOD accuracy with other methods and examine ablations, representation shifts, and selector scores.
3.1. Experimental Setup
- Evaluation uses SpurAudio 5-way 1-shot and 5-shot episodes with 10 queries per class, averaging over 1,000 episodes.The study compares ResNet12 and Conv64; FULLREP uses the same frozen backbone and episodic classifier without representation selection.Source: §3.1 · p. 3
3.2. Main Results
- SAMPLESELECT has the highest OOD accuracy across four backbone/shot settings, beating FULLREP by 5.982, 4.902, 6.603, and 8.375 percentage points, respectively.The deltas are reported in order for ResNet12 1-shot and 5-shot, then Conv64 1-shot and 5-shot.Source: §3.2 · p. 3
3.3. Ablation and Sensitivity Analysis
- Removing random-soft selection, Gumbel relaxation, or cross-background contrastive supervision lowers OOD accuracy; tested positive λcon values beat zero, while r from 0.4 to 0.9 changes performance modestly.The figure also compares component-removal drops and OOD gains over FULLREP across retention ratios and contrastive weights.Source: §3.3 · p. 3, Figure 3 · p. 4
3.4.1. Concentrated and Class-Dependent Shift
- The top 10%, 20%, and 30% of channels account for 82.80%, 92.85%, and 96.76% of positive corrected shift mass, and affected subsets vary across foreground classes.The shift statistic is corrected using a within-class permutation null; the authors characterize this analysis as diagnostic, not causal identification.Source: §3.4.1 · pp. 3–4, Figure 4 · p. 4
3.4.2. Shift and OOD Degradation
- All 18 Spearman correlations across seeds, shot settings, and measures are positive, with ρ between 0.268 and 0.455, associating larger representation shifts with larger predictive degradation.The measures are accuracy, predicted probability, and classification margin across 1,000 episodes.Source: §3.4.2 · p. 4
3.4.3. Selector Scores Reflect Functional Importance
- Blocking the highest-scored 30% of channels lowers accuracy by 3.033 points on average, versus 1.064 points for random blocking; blocking the lowest-scored channels has almost no effect.The episodic classifier is held fixed during the blocking comparison.Source: §3.4.3 · p. 4, Figure 4 · p. 4
4. CONCLUSION
The paper presents SAMPLESELECT, a sample-conditioned fixed-budget selector for few-shot audio classification under class–background co-occurrence shift. On SpurAudio it achieves the best OOD accuracy in all four backbone/shot settings, with analyses relating channel shifts and selector scores to predictive behavior.
4. CONCLUSION
- SAMPLESELECT learns per-example feature scores with the source encoder frozen, using source-class discrimination and cross-background contrastive supervision, then applies deterministic Top-k selection.Few-shot evaluation uses per-example masks and a support-only linear head without query labels, query-set statistics, or test-time background labels.Source: §4 · p. 4
- On SpurAudio, SAMPLESELECT achieves the best OOD accuracy in all four backbone/shot settings and improves on FULLREP by 5.982, 4.902, 6.603, and 8.375 percentage points.The values correspond in order to ResNet12 1-shot and 5-shot, then Conv64 1-shot and 5-shot.Source: §4 · p. 4
- Ablations and further analyses support the gains’ dependence on learned selection and cross-background supervision, alongside concentrated, class-dependent shifts associated with predictive degradation.The analyses also find selector scores consistent with differences in channels’ functional importance.Source: §4 · p. 4