PaperlayerView exact revision on arXiv

Source-linked AI summary

Full Collapse Is Suboptimal: Controlled Partial Neural Collapse for Long-Tailed Recognition

Hyeontaek Hwang, Cheonghwan Hur, Daeyoung Kim

arXiv:2610.36460v1cs.CVcs.AI

TL;DR

Long-tailed recognition methods that impose full neural collapse can over-contract tail classes and remove within-class variability. This paper coordinates CASA and ETF-AM to control collapse dimensionality and class-specific clearance, retaining tail diversity while head classes collapse along an ETF. It reports the best overall accuracy over prior methods on four long-tailed benchmarks.

  • Problem

    Full neural collapse can over-contract tail classes under imbalance, removing within-class variance that tail generalization needs.

    Source: §1 · p. 1

  • Method

    The method coordinates CASA’s feature-conditioned node gating with ETF-AM’s adaptive margins to control collapse dimensionality and clearance, allowing tail classes to retain within-class diversity.

    Source: §1 · p. 1

  • Results

    The method achieves the best overall accuracy over prior methods on CIFAR100-LT, ImageNet-LT, Places-LT, and iNaturalist-2018.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    Measured spreads are consistent with the paper’s prediction that optimal within-class spread is ordered by class frequency.

    Source: §6 · p. 8

    The claim is verified for long-tailed image classification; transfer to long-tailed detection or segmentation remains open.

    Source: §5 · p. 8

Abstract

from arXiv · show

Recent long-tailed recognition (LTR) methods incorporate neural collapse (NC) and align features to a fixed Equiangular Tight Frame (ETF) as a uniform target for all classes. However, forcing tail classes to fully collapse over-contracts their samples and weakens generalization. To address this, we propose a controlled partial neural collapse, realized through two coordinated controls on a backbone: the Class-Adaptive Sparse Activation (CASA) classifier and the ETF-constrained adaptive margin (ETF-AM) loss. Rather than forcing a uniform full collapse, this regime allows head classes to collapse along the ETF while tail classes retain within-class diversity. This prevents the tail from being forced into full collapse, which is suboptimal under imbalance. To set the dimensionality of collapse, CASA induces class-related node specialization through feature-conditioned gating so that tail classes occupy their own feature directions. For the clearance axis, ETF-AM applies a frequency-adaptive margin, with smaller margins for head classes and larger for tail classes. On CIFAR100-LT, ImageNet-LT, Places-LT, and iNaturalist-2018, our method achieves the best overall accuracy over prior methods. In the empirical analysis, within-class spread increases monotonically from head to tail classes, consistent with tail classes retaining variability rather than fully collapsing.

1. Introduction

Long-tailed recognition is difficult because head classes dominate training data, while methods that impose full neural collapse can over-contract tail classes and remove variability needed for their generalization. The paper proposes controlled partial collapse: head classes collapse along an ETF while tail classes retain within-class diversity, using CASA and ETF-AM to control collapse dimensionality and clearance.

1. Introduction

  • Long-tailed recognition addresses models’ head-class bias and reduced tail accuracy when head classes have most samples and tail classes only a handful.The passage notes that this limits reliable deployment in safety-critical domains.Source: §1 · p. 1
  • Prior neural-collapse methods target full collapse across classes, but under imbalance this can over-contract tail samples and remove within-class variance needed for their generalization.The paper frames full collapse as optimal only for balanced data and proposes retaining positive tail spread instead.Source: §1 · p. 1
  • Controlled partial collapse lets head classes collapse along the ETF while tail classes retain within-class diversity, balancing within-class collapse and inter-class separation.The paper identifies these as distinct neural-collapse properties, NC1 and NC2, with a trade-off favoring positive tail spread over full collapse.Source: §1 · p. 1
  • CASA uses feature-conditioned node gating to control collapse dimensionality, while ETF-AM fixes the classifier to an ETF and assigns larger margins to tail classes.The adaptive margins widen tail classes’ training-to-inference clearance.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • The method achieves the best overall accuracy over prior methods on CIFAR100-LT, ImageNet-LT, Places-LT, and iNaturalist-2018.The empirical analysis reports that further tail compression degrades tail-class generalization.Source: §1 · p. 2

2. Background and Related Work

Prior long-tailed recognition work uses reweighting, loss and logit adjustments, margins, or neural-collapse geometry. The paper distinguishes its approach by imposing an explicit geometric constraint while relaxing full collapse for tail classes.

2. Background and Related Work

  • Existing long-tailed methods use reweighting, loss design, logit adjustment, and margin adjustment, often with margins in an unconstrained classifier and no explicit final-embedding geometry constraint.The passage lists representative bibliography markers for these method families.Source: §2 · p. 2
  • Neural-collapse methods for long-tailed recognition fix the classifier to an ETF or regularize features toward it, while pursuing full collapse for every class.Neural collapse comprises within-class feature collapse, ETF class means, classifier alignment, and nearest-center decisions.Source: §2 · p. 2
  • Analyses find that imbalance can move optimal geometry away from the balanced ETF, including shared minority directions, class-dependent orthogonal means, or multiple centers for minority classes.These alternatives are reported in [8],, and, respectively.Source: §2 · p. 2
  • When C exceeds d + 1, a simplex ETF is impossible, and prior work instead studies generalized maximal-separation frames; this paper’s analysis acts on within-class spread and applies to any maximal-separation target.The cited related work targets the inter-class frame under full collapse.Source: §2 · p. 2
  • Recent AllNC and Space Alignment methods seek full neural-collapse recovery toward the balanced ETF for every class, whereas this paper argues for relaxing that target for tail classes.The passage also contrasts their heavier training machinery with a single-backbone approach.Source: §2 · p. 2

3. Methodology

The methodology motivates controlled partial collapse with a finite-sample risk model, then implements it through class-related feature subspaces and frequency-dependent clearance. CASA supplies instance-conditioned sparse node selection, while ETF-AM fixes an ETF classifier and adjusts class margins.

3. Methodology

  • In a finite-sample class-conditional model, full collapse is not risk-optimal for classes below a sample-count threshold, and under an additional monotonicity condition scarcer classes have wider risk-optimal spread.The cross-class ordering depends on the stated monotonicity condition over the feasible interval.Source: §3 · p. 3

3.1.1. Detailed Architecture

  • CASA assigns each class a feature subspace through node-level gating, so classes use distinct subspaces around means that share the fixed ETF target.Its gate selects a subset of specialized nodes for each instance, and training thresholds the soft mask while inference uses the soft mask for smoother aggregation.Source: §3.1.1 · pp. 3–4

3.1.2. Instance-Conditioned Gating and Its Class Structure

  • Because the gate is feature-conditioned and optimized with class-wise loss, same-class samples receive correlated gate inputs and empirically develop similar masks.A straight-through estimator lets gradients flow through activated nodes, and a spike-and-slab prior encourages sparse masks while balancing sparsity against signal.Source: §3.1.2 · p. 4

3.2. ETF-Constrained Adaptive Margin Loss

  • ETF-AM combines a fixed ETF classifier with per-class adaptive margins, giving scarcer classes wider training-to-inference clearance and head classes smaller margins.The margin scale uses LDAM-style frequency scaling, while the exponent is a design choice rather than a rate derived from the theorem and corollary.Source: §3.2 · p. 4, §3.2.1 · pp. 4–5

3.2.1. ETF-Constrained Margin Formulation

  • With an ETF-constrained head, subtracting the true-class margin makes the training condition stricter than inference and yields a single per-class clearance independent of the competing class.Because ETF distances are fixed, the network reduces the margin-adjusted loss through the backbone and gate rather than rotating the classifier; the overall loss also includes sparse regularization.Source: §3.2.1 · p. 5

4. Experiments

The experiments evaluate long-tailed recognition using top-1 accuracy and neural-collapse diagnostics across four datasets. The method leads prior approaches on the reported benchmarks, while ablations and spread and mask analyses examine its components and partial-collapse behavior.

4.1.1. Metric

  • Top-1 accuracy and FDR assess recognition and neural-collapse geometry, while per-class spread separately measures within-class relaxation.The evaluation uses Many (> 100 samples), Medium (> 20 and ≤100), and Few (≤20) splits.Source: §4.1.1 · p. 5

4.1.2. Dataset

  • Four long-tailed benchmarks cover downsampled CIFAR100, Pareto-subsampled ImageNet and Places, and naturally long-tailed iNaturalist-2018.The iNaturalist dataset has 8,142 classes, 437K training images, and 24K validation images.Source: §4.1.2 · p. 5

4.1.3. Implementation Details

  • Experiments use dataset-specific ResNet or ResNeXt backbones and balanced test sets, with a learnable classifier for iNaturalist because no simplex ETF exists.Places-LT uses ImageNet-1k pretrained weights, while CIFAR100-LT and ImageNet-LT train from scratch; post-training logit adjustment is applied.Source: §4.1.3 · pp. 5–6

4.2. SOTA Benchmark

  • 54.73% (+1.38%p) on CIFAR100-LT, 56.90% (+1.20%p) on ImageNet-LT, and 41.20% (+0.56%p) on Places-LT were the best overall accuracies over prior approaches.Gains were concentrated in Medium and Few splits while Many-class accuracy was preserved.Source: Table 1 · p. 6, Table 2 · p. 6, Table 3 · p. 6
  • 73.50% overall top-1 accuracy on iNaturalist-2018 outperformed prior methods, with larger within-class spread for tail than head classes.Because no simplex ETF exists for this dataset, the method uses a learnable classifier; the partial-collapse result still applies.Source: Table 4 · p. 6, §4.2 · p. 6

4.3. Ablation Study

  • Removing ETF-AM drops overall accuracy to 52.42% and reduces Medium and Few accuracy by 2.4 and 4.9 points; removing the gate yields 53.60%.The parameter-matched CASA result is 54.73%, versus 52.90% for soft attention and 48.80% for an MLP; a fixed margin yields 52.80%. The text attributes the tail gain to frequency-scaled margins and describes gate removal as a smaller, more uniform drop.Source: §4.3 · pp. 6–7, Table 6 · p. 7

4.4. Empirical Support for Partial Collapse

  • Few-class spread is 1.76× Many-class spread on CIFAR100-LT and 1.40× on iNaturalist-2018 under ETF-AM, consistent with partial collapse.On CIFAR100-LT, the Few-to-Many ratio is 1.33 under the CE baseline; on iNaturalist-2018 it is 1.01.Source: Table 7 · p. 7, §4.4 · p. 7

4.5. Mask Consistency

  • Generated masks show class-related structure: train intra-class pairs reach cosine similarity 0.96 (KL 0.001), versus inter-class 0.48 (KL 1.115).Test-set intra-class similarity is 0.81 (KL 0.046) versus inter-class 0.72 (KL 0.430); same-class pairs also concentrate at lower divergence in the train and test histograms.Source: §4.5 · pp. 7–8, Table 8 · p. 8, Figure 3 · p. 8

5. Limitations

The method has geometric and theoretical scope boundaries: a simplex ETF is unavailable when classes outnumber the embedding dimension by more than one, and the theorem depends on modeling assumptions. The empirical verification covers long-tailed image classification; transfer to detection or segmentation remains open.

5. Limitations

  • When C > d + 1, a simplex ETF does not exist, so the method uses a learnable classifier whose pairwise distances vary and whose Γ requires tuning.Clearance then holds per pair rather than per class.Source: §5 · p. 8
  • The theorem assumes a finite-sample class-conditional model; proving the interior optimum for the trained network remains open.The cross-class ordering additionally requires ϕ to decrease throughout the feasible interval.Source: §5 · p. 8
  • The claim is verified for long-tailed image classification, while transfer to long-tailed detection or segmentation remains open.Source: §5 · p. 8

6. Conclusion

The paper concludes that controlled partial neural collapse preserves within-class diversity in tail classes while allowing head classes to collapse along an ETF. It reports best performance across four long-tailed benchmarks and measurements consistent with greater optimal spread for less frequent classes.

6. Conclusion

  • The method achieves the best performance on CIFAR100-LT, ImageNet-LT, Places-LT, and iNaturalist-2018.Source: §6 · p. 8
  • Controlled partial collapse combines ETF-AM for training-to-inference clearance with CASA for collapse dimensionality.Head classes effectively collapse along a fixed ETF while tail classes retain within-class diversity.Source: §6 · p. 8
  • Under a finite-sample class-conditional model, full collapse fails to minimize risk for sample-scarce classes, and measurements are consistent with optimal spread ordered by class frequency.Source: §6 · p. 8

Recognition

The appendix contrasts the full-collapse training optimum of the UFM with a finite-sample model in which estimation cost depends on class count. In that model, positive within-class spread can minimize test risk for classes meeting a sensitivity condition.

A. Finite-Sample Model for the Optimality of Partial Collapse

  • The finite-sample class-conditional analysis shows that positive within-class spread minimizes per-class risk for classes satisfying the sensitivity condition, with a count threshold and conditional ordering across classes.The ordering requires an additional monotonicity condition on ϕ.Source: A. Finite-Sample Model for the… · p. 12

A.1. The Training Optimum Under the UFM Is Full Collapse

  • Under the UFM's fixed inter-class target and finite feature budget, the training optimum is full collapse for every class, regardless of imbalance.The loss favors increasing the class-mean norm, forcing spread to zero; adaptive margins do not change that conclusion.Source: §A.1 · p. 12

A.2. The Finite-Sample Class-Conditional Model

  • The class-conditional model represents signed test margin as a separation cost from spread plus test noise and finite-sample feature-estimation noise.Its estimation variance depends on training count; the analysis also assumes training and test spreads match and classes are decoupled with shared parameters.Source: §A.2 · p. 12

A.3. Proofs of Theorem 1 and Corollary 1

The proofs establish an interior risk-minimizing spread for classes satisfying Eq. (3), then resolve that condition into a training-count threshold and a conditional ordering. The ETF appendix derives the reference scale and effective margin from the ETF geometry.

A.3. Proofs of Theorem 1 and Corollary 1

  • For classes satisfying Eq. (3), per-class risk has an interior global minimum rather than full collapse.The result is a per-class existence claim and is silent about classes that do not satisfy the condition.Source: §A.3 · p. 13
  • Under the assumed 1/n_c scaling, Eq. (3) holds below a training-count threshold; if ϕ decreases throughout the feasible interval, optimal spread is no greater for the less-scarce class.The threshold identifies where full collapse is suboptimal, while the ordering is the pattern used by ETF-AM.Source: §A.3 · p. 13

B. Reference Scale Determination Under ETF

C. Clearance Derivation Under the ETF Constraint

Under the ETF constraint, the adaptive margin defines a training-to-inference clearance that is independent of the competing class and unchanged by gating. The section also describes metrics for inter-class separation and mask consistency.

C. Clearance Derivation Under the ETF Constraint

  • The margin creates parallel training and inference boundaries whose distance defines the clearance from the inference boundary after the margin is removed.Under the ETF constraint, the clearance depends only on the true class; gating does not change it because the ETF fixes the classifier geometry.Source: C. Clearance Derivation Under the ETF… · p. 14
  • The clearance is a nominal buffer for boundary-normal variation, not a measure or guarantee of learned within-class spread.Source: C. Clearance Derivation Under the ETF… · p. 14

D.1. Fisher Discriminant Ratio

  • FDR is the ratio of between-class to within-class scatter and is used to assess inter-class separation, not uniform within-class compactness.Tail spread contributes little to total within-class scatter because tail classes are sample-scarce.Source: §D.1 · p. 15

D.2. Metrics for Mask Consistency

  • Cosine similarity and symmetric KL divergence compare mask distributions for same-class and different-class sample pairs.Source: §D.2 · pp. 15–16

E. Training Settings

The supplementary experiments examine training settings, class-frequency patterns in within-class spread, and the effect of a Few-class contraction penalty.

E. Training Settings

  • Places-LT uses (ϑ, ϖ) = (0.3, 0.3), a setting reported to activate more weights for Few classes.Experiments used Ubuntu 22.04 servers and PyTorch 2.10.0 with Python 3.13.Source: E. Training Settings · p. 16

F.1. Frequency Dependence of the Within-Class Spread

  • All four classwise log–log OLS slopes are negative, and every split retains the ordering Many < Medium < Few.Each class contributes one equally weighted observation; the fits include 100 CIFAR100-LT classes and 8,142 iNaturalist classes.Source: §F.1 · p. 17, Table 12 · p. 17
  • The study compares training and test spread by signs and orderings rather than magnitudes, while assuming equal training and test spread for tractability.The iNaturalist slopes agree closely across splits but are smaller in magnitude than those on CIFAR.Source: §F.1 · p. 17

F.2. Controlled Collapse Intervention

  • On the test set, normalized Few-class spread and post-LA Few-class accuracy both decrease monotonically as the contraction penalty increases.From λ = 0 to 0.3, test spread decreases by 19.7±1.1% and post-LA Few-class accuracy by 3.69±1.35 points; training balanced accuracy remains at least 99.6%.Source: §F.2 · p. 17
  • The contraction sweep is a one-sided stress test: its λ = 0.3 setting does not enforce exact sc = 0 and does not identify an optimum.The sweep also does not show that spread alone causes the accuracy change.Source: §F.2 · p. 17

F.3. Extended Ablation Study

The extended ablations examine how CASA hyperparameters affect mask sparsity, FDR, and classification accuracy, and offer a tuning procedure.

F.3. Extended Ablation Study

  • Increasing either ϑ or ϖ reduces mask sparsity and node specialization, alongside lower FDR and overall accuracy on both datasets.The model's behavior approaches performance without a gating layer.Source: §F.3 · p. 17
  • On ImageNet-LT, increasing ϑ from 0.5 to 0.9 lowers FDR from 9.19 × 10^2 to 8.42 × 10^2 and average accuracy from 56.90% to 55.91%.Increasing ϖ from 0.1 to 0.9 while fixing ϑ further lowers FDR to 8.21 × 10^2 and accuracy to 53.15%.Source: §F.3 · p. 17
  • For a new dataset, the authors recommend first tuning ϑ with a small fixed ϖ, then fine-tuning ϖ around its initial value.The suggested searches are ϑ in [0.1, 0.3, 0.5, 0.7] and ϖ in [0.05, 0.1, 0.3].Source: §F.3 · p. 17

F.4. Mask Mode at Training and Inference

The experiments vary training and inference mask modes, assess residual class-related mask structure, and inspect classification errors and mask similarities for visually similar classes.

F.4. Mask Mode at Training and Inference

  • Hard training raises train FDR from 6.97×10^4 to 9.94×10^4 and test FDR from 1.21×10^2 to 1.28×10^2, while inference mode leaves both unchanged.Accuracy depends on both modes, and the hard-training/soft-inference pairing is best overall; soft training costs more than two points, concentrated on Medium and Few.Source: §F.4 · p. 17, Table 14 · p. 18

F.5. Residual Mask Structure

  • After subtracting the dataset-mean mask, an intra/inter similarity gap remains on both splits, but test-set values are much smaller.The authors characterize this as empirical class-related residual structure, not a constraint enforced by the objective.Source: §F.5 · p. 18, Table 15 · p. 18

F.6. Qualitative Result

  • In the confusion matrix, head classes are nearly error-free, while tail errors include streetcar versus bus and whale versus dolphin.The authors attribute these errors more likely to semantic overlap within the class or background than to the long-tailed property.Source: §F.6 · p. 18, Figure 6 · p. 19
  • For correctly classified Few/Many pairs, masks are distinct; misclassified samples show high mask similarity to the Many class.The comparisons cover streetcar/bus (11/262 training samples) and whale/dolphin (6/119). Removing ETF-AM increases mask similarity.Source: §F.6 · p. 18, §F.4 · p. 18, Figure 7 · p. 20

Related papers

Loading 2610.36460v1…