How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?

• OOD detection determines whether a test input belongs to the known distribution or comes from an unknown distribution.Source: §1 · p. 1
• Distance-based methods use feature embeddings and assume OOD samples lie relatively far from ID data clusters.Source: §1 · p. 1
• Existing contrastive objectives can suffice for ID classification while remaining sub-optimal for OOD detection.Source: §1 · p. 1
• CIDER jointly optimizes dispersion and compactness losses to create hyperspherical embeddings with separated class prototypes and samples close to their prototypes.Source: §1 · p. 2
• 42.36% relative improvement in ID-OOD separability is achieved by CIDER compared to SupCon on CIFAR-10.Source: §1 · p. 2
• 13.33% error reduction in FPR95 is achieved by CIDER compared to the latest rival on CIFAR-100.Source: §1 · p. 2
• CIDER establishes the relationship between hyperspherical embedding quality and OOD detection and remains effective under varied settings, including ImageNet.Source: §1 · p. 2

• OOD detection is treated as binary classification between samples from the ID distribution and samples from an OOD distribution.Source: §2 · p. 2
• An OOD score classifies samples as ID when their score exceeds a threshold selected to retain a high fraction of correctly classified ID data.Source: §2 · p. 2
• Hyperspherical embeddings lie on the surface of a unit hypersphere, whose points have constant Euclidean norm.Source: §2 · p. 2
• CIDER’s compactness and dispersion terms encourage samples to remain close to class prototypes while different prototypes become more angularly separated.Source: Figure 1 · p. 3

Framework overview

• CIDER uses an encoder and projection head to produce normalized hyperspherical feature embeddings for its training objective.The encoder maps augmented inputs to high-dimensional features, while the projection head produces lower-dimensional representations that are normalized before applying the loss.Source: §3 · p. 3

Hyperspherical modeling

• The von Mises–Fisher model represents each class with a unit-norm prototype, while κ controls concentration around its mean direction.Larger κ produces stronger concentration; κ = 0 gives a uniform distribution on the hypersphere.Source: §3.1 · p. 3

Compactness loss

• The compactness loss encourages each sample embedding to align closely with its class prototype and assigns higher probability to the correct class than to incorrect classes.The loss follows from the negative log-likelihood of the class-conditional vMF model and uses the sample’s class index and a temperature parameter.Source: §3.2 · pp. 3–4, §3.1 · p. 3

Dispersion loss

• The dispersion loss maximizes angular distances among different class prototypes, because OOD samples may lie between ID clusters.Larger angular separation between ID prototypes can improve the separability of OOD samples that fall between those clusters.Source: §3.2 · p. 4, Figure 2 · p. 4

Combined objective

• CIDER combines dispersion and compactness as LCIDER = Ldis + λcLcomp to shape embeddings for both ID classification and OOD detection.The coefficient λc controls the relative importance of the two loss terms, which are described as complementary.Source: §3.2 · p. 4

Prototype estimation and update

• CIDER updates class prototypes with an exponential-moving-average rule rather than repeatedly computing means over all training samples.The update uses factor α to combine the previous prototype with the normalized embedding of samples from the class.Source: §3.2 · p. 4

Experimental setup

• CIFAR-100 experiments use ResNet-34, with CIFAR-100 as ID and SVHN, Places365, Textures, LSUN, and iSUN as OOD datasets.The evaluation reports FPR95, AUROC, and ID classification accuracy.Source: §4.1 · p. 5

Main results

• CIDER improves OOD detection over competitive methods, reducing FPR95 by 20.3% versus SSD+ and 13.3% versus KNN+.It also reduces average FPR95 by 20.6% compared with CSI.Source: §4.2 · pp. 5–6, Table 1 · p. 6

OOD score ablation

• CIDER improves performance with both non-parametric KNN and parametric Mahalanobis scores.CIDER with KNN reduces FPR95 by 13.33% versus SupCon+KNN, while CIDER+Maha reduces FPR95 by 22.77% versus SSD+.Source: §4.2 · p. 6, Table 2 · p. 6

Loss ablation

• The compactness loss alone supports ID classification, while inter-class dispersion from the dispersion loss is key to OOD detection.The loss components work synergistically to improve hyperspherical embeddings for both tasks.Source: §4.2 · p. 6, Table 3 · p. 7

Embedding analysis

• CIDER embeddings are more compact and distribute classes more uniformly than CE embeddings in UMAP visualizations.The visualization links improved compactness and dispersion with greater OOD separability.Source: §4.3 · p. 7, Figure 3 · p. 7
• CIDER improves inter-class dispersion by 12.03 degrees over SSD+ and yields a relative 42.36% improvement in ID-OOD separability over SupCon across five OOD datasets.Higher separability indicates that OOD samples are easier to detect.Source: §4.3 · pp. 7–8

Additional analysis

• CIDER remains competitive on large-scale ImageNet-100 evaluations and across ablations involving training and model settings.The ImageNet-100 study fine-tunes a pre-trained ResNet-34, while additional experiments vary batch size and architecture.Source: §4.4 · p. 8, Figure 4 · p. 8

Contrastive representation learning

• Contrastive representation learning aligns positive samples while dispersing negative samples across supervised, semi-supervised, and unsupervised settings.Source: §5 · p. 9

Representation learning for OOD detection

• Prior OOD studies apply off-the-shelf losses such as SupCon and SimCLR, whereas CIDER focuses on hyperspherical embeddings.Source: §5 · p. 9

Deep metric learning

• Deep metric learning develops embeddings for verification, re-identification, and retrieval, but prior work does not focus on embeddings designed for OOD detection.Source: §5 · p. 9

• CIDER jointly optimizes dispersion and compactness losses to promote strong ID-OOD separability.Source: §6 · p. 9
• CIDER achieves superior performance on common OOD benchmarks, including large-scale OOD detection tasks.Source: §6 · p. 9
• The work introduces measurements linking hyperspherical embedding quality with OOD detection performance and evaluates CIDER through extensive ablations.Source: §6 · p. 9

CIDER training framework

• CIDER jointly optimizes compactness for sample-prototype alignment and dispersion for larger angular distances among class prototypes.Source: A ALGORITHM DETAILS AND DISCUSSIONS · p. 14

Prototype updates

• Class prototypes are updated by exponential moving average rather than receiving direct gradients from the dispersion loss.Source: A ALGORITHM DETAILS AND DISCUSSIONS · p. 14

Relation to contrastive losses

• CIDER promotes prototype-to-prototype dispersion, unlike uniformity losses that promote instance-to-instance uniformity from randomly sampled pairs.Source: A ALGORITHM DETAILS AND DISCUSSIONS · p. 14

Relation to cross-entropy

• CIDER is designed for hyperspherical embeddings, whereas cross-entropy does not explicitly optimize intra-class compactness and inter-class dispersion in feature space.Source: A ALGORITHM DETAILS AND DISCUSSIONS · pp. 14–15

Embedding measurements

• Embedding quality is measured through inter-class dispersion and intra-class compactness using class prototypes or representative samples.Source: A ALGORITHM DETAILS AND DISCUSSIONS · p. 15

Software and hardware

• Experiments run on RTX-2080Ti GPUs for small or medium settings and A100 GPUs for large batches or larger encoders.Source: B EXPERIMENTAL DETAILS · p. 15

Architecture

• CIDER uses a projection head with output dimension 128, with a two-layer nonlinear head for CIFAR-10 and CIFAR-100.Source: B EXPERIMENTAL DETAILS · p. 15

Training

• CIFAR-10 models train for 100 epochs, while CIFAR-100 models train for 200 epochs under scheduled learning-rate decay.Source: B EXPERIMENTAL DETAILS · p. 15

OOD detection score

• The default OOD score is non-parametric KNN, using K = 100 for CIFAR-10 and K = 300 for CIFAR-100.Source: B EXPERIMENTAL DETAILS · p. 15

Batch size

• CIDER outperforms SupCon across different batch sizes, aided by instance-to-prototype compactness and class-conditional prototypes that reduce batch-size dependence.The prototype updates capture class-average statistics and provide a memory-efficient alternative to instance-to-instance distance measurement.Source: C ADDITIONAL ABLATION STUDIES · p. 15, Figure 6 · p. 16

Loss weights

• CIDER performance is relatively stable for λc from 0.5 to 2, with best performance around λc ∈[1, 2].This reduces the need for extensive tuning of the loss scale.Source: C ADDITIONAL ABLATION STUDIES · pp. 15–16

Prototype updates

• For CIFAR-100, α = 0.5 achieves average FPR95 of 46.89% under KNN, while larger α values improve CIFAR-10 performance.Detection remains competitive across a wide range of prototype moving-average discount factors.Source: C ADDITIONAL ABLATION STUDIES · p. 16

Optimization

• A relatively high initial learning rate is preferable, whereas a small temperature τ produces better OOD detection performance.Too small an initial learning rate, such as 0.1, degrades performance.Source: C ADDITIONAL ABLATION STUDIES · p. 16

Architecture

• CIDER improves FPR95 across various test sets with ResNet-50, following the trend observed with ResNet-34.The architecture ablation attributes the improvement to the learned representation.Source: C ADDITIONAL ABLATION STUDIES · p. 16

Prototype update rules

• CIDER with EMA improves average FPR95 by 5.08% and has smaller standard deviation than learnable prototypes, indicating stronger training stability.The comparison averages three independent CIFAR-10 runs.Source: C ADDITIONAL ABLATION STUDIES · p. 17, Table 5 · p. 17

Stability

• Across three independent seeds, CIDER yields consistently strong OOD detection performance on CIFAR-10 and CIFAR-100.The reported stability uses ResNet-18 for CIFAR-10 and ResNet-34 for CIFAR-100.Source: C ADDITIONAL ABLATION STUDIES · p. 17, Table 6 · p. 17

Fine-tuning setup

• CIDER and SupCon are compared by fine-tuning ImageNet-pre-trained ResNet-34 on ImageNet-100 for 10 epochs.The final residual block and nonlinear projection head are updated while earlier residual blocks are frozen.Source: D RESULTS ON LARGE-SCALE DATASETS · p. 17

Architecture context

• CIDER’s architecture ablation on ResNet-50 reports results based on that network.The supplied architecture caption identifies the evaluation backbone but does not state a numerical comparison.Source: Figure 9 · p. 18

Results

• With the same KNN detection score, CIDER consistently outperforms SupCon across all OOD test datasets.The comparison is reported for ImageNet-100 ID data.Source: Figure 10 · p. 18

Evaluation setup

• For contrastive-loss methods, CIFAR-10 uses the same network encoder and embedding dimension while varying only the training objective.The evaluation uses Mahalanobis scoring for SSD+, CE+SimCLR, SupCon, and CIDER.Source: E RESULTS ON CIFAR-10 · p. 18

Evaluation protocol

• For contrastive methods including KNN+, SSD+, and CIDER, ID classification uses linear probes on normalized features.This follows the common practice used for contrastive representation learning.Source: F ID CLASSIFICATION ACCURACY · p. 18

CIFAR-10 results

• CIDER is described as achieving strong OOD detection performance and ID classification accuracy on CIFAR-10.The supplied table caption does not provide the corresponding numerical values.Source: Table 7 · p. 19

CIFAR-10

• Table 8 reports ID classification accuracy on CIFAR-10.The supplied table passage provides the table’s topic but no numerical values.Source: Table 8 · p. 19

CIFAR-100

• Table 9 reports ID classification accuracy on CIFAR-100.The supplied table passage provides the table’s topic but no numerical values.Source: Table 9 · p. 19