PaperlayerView exact revision on arXiv

Source-linked AI summary

SmoothOperator: Enhancing Representations for Fine-grained Open-set Recognition via Modulated Label Smoothing

Thiru Thillai Nadarasar Bahavan, Yu Xia, Sachith Seneviratne, Saman Halgamuge

arXiv:2610.00851v1cs.CVcs.LG

TL;DR

OSR must classify known classes while rejecting unseen ones, but unknown classes are unavailable during training. SmoothOP adapts label smoothing using each sample’s prominence, based on how clearly its class stands out from its strongest competitor. On the Semantic Shift Benchmark, the augmented variants generally outperform their base objectives across datasets, semantic-shift degrees, and post-processors, with gains up to 4.7% in AUROC, OSCR, and accuracy.

  • Problem

    OSR requires rejecting unseen classes that are unavailable during training, while maintaining accurate classification of known classes.

    Source: §1 · p. 2

  • Method

    SmoothOP adapts each sample’s smoothing coefficient using prominence, the embedding-space signal of how clearly its class stands out against its strongest competing class.

    Source: §1 · p. 2

  • Results

    SmoothOP variants generally outperform their base objectives across Semantic Shift Benchmark datasets, semantic-shift degrees, and OSR post-processors, with gains up to 4.7% in AUROC, OSCR, and accuracy.

    Source: §1 · p. 2

  • Takeaways & Limitations

    SmoothOP plugs into four spherical representation objectives at minimal training cost and yields class clusters that are more compact and better dispersed.

    Source: §1 · p. 2

    SmoothOP redistributes smoothing around a base coefficient, so objectives tuned to little or no smoothing leave little for it to redistribute; adapting the base coefficient is a future step.

    Source: §7 · p. 9

Abstract

from arXiv · show

Open Set Recognition (OSR) aims to enable models to accurately classify known classes while rejecting samples from unseen classes. A key challenge in OSR lies in the inability to model the unbounded distribution of unknown classes during training, often leading to the misclassification of samples from these classes. Rather than modeling unknowns, recent work shapes the feature space so that known classes are compact and well separated, and spherical representation learning methods have achieved strong results this way. Label smoothing has been identified as one of the key drivers of this success, yet it applies the same coefficient to every training sample, regardless of how well each sample is already embedded. We show that the spherical representation learning objectives used in OSR share a single alignment--uniformity structure in which labels enter only through the alignment term. Label smoothing therefore acts as an alignment dial, and a fixed coefficient sets this dial to the same value for every sample. We propose a plug-in, SmoothOperator (SmoothOP), which sets the smoothing coefficient of each sample from its \textbf{prominence}, an embedding-space signal measuring how clearly the sample's own class stands out against its strongest competing class. Our method integrates into four existing spherical representation learning methods at minimal training overhead. SmoothOP assigns strong smoothing to samples with high prominence, which reduces their alignment and relaxes their pull. On the Semantic Shift Benchmark, SmoothOP-augmented variants generally outperform their base objectives across datasets, degrees of semantic shift, and OSR post-processors, with gains of up to 4.7\% in AUROC, OSCR, and closed-set accuracy.

1 INTRODUCTION

OSR must reject classes unavailable during training while retaining accurate classification of known classes, motivating compact, well-separated known-class representations. SmoothOP adapts label smoothing to each sample’s prominence, and its variants generally outperform their base objectives on the Semantic Shift Benchmark.

1 INTRODUCTION

  • OSR addresses the failure of closed-set classifiers to reject unseen classes; compact, well-separated known-class clusters leave other feature-space regions for unknowns.The motivating objective is to preserve classification of known classes while identifying novel classes as unknown.Source: §1 · pp. 1–2
  • Across the spherical objectives, labels affect alignment but not uniformity, so label smoothing changes alignment alone and a fixed coefficient applies the same setting to every sample.Increasing smoothing shifts some alignment pull from the sample’s own class toward other classes.Source: §1 · p. 2
  • SmoothOP sets each sample’s smoothing coefficient from prominence, relaxing the pull on already separated samples and strengthening it for samples near a competing class.Prominence measures how clearly the sample’s own class stands out against its strongest competitor.Source: §1 · p. 2
  • Up to 4.7% gains in AUROC, OSCR, and accuracy are reported for SmoothOP variants, which generally outperform base objectives across datasets, semantic-shift degrees, and post-processors.The reported evaluation is on the Semantic Shift Benchmark.Source: §1 · p. 2

2 RELATED WORK

The related work spans discriminative, generative, and hybrid OSR methods, alongside a newer focus on representation learning. The paper situates its evaluation in fine-grained semantic shift, a comparatively understudied setting.

2 RELATED WORK

  • Discriminative methods modify classifiers or feature-space boundaries, while generative methods detect unknowns through synthesis or reconstruction; hybrid approaches combine the two families.The cited limitations include difficulty characterizing open space, Euclidean-geometry reliance, computational cost, and difficulty with unknowns resembling known classes.Source: §2 · pp. 2–3
  • Representation learning is presented as a promising OSR direction after well-calibrated logit-based classifiers were shown to rival dedicated OSR systems.Vaze et al. attribute OSR gains primarily to stronger closed-set architectures and training procedures.Source: §2 · p. 3
  • Semantic shift introduces novel classes while the input domain remains largely unchanged, and the SSB isolates this setting from low-level covariate noise.Unknown classes in the described setting share significant visual attributes with known classes.Source: §2 · p. 3
  • Post-processors can alter score functions to separate known from unknown samples, but the cited discussion says they do not improve classification performance.The passage distinguishes these inference-time methods from training-time approaches.Source: §2 · p. 3
  • Fine-grained OSR is comparatively understudied, and a cited hierarchical adversarial method requires structured label hierarchies such as taxonomic annotations.That requirement restricts applicability to domains where such metadata is available.Source: §2 · p. 3

3 PROBLEM FORMULATION AND EVALUATION METRICS

The formulation distinguishes closed-set evaluation on known classes from OSR evaluation on the union of known and unknown samples. In OSR, a score and threshold determine whether to assign a known-class prediction or reject the sample as unknown.

3 PROBLEM FORMULATION AND EVALUATION METRICS

  • The supervised training set consists of image-label pairs drawn from the image space and the set of known classes.Labels are represented as one-hot vectors over the known classes.Source: §3 · p. 3
  • Closed-set testing uses samples labeled only from training categories, whereas OSR testing combines known samples with samples from disjoint unknown classes.The unknown set is defined separately from the known class set.Source: §3 · p. 3
  • At inference, the model predicts the highest-probability known class when its known-class score meets threshold θ; otherwise, it predicts unknown.The score function indicates whether a test sample belongs to any known class.Source: §3 · p. 3

4 METHODOLOGY

The method intervenes in the representation-learning stage of a three-stage OSR pipeline, using per-sample prominence to adapt label smoothing. Its rationale is that smoothing adjusts alignment without changing uniformity.

4 METHODOLOGY

  • The pipeline first shapes representations, then trains a linear classifier on a frozen encoder, and finally scores samples for unknown rejection before classifying those deemed known.SmoothOP plugs into the first-stage representation objective.Source: §4 · p. 4, §4.1 · p. 5

4.1 GENERAL REPRESENTATION LEARNING FRAMEWORK

  • Stage 1 augments inputs, smooths targets, optionally applies Mixup, and trains an encoder with a projection head on normalized projections using one of four representation objectives.The four objectives are ConOSR, SpHOR, SupCON, and SpHOR-Lite.Source: §4.1 · p. 4

4.2 REPRESENTATION OBJECTIVES: ALIGNMENT AND UNIFORMITY

  • The four objectives decompose into alignment and uniformity, with the uniformity term independent of targets and the alignment term dependent on target-based attraction weights.Label smoothing therefore changes alignment alone, leaving uniformity unchanged.Source: §4.2 · p. 6
  • Increasing smoothing transfers attraction away from the ground-truth class in SpHOR variants, while reducing positive weights and increasing negative weights in SupCON and ConOSR.The changes apply to alignment; uniformity remains unchanged.Source: §4.2 · p. 6

4.3 PROMINENCE AND ADAPTIVE LABEL SMOOTHING

  • An observer classifier estimates each sample’s prominence from gradient-detached embeddings, using the true-class probability relative to the strongest competing class.This score is mapped to an individual smoothing coefficient.Source: §4.3 · p. 6
  • Low-prominence samples lie near a competing class and receive a stronger class pull, whereas already well-separated, high-prominence samples receive stronger smoothing and a relaxed pull.The method’s motivation is to focus stronger alignment on samples near the edge of a class cluster.Source: §4.3 · p. 6
  • The sigmoid modulator bounds each coefficient between 0 and 2ε, recovers fixed smoothing when α is zero, and decreases with the margin u_i when α is positive.The modulating strength is α.Source: §4.3 · p. 7
  • For Mixup samples, constituent margins are retrieved from unmixed samples and combined by weighted interpolation to produce one coefficient for the mixed target.The composite prominence is defined as a weighted combination of constituent prominence.Source: §4.3 · p. 7

5 EXPERIMENTAL SETUP

The experiments use three fine-grained Semantic Shift Benchmark datasets and compare SmoothOP-augmented representation objectives under multiple open-set scoring functions. The reported tables cover open-set metrics, closed-set accuracy, and feature geometry.

5 EXPERIMENTAL SETUP

  • The Semantic Shift Benchmark evaluation uses CUB, FGVC-Aircraft, and Stanford Cars, each with Easy and Hard unknown-class test splits.The splits are based on unknown classes’ semantic similarity to training classes.Source: §5 · p. 7
  • Base smoothing is 0.3 on CUB and 0.2 on Aircraft and Cars; each dataset–method pair’s α is selected by three-fold cross-validation using mean OSCR under MSP.The base smoothing values are adopted from Vaze et al. (2022) without further tuning.Source: §5 · p. 7
  • Four representation methods are augmented: SpHOR, SpHOR-Lite, ConOSR, and SupCON, using a Places-365-pretrained ResNet-50 backbone.Training uses three fixed seeds; the classifier head is retrained with the backbone frozen before evaluation.Source: §5 · pp. 7–8
  • Evaluation uses MLS, nnGuide, PostMAX, and MSP, assessing unknown rejection with AUROC, known-class accuracy, and both jointly with OSCR.The reported tables include OSCR and AUROC, closed-set accuracy, and feature-geometry gains in compactness and dispersion.Source: §5 · p. 8, Table 2 · p. 8, Table 3 · p. 8

6 RESULTS AND DISCUSSION

SmoothOP generally improves open-set and closed-set performance across the Semantic Shift Benchmark, while its feature-geometry results show tighter classes without reduced inter-class separation. The geometry results also track open-set performance across the reported settings.

6 RESULTS AND DISCUSSION

  • SmoothOP improves AUROC, closed-set performance, and OSCR across training-testing regimes, with overall consistency across three datasets.The authors report minor fluctuations in some sub-metrics and say the performance tax is insignificant.Source: §6 · p. 9
  • SmoothOP reduces intra-class compactness Comp in 11 of 12 settings and increases inter-class dispersion Disp in all 12.The authors describe the resulting geometry as compact and well separated, linking it to reduced open-space risk.Source: §6 · p. 9
  • The one setting without a Comp improvement, ConOSR on CUB, also regresses on the Easy split under every post-processor; SpHOR has the most consistent geometric and open-set gains.The reported geometry-performance relationship is described across the settings examined.Source: §6 · p. 9

7 LIMITATIONS AND FUTURE WORK

SmoothOP redistributes a base objective’s label smoothing rather than adding smoothing, so its effect depends on the base coefficient. The section also records the authors’ AI-use, ethics, and reproducibility statements.

7 LIMITATIONS AND FUTURE WORK

  • SmoothOP’s effect scales with the base smoothing coefficient ε; objectives tuned to little or no smoothing leave little smoothing to redistribute.The authors identify making ε adaptive as a natural next step.Source: §7 · p. 9

AI USE STATEMENT

  • Generative AI tools were used only for manuscript grammar checking, not data analysis, model development, experimental design, or result interpretation.The authors state that research decisions and scientific contributions remain their responsibility.Source: AI USE STATEMENT · p. 9

ETHICS STATEMENT

  • The authors state that the work follows the ICLR Code of Ethics and uses standard image-classification benchmarks without personal data, sensitive content, or harmful applications.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • Experiments use the public Semantic Shift Benchmark’s published splits, and reported numbers are means over three fixed seeds (1, 2, 3).The authors state that hyperparameters, backbone initialization, hardware, code, and configuration files are included in the supplementary materials.Source: REPRODUCIBILITY STATEMENT · p. 10

A EVALUATION METRICS

The appendix defines open-set evaluation using known and unknown test samples and metrics for rejection and classification together. It also specifies augmentation hyperparameters and a frozen-backbone classifier retraining stage.

A EVALUATION METRICS

  • AUROC measures threshold-independent discrimination between known classes C and unknown classes U, but does not account for classification accuracy.The section introduces CCR and FPR at an operational threshold θ to connect unknown rejection with known-class classification.Source: A EVALUATION METRICS · p. 13
  • OSCR summarizes performance across operational thresholds as the area under the CCR-versus-FPR curve, while CCR5 is threshold-dependent at FPR = 5%.The section also states that OOSA is not used because its threshold estimate requires a surrogate set and validation dataset.Source: A EVALUATION METRICS · p. 13

B.1 HYPERPARAMETER AND DATA AUGMENTATION

  • Table 5 lists fine-grained dataset hyperparameters adopted directly from MLS; RandAugment is controlled by transformation count N and magnitude M.Source: Table 5 · p. 14, §B.1 · p. 14

B.2 STAGE 02 TRAINING

  • Stage 2 freezes the backbone, caches 2048-dimensional image features, and trains a linear classifier head for 20 epochs with cross-entropy and no label smoothing.The head uses Adam with learning rate 10^-4, without weight decay or a learning-rate schedule.Source: §B.2 · p. 14

B.3 DATASET

The Semantic Shift Benchmark evaluates fine-grained datasets by training on known classes and reserving unknown classes for open-set evaluation. The scoring functions range from maximum-confidence or logit scores to calibrated and similarity-weighted scores.

B.3 DATASET

  • SSB uses CUB, FGVC-Aircraft, and Stanford Cars, with known classes for training and unknown classes reserved for open-set evaluation.The benchmark varies semantic shift across its fine-grained datasets.Source: §B.3 · p. 14
  • CUB contains 11,788 images from 200 bird species; SSB uses 100 known species and groups the remainder by semantic similarity.Subtle inter-class differences and limited per-class training samples make the dataset challenging.Source: §B.3 · p. 14
  • FGVC-Aircraft uses 50 of 100 variants as known classes, while Stanford Cars uses 98 of 196 models; remaining classes are unknown.Aircraft partitions follow a semantic hierarchy; car images vary in viewpoint and fine-grained model and production-year distinctions.Source: §B.3 · p. 14

B.4 SCORING FUNCTIONS

  • MSP scores the largest softmax probability, while MLS scores the largest logit across classes.Both are defined over classifier outputs.Source: §B.4 · p. 15
  • PostMax normalizes the maximum logit by feature magnitude, then maps it through a GPD fitted to correctly classified training samples.Feature-norm division counteracts larger norms observed for unknown samples on large-scale models; the GPD maps the score to a calibrated probability.Source: §B.4 · p. 15
  • NNGuide multiplies a base classifier score by top-k confidence-weighted similarities, lowering scores for far-OOD inputs while separating near-OOD inputs around class boundaries.Confidence-weighted neighbors direct the search toward high-confidence in-distribution regions.Source: §B.4 · p. 15

C.1 OPERATING CURVE ANALYSIS

SmoothOP-enhanced operating curves generally outperform their baselines across the illustrated methods, with stronger curves on hard splits. The Observer classifier adds almost negligible training cost.

C.1 OPERATING CURVE ANALYSIS

  • On hard splits, enhanced variants shift OSCR curves upward, maintaining higher known-class CCR at lower unknown-class FPR.The passage attributes this pattern to focusing on hard positives during training.Source: §C.1 · p. 15
  • SmoothOP pushes OSCR baselines outward almost consistently across operating points for SupCON, ConOSR, and SpHOR-Lite with MSP.The curves are averaged over three seeds at fixed FPR and shown for easy and hard splits.Source: Figure 2 · p. 16, Figure 3 · p. 16, Figure 4 · p. 16

C.2 FLOPS FOR OBSERVER CLASSIFIER

  • The Observer classifier’s training cost is almost negligible; it trains in parallel with the backbone using cross-entropy to supply ui.Table 7 reports FLOPs for ResNet-50 and notes that the cost becomes smaller with heavier backbones.Source: §C.2 · p. 17, Table 7 · p. 17

D.1 ABLATION: INTER-BATCH AND INTRA-BATCH VARIATION DECOMPOSED.

The appendix tests SmoothOP’s per-sample assignment, temporal behavior, margin-blending choice, and representation geometry, alongside its validation and integration procedures. Its reported results favor per-sample assignment and margin blending, show a separability trade-off, and describe smoothing that increases over training.

D.1 ABLATION: INTER-BATCH AND INTRA-BATCH VARIATION DECOMPOSED.

  • On average, ∆rev > ∆bm > 0 (+0.25 > +0.18 on easy, +0.34 > +0.05 on hard), supporting an intra-batch assignment effect.Table 8 averages OSCR across four post-processors and three seeds.Source: §D.1 · pp. 17–18, Table 8 · p. 18
  • Inverting the assignment is more harmful than removing it, while negative batch-mean gains are concentrated in some methods, including SpHOR-Lite.The authors note SpHOR is used in practice rather than the SpHOR-Lite ablation.Source: §D.1 · p. 18

D.2 TEMPORAL BEHAVIOR OF THE SMOOTHING METHOD

  • For SpHOR/CUB, the smoothing coefficient rises from λ = 0.3 to 0.524 as training progresses, also functioning as a time-varying smoothing schedule.The passage describes this increase as expected as the model becomes more confident.Source: §D.2 · pp. 18–19

D.3 ABLATION: MIXING SMOOTHING COEFFICIENTS VS MIXING MARGINS.

  • Margin blending leads coefficient blending on 11 of 12 cells and by +0.27 OSCR points on both splits.The reported comparison uses shared hyperparameters and averages three seeds.Source: Table 9 · p. 19, §D.3 · p. 19

D.4 HOW DOES SMOOTHOP OPTIMIZE REPRESENTATION GEOMETRY FOR SEPARABILITY?

  • Adding SmoothOP improves norm separability by +2.94 AUROC points on Easy and +1.57 on Hard, while angular separability drops by 0.0032 on Easy and 0.0039 on Hard.The passage characterizes the norm-separability gains as several times larger than the angular-separability losses, with a net effect of better unknown rejection.Source: Table 10 · p. 20, §D.4 · p. 20

D.5 HYPERPARAMETER ANALYSIS

  • The validation rule selects α by OSCR with MSP, using a different fine-grained dataset as the unknown validation set for each split.The procedure tests α values from {8, 4, 3, 2, 1.5, 1, 0.5}.Source: §D.5 · p. 21

E METHODOLOGY: INTEGRATION

  • SmoothOP transforms label-space targets independently of the loss function, and each base method is recovered exactly when ε_i ≡ 0.The transformation reaches the original one-hot vector or the uniform distribution at its limits.Source: E METHODOLOGY: INTEGRATION · p. 21

E.1.1 SPHOR

  • SpHOR maps features to a hyperspherical embedding space with a Von Mises-Fisher Alignment Loss and adds Mixup-generated virtual samples to simulate open space.SpHOR-Lite omits the Mixup term as an ablation.Source: §E.1.1 · pp. 21–22

E.1.2 ABLATION WITHOUT MIXUP: SPHOR-LITE

The appendix specifies how SmoothOP relates to several spherical and contrastive objectives. It describes the role of Mixup in ConOSR, how smoothing shifts contrastive target weights, and which target-dependent terms appear in the objective decompositions.

E.1.2 ABLATION WITHOUT MIXUP: SPHOR-LITE

  • SpHOR-Lite is the SpHOR ablation that drops the Mixup term.The paper reports this variant to check whether results hinge on Mixup.Source: §E.1.2 · p. 22

E.1.3 SUPERVISED CONTRASTIVE LEARNING

  • SupCon-ST defines the supervised contrastive representation objective on L2-normalized projection features, using pairwise predictions and soft pairwise targets.The predictions are softmax-normalized cosine similarities, and the targets derive from label vectors.Source: §E.1.3 · p. 22

E.1.4 CONOSR

  • ConOSR adds contrastive training on Mixup-generated virtual samples that simulate open space, separating clean-sample and Mixup-sample terms.This separation gives each term contrast labels with a uniform norm, allowing the target to reduce to an inner-product expression.Source: §E.1.4 · p. 22

E.1.5 INTEGRATION WITH SUPCON AND CONOSR

  • For SupCON and ConOSR, increasing smoothing shifts each anchor’s target weight from positive pairs toward negative pairs; decreasing it sharpens the target toward positives.Smoothing applies to anchor labels, not contrast labels.Source: §E.1.5 · p. 23

F.1 DECOMPOSITION OF SPHOR-LITE AND SPHOR

  • In the SpHOR-Lite and SpHOR decomposition, targets enter the objective through the target-dependent alignment loss, while the prototype regularizer depends only on class prototypes.The SpHOR derivation extends to Mixup samples without changing the derivation.Source: §F.1 · p. 24

F.2 DECOMPOSITION OF SUPCON AND CONOSR

  • Figure 5 compares SpHOR OSCR operating curves with and without SmoothOP across Easy and Hard splits; Figure 6 tracks batch-mean smoothing and its inter-decile range over training.Figure 5 reports curves averaged over three seeds at fixed FPR; Figures 7 and 8 show α hyperparameter sweeps for the method pairs named in their captions.Source: Figure 5 · p. 25, Figure 6 · p. 25, Figure 7 · p. 26
Loading 2610.00851v1…