Source-linked AI summary
Semantically Aligned Bias Reducing Zero Shot Learning
Akanksha Paul, Narayanan C. Krishnan, Prateek Munjal
TL;DR
ZSL uses semantic relationships to recognize unseen classes, but hubness and seen-class bias remain challenges that existing methods often address separately. SABR combines a semantically aligned, discriminative latent space with bias-reducing generators, and outperforms state-of-the-art methods across conventional and generalized settings.
Problem
ZSL must recognize unseen classes from semantic relationships, while hubness and seen-class bias hinder performance and existing methods focus on only one of these problems.
Source: §1 · p. 1Method
SABR learns a discriminative latent space that preserves semantic relationships, then synthesizes unseen-class representations with bias reduction via early stopping inductively or weak transfer transductively.
Source: §1 · p. 2Results
SABR outperforms state-of-the-art algorithms by ~1.5-9% in conventional ZSL and ~2-14% in generalized ZSL across inductive and transductive settings on most datasets.
Source: §6 · p. 8Takeaways & Limitations
Experiments support SABR as an approach addressing both hubness and seen-class bias across conventional and generalized ZSL in inductive and transductive settings.
Source: §1 · pp. 1–2SABR-T's transfer constraint assumes seen- and unseen-class generators should be similar because their labels and latent representations share common spaces.
Source: §3.2.3 · p. 4
Abstract
from arXiv · showhide
Zero shot learning (ZSL) aims to recognize unseen classes by exploiting semantic relationships between seen and unseen classes. Two major problems faced by ZSL algorithms are the hubness problem and the bias towards the seen classes. Existing ZSL methods focus on only one of these problems in the conventional and generalized ZSL setting. In this work, we propose a novel approach, Semantically Aligned Bias Reducing (SABR) ZSL, which focuses on solving both the problems. It overcomes the hubness problem by learning a latent space that preserves the semantic relationship between the labels while encoding the discriminating information about the classes. Further, we also propose ways to reduce the bias of the seen classes through a simple cross-validation process in the inductive setting and a novel weak transfer constraint in the transductive setting. Extensive experiments on three benchmark datasets suggest that the proposed model significantly outperforms existing state-of-the-art algorithms by ~1.5-9% in the conventional ZSL setting and by ~2-14% in the generalized ZSL for both the inductive and transductive settings.
1. Introduction
ZSL uses semantic relationships between seen and unseen classes to recognize unseen instances, but hubness and bias toward seen classes remain challenges. SABR addresses both through a learned latent space and bias-reducing generators, with inductive and transductive variants.
1. Introduction
- Inductive ZSL provides seen-class instances and unseen-label embeddings during training, while transductive ZSL also provides unlabeled unseen instances; conventional and generalized ZSL differ in whether test data come only from unseen classes or from both seen and unseen classes.Generalized ZSL is described as the more realistic testing setting.Source: §1 · p. 1
- Hubness can degrade performance in conventional and generalized ZSL, while bias toward seen classes generally harms recognition of unseen classes.Hubness arises when transformed data become hubs near class embeddings.Source: §1 · p. 1
- SABR learns an intermediate latent space that preserves semantic relationships between class embeddings while retaining discriminating information among classes.Its loss function is designed to make the embedding space both discriminative and semantically aligned.Source: §1 · p. 2
- SABR-I synthesizes unseen-class representations with a conditional Wasserstein GAN and reduces seen-class bias by early stopping, using a simulated ZSL validation problem on seen-class data.The synthetic representations support learning a classifier for ZSL and GZSL.Source: §1 · p. 2
- SABR-T learns a generator for unseen-class latent instances, minimizing marginal-distribution differences while weakly transferring conditional information from the seen-class generator.The transfer constraint is intended to reduce bias toward seen classes.Source: §1 · p. 2
- SABR outperforms state-of-the-art approaches in conventional and generalized ZSL in both inductive and transductive settings.The empirical evaluation spans all zero-shot learning datasets described in the passage.Source: §1 · p. 2
2. Related Work
Earlier ZSL work projected instances into semantic label space or learned compatibility functions, while later work addressed hubness and class separability. SABR-I adds seen-class bias reduction to semantic-relation preservation, and SABR-T differs from QFSL in its approach to bias and hubness.
2. Related Work
- Early methods projected instances into semantic label space, while DEVISE, ALE [1], and SJE learned bilinear compatibility functions; ESZSL added a regularizer, and LATEM [23] and CMT [19] learned nonlinear functions.Zhang et al. [29] demonstrated hubness and proposed an intermediate visual space; Zhang et al. [29] and Ba et al. [5] used joint embedding spaces.Source: §2 · p. 2
- Annadani et al. [3] used semantic structure while maintaining class separability; SABR-I also preserves semantic relations in visual space and reduces seen-class bias.Source: §2 · p. 2
- Song et al. [20] reduce bias with a uniform classifier-output prior for unseen instances, whereas SABR-T avoids that prior and also learns an optimal latent space to mitigate hubness.SABR addresses bias in both inductive and transductive versions.Source: §2 · p. 2
- Verma et al. [21] model class-conditional distributions as Gaussians, while Bucher et al. [6] and Xian et al. [25] generate pseudo-instances with a conditional generator trained on seen classes.SABR-I differs by learning a discriminative, semantically structured embedding space and using unlabeled unseen data to reduce seen-class bias.Source: §2 · p. 2
3. Methodology
SABR defines inductive and transductive settings according to whether unlabeled unseen instances are available, and builds latent representations and generators for both. Its approach combines semantic-label relationships with class discrimination, then uses generated representations to train classifiers for conventional and generalized ZSL.
3.1. Problem Definition
- The method defines inductive ZSL using labeled seen instances and unseen-class labels, while transductive ZSL also provides unlabeled unseen instances; conventional ZSL predicts unseen labels, whereas generalized ZSL permits seen or unseen test classes.Source: §3.1 · p. 3
3.2. Semantically Aligned Bias Reducing ZSL
- SABR-I uses a two-tier model for inductive ZSL, while SABR-T adds a third tier for transductive ZSL.Source: §3.2 · p. 3
3.2.1 Learning the Optimal Latent Space
- A learned latent space preserves semantic relations among classes while encoding discriminative information, using a transformation jointly trained with a classifier and semantic-label regressor.The regressor is trained to relate each instance representation to its semantic label embedding; the resulting latent space supports later synthesis of unseen-class instances.Source: §3.2.1 · pp. 3–4
3.2.2 Bias Reducing Generator Network for SABR-I
- SABR-I trains a conditional generator on seen-class latent representations, then uses its synthetic unseen-class representations to train classifiers; cross-validation selects an early-stopping point to reduce seen-class bias.Source: §3.2.2 · p. 4
3.2.3 Bias Reducing Generator Network for SABR-T
- SABR-T trains an unseen-class generator to model unlabeled unseen latent instances and weakly transfers conditional information from the seen-class generator.The transfer strength is tuned by cross-validation: no transfer leaves the unseen generator independent, while excessive transfer makes it identical to the seen generator and risks seen-class bias.Source: §3.2.3 · pp. 4–5
3.3. Classification and Evaluation Metric
- Synthetic labeled unseen representations train a softmax classifier for conventional ZSL; for generalized ZSL, they are combined with labeled seen representations, and performance is measured by per-class accuracy and its seen/unseen harmonic mean.Source: §3.3 · p. 5
4. Experiments
The experiments use three ZSL benchmarks and the proposed Xian et al. splits, with seen-class cross-validation for tuning. Implementation uses ResNet-101 features and specified neural-network architectures for the latent-space transformation and generators.
4.1. Datasets
- The evaluation uses AWA2, CUB, and SUN, with 40/10, 150/50, and 580/72 training/test classes, respectively.The datasets contain 37,322, 11,788, and 14,340 images, respectively; their class labels use 85-, 312-, and 102-dimensional vectors.Source: §4.1 · p. 5
4.2.1 Learning the optimal latent space
- Experiments use ResNet-101 features and a jointly learned latent-space network with a two-layer 2048/1024-unit transformation, a softmax classifier, and a semantic-label regressor.The learning rate is 0.001 and γ is 0.01 across datasets.Source: §4.2.1 · p. 5
4.2.2 Learning the bias reducing generators
- The generators use a 2048-unit hidden layer and discriminators a 4096-unit hidden layer; the seen-class conditional WGAN is trained before adaptation to unseen classes.The gradient-penalty coefficient λ is 10 and the latent-representation coefficient β is 0.1 across datasets; other hyperparameters are tuned by stratified cross-validation on seen-class training data.Source: §4.2.2 · pp. 5–6
5. Results and Discussion
SABR’s results cover bias reduction in inductive and transductive settings, conventional and generalized ZSL, and performance with reduced unlabeled data. Across the reported benchmarks, its methods outperform prior approaches in several settings, with gains varying by dataset and comparison.
5.1. Bias reduction in SABR-I
- For SABR-I, unseen-class MCA_u begins falling after epoch 45 on AWA2, while seen-class MCA_s plateaus; early stopping is set at 40, 45, and 95 epochs for AWA2, CUB, and SUN.The decline in harmonic mean is most evident on AWA2 and less pronounced on CUB and SUN.Source: §5.1 · p. 6
5.2. Bias reduction in SABR-T
- SABR-T selects ω by cross-validation; the chosen values are 0.008 for AWA2 and 0.002 for both CUB and SUN.The selected values are fixed for the remaining experiments.Source: §5.2 · p. 6
5.3. Performance in the conventional ZSL setting
- In conventional ZSL, SABR-I exceeds prior inductive results by 1.4%, 5.07%, and 1.3% on AWA2, CUB, and SUN; SABR-T gains 9.2%, 1.4%, and 3.6% in the transductive setting.No single prior method leads across all datasets and both settings; Table 1 presents conventional ZSL results.Source: §5.3 · p. 6, Table 1 · p. 7
5.4. Performance in the generalized ZSL setting
- SABR-I outperforms PSR by 14.65%, 22.89%, and 12.31% on AWA2, CUB, and SUN, while SABR-T beats QFSL by 7.61% on AWA2 and 9.38% on SUN in generalized ZSL.On CUB, SABR-T is 2.9% below QFSL; Table 2 presents generalized ZSL results.Source: §5.4 · p. 7, Table 2 · p. 8
- SABR-I exceeds f-CSLWGAN by 7.09% on CUB, while its SUN result is 2.1% higher.The authors speculate that SUN’s smaller number of training samples per class is insufficient to learn the optimal latent space and seen-class generator.Source: §5.4 · p. 7
5.5. Amount of unlabeled data for transductive learning
- SABR-T’s performance peaks using 6.5% of AWA2’s unlabeled instances, 10% on CUB, and 35% on SUN.Performance on AWA2 and SUN rises as more unlabeled instances are used during training.Source: §5.5 · pp. 7–8
6. Summary and Future Work
SABR addresses hubness and seen-class bias through a semantic-discriminative latent space and setting-specific bias-reduction methods. Across three benchmark datasets, it outperforms state-of-the-art methods in conventional and generalized ZSL, with peak transductive performance using only a fraction of unlabeled unseen-class instances.
6. Summary and Future Work
- SABR outperforms state-of-the-art methods by ~1.5-9% in conventional ZSL and ~2-14% in generalized ZSL across inductive and transductive settings on most benchmark datasets.Experiments cover AWA2, CUB, and SUN.Source: §6 · p. 8
- SABR reduces hubness with a latent space that preserves semantic relationships between labels while encoding class-discriminating information.It reduces seen-class bias through cross-validation in the inductive setting and a weak transfer constraint in the transductive setting.Source: §6 · p. 8
- SABR reaches peak transductive performance using only a fraction of the unlabeled unseen-class instances for training.Source: §6 · p. 8
- Future work could explore word-vector or combined semantic embeddings and extract label embeddings from auxiliary sources.The authors note that auxiliary sources could help extend ZSL to real-world scenarios.Source: §6 · p. 8