Source-linked AI summary
Generalized Zero-Shot Learning Via Over-Complete Distribution
Rohit Keshari, Richa Singh, Mayank Vatsa
TL;DR
Existing supervised DNNs can perform poorly on classes absent from training, and synthetic unseen-class samples may miss hard cases. The paper generates over-complete distributions and applies OBTL and center loss; it reports improved performance across ZSL and GZSL benchmarks.
Problem
Synthetic unseen-class distributions may fail to mimic real unseen data, including hard samples, limiting their usefulness for training discriminative classifiers.
Source: §1 · p. 2Method
The paper uses a CVAE to generate OCDs with hard samples near competing classes, then applies OBTL and center loss to encourage separability and reduce within-class spread.
Source: §1 · p. 2, §3.1 · p. 3Results
Experiments on three benchmark databases with ZSL and GZSL protocols show improved performance for the proposed approach.
Source: §5 · p. 8Takeaways & Limitations
The authors suggest OCDs with inter-class and intra-class distance optimization may also be used in GANs, heterogeneous metric learning, and face recognition with disguise variations.
Source: §5 · p. 8OCD generation assumes class attributes represent classes and are separated in attribute space, and keeps those attributes unchanged.
Source: §3.1 · p. 4
Abstract
from arXiv · showhide
A well trained and generalized deep neural network (DNN) should be robust to both seen and unseen classes. However, the performance of most of the existing supervised DNN algorithms degrade for classes which are unseen in the training set. To learn a discriminative classifier which yields good performance in Zero-Shot Learning (ZSL) settings, we propose to generate an Over-Complete Distribution (OCD) using Conditional Variational Autoencoder (CVAE) of both seen and unseen classes. In order to enforce the separability between classes and reduce the class scatter, we propose the use of Online Batch Triplet Loss (OBTL) and Center Loss (CL) on the generated OCD. The effectiveness of the framework is evaluated using both Zero-Shot Learning and Generalized Zero-Shot Learning protocols on three publicly available benchmark databases, SUN, CUB and AWA2. The results show that generating over-complete distributions and enforcing the classifier to learn a transform function from overlapping to non-overlapping distributions can improve the performance on both seen and unseen classes.
1. Introduction
Small-sample training can leave DNNs prone to overfitting, while ZSL tests on classes absent from training. The paper proposes generating hard, over-complete samples and using losses to improve class separability.
1. Introduction
- DNNs can overfit small-sample training sets, and ZSL addresses test classes with no training examples.The introduction also names transfer learning, life-long learning, self-taught learning, and one-shot learning as approaches to generalization.Source: §1 · p. 1
- ZSL evaluates performance on unseen classes, whereas GZSL tests on both seen and unseen classes and seeks performance across both sets [39].The introduction presents these as the two protocols for classification with limited or no training data.Source: §1 · p. 1
- Synthetic unseen-class samples may fail to mimic real unseen distributions, including their hard samples, so training on them does not necessarily improve performance.Source: §1 · p. 2
- The paper proposes OCDs containing challenging samples near other classes, with OBTL for inter-class separability and CL to reduce within-class spread.The authors report that training on OCDs can improve separability of seen and unseen classes in the learned feature space.Source: §1 · p. 2
2. Related Work
Prior ZSL work learns mappings to model, attribute, or semantic spaces, while GZSL research includes synthetic-data generation and other approaches to reduce bias. The paper frames generalization and bias across seen and unseen classes as remaining challenges.
2. Related Work
- ZSL methods learn mappings into model or attribute spaces, including ALE, DCN, and DAP; Xian et al. proposed a more challenging evaluation protocol.Source: §2 · p. 2
- For GZSL, Verma et al. used a CVAE and a supervised linear SVM, while Gao et al. combined CVAE and GAN with semantic-similarity preservation during reconstruction.Source: §2 · p. 2
- Other GZSL approaches include cGAN/RAS synthesis, triplet loss for class-level over-fitting, UVDS with diffusion regularization, expert gating, semantic-to-visual adversarial mapping, and VAE distribution alignment.Source: §2 · p. 2
- The paper identifies generalization on test data and reducing bias across seen and unseen classes as continuing challenges.Source: §2 · p. 2
3. Proposed Framework
OCD-CVAE combines attribute-conditioned generation with an attribute regressor trained using OBTL and center loss. Its over-complete distributions introduce hard samples near competing classes through a three-phase training framework.
3. Proposed Framework
- The framework combines an attribute-conditioned CVAE encoder and decoder with a regressor trained using Online Batch Triplet Loss and center loss.The regressor maps generated samples to predicted attributes.Source: §3 · p. 2
3.1. Over-Complete Distribution (OCD)
- OCD generates hard samples near competing classes by shifting the latent distribution toward other classes while keeping class attributes unchanged.The over-complete distribution has a larger standard deviation than the approximated distribution.Source: §3.1 · pp. 3–4
3.2. Proposed OCD-CVAE Framework Training
- Training proceeds in three phases: CVAE optimization, regressor training with OBTL and center loss, then generator and regressor training with additional losses.The third phase uses discriminator-driven and regularization losses alongside OBTL and center loss.Source: §3.2 · p. 4, §3.2.2 · pp. 5–6
3.2.1 Loss Functions
- OBTL minimizes batch-generated triplets, reducing hard-negative search space and training time, while center loss minimizes deviation from class centers.Triplet loss is used to increase inter-class distance and decrease intra-class distance.Source: §3.2.1 · p. 5
3.2.2 Learning Phase of the Proposed Model
- The first phase trains the CVAE on seen data using conditional marginal likelihood and KL-divergence; the second trains the regressor to map synthetic samples to attributes.In the third phase, discriminator prediction and regularization losses train the generator, with regularization intended to retain class-specific samples under random latent sampling.Source: §3.2.2 · pp. 5–6
3.3. Implementation Details
- Implementation uses batch size 256, with hard-sample parameters µHP = 0, σHP = 0.12, and σ′HP = 0.5.The reported µ size is 256 × 100, with row-wise shuffling.Source: §3.3 · p. 6
4. Experimental Results and Analysis
Experiments on AWA2, CUB, and SUN evaluate conventional ZSL and GZSL, then examine component ablations and hyper-parameter choices. The reported results show benchmark improvements, with performance varying by dataset and protocol.
4.1. Database Details
- The experiments use AWA2, with 37,322 samples and 85-dimensional attributes; CUB has 11,788 images and 312-dimensional attributes, while SUN has 14,204 samples and 102-dimensional attributes.The paper describes AWA2 attributes as manually marked by human experts.Source: §4.1 · p. 6
4.2. Evaluation Protocol
- ZSL generates OCD for unseen classes and reports unseen-class accuracy on standard and proposed splits; GZSL trains with seen and synthetically generated unseen classes and tests both.The GZSL evaluation uses an 80–20 seen-class train-test split and reports average per-class accuracies for protocols A and B.Source: §4.2 · p. 6
4.3. Conventional Zero-Shot Learning (ZSL); 4.5. Generalized Zero-Shot Learning (GZSL)
- On PS ZSL, OCD-CVAE improves state-of-the-art accuracy by 1.8% on AWA2, 0.1% on CUB, and 0.7% on SUN; in GZSL, it improves AWA2 H by 1.8%.For GZSL, the method is among the top three on the more challenging unseen-class protocol across all three datasets and among the two best on SUN and CUB.Source: §4.3 · p. 6, Table 2 · p. 7, §4.5 · p. 7
4.4. Ablative Study
- OCD+OBTL+CL gives the best ablation results, followed by OCD+OBTL, while OBTL-only and CL-only perform poorly.The authors attribute the standalone losses’ poor performance to insufficient hard samples for gradient backpropagation.Source: §4.4 · p. 7, Table 3 · p. 7
4.5. Generalized Zero-Shot Learning (GZSL)
- In GZSL, seen-class test results on protocol B are better than unseen-class results on protocol A.The evaluation reports average per-class accuracies for A and B and their harmonic mean H.Source: §4.5 · p. 7
4.6. Hyper-Parameter Selection
- With OCD, generating 400 to 600 samples improves performance; without OCD, increasing the sample count does not affect it.The selected standard deviation is 0.5, which the experiments report as yielding the best results.Source: §4.6 · pp. 7–8, Figure 5 · p. 8
5. Conclusion
The paper uses over-complete distributions of hard samples to train discriminative classifiers for ZSL and GZSL. Experiments report improved performance, and the authors note potential use in other frameworks and applications.
5. Conclusion
- OCD generates all possible hard samples for a class that are closer to competing classes, and is used to train discriminative classifiers for ZSL and GZSL.Source: §5 · p. 8
- Experiments on three benchmark databases with both ZSL and GZSL protocols show improved performance for the proposed approach.Source: §5 · p. 8
- The authors report that OCD helps ensure class separability and improves classification performance.Source: §5 · p. 8
- The authors identify potential applications of OCD with inter-class and intra-class distance optimization in GANs, heterogeneous metric learning, and face recognition with disguise variations.Source: §5 · p. 8