PaperlayerView exact revision on arXiv

Source-linked AI summary

CliqueCNN: Deep Unsupervised Exemplar Learning

Miguel A. Bautista, Artsiom Sanakoyeu, Ekaterina Sutter, Björn Ommer

arXiv:1608.08792v1cs.CV

TL;DR

Supervised similarity labels are costly, while deep exemplar learning is impaired by a single positive, many negatives, and unreliable sample relationships. CliqueCNN forms compact cliques from reliable local distances and trains on mutually distant clique batches; it reports improved posture analysis and retrieval with finer similarity structure.

  • Problem

    Supervised similarity labels are costly, and fine-grained similarities among images, objects, and parts are difficult to annotate.

    Source: §1 · p. 1

  • Method

    CliqueCNN groups samples with reliable local similarities into compact cliques and trains the CNN to classify cliques in batches of mutually distant groups.

    Source: §1 · p. 2

  • Results

    CliqueCNN significantly improves over state-of-the-art approaches for posture analysis and retrieval, with finer similarity structure particularly useful for detailed posture analysis.

    Source: §1 · p. 2, §4 · p. 9

  • Takeaways & Limitations

    The learned representation imputes initially unavailable similarities and generalizes them to unseen data.

    Source: §1 · p. 2

    Most initial similarity estimates are unreliable; the method can reliably identify highly similar and far-away samples, but most samples cannot be properly ranked against an exemplar.

    Source: §2.1 · p. 4

Abstract

from arXiv · show

Exemplar learning is a powerful paradigm for discovering visual similarities in an unsupervised manner. In this context, however, the recent breakthrough in deep learning could not yet unfold its full potential. With only a single positive sample, a great imbalance between one positive and many negatives, and unreliable relationships between most samples, training of Convolutional Neural networks is impaired. Given weak estimates of local distance we propose a single optimization problem to extract batches of samples with mutually consistent relations. Conflicting relations are distributed over different batches and similar samples are grouped into compact cliques. Learning exemplar similarities is framed as a sequence of clique categorization tasks. The CNN then consolidates transitivity relations within and between cliques and learns a single representation for all samples without the need for labels. The proposed unsupervised approach has shown competitive performance on detailed posture analysis and object classification.

1 Introduction

Visual similarity learning supports many vision tasks, but supervised similarity labels are costly and fine-grained relationships are difficult to annotate. CliqueCNN groups samples into compact, mutually distant cliques for batch-wise classification, and the paper reports improved posture analysis and retrieval.

1.1 Exemplar Based Methods for Similarity Learning

  • Exemplar methods split category-level recognition into simpler subtasks, training separate classifiers to learn individual exemplars’ similarities against a large set of negatives; they have been used in segmentation, grouping, retrieval, and object recognition.Source: §1 · p. 1

1 Introduction

  • Unsupervised deep similarity learning is attractive because supervised labels are costly and fine-grained similarities among images, objects, and parts are difficult to annotate.CNN exemplar learning is also impaired by a single positive, many negatives, and mostly unknown or contradictory sample relationships.Source: §1 · pp. 1–2
  • CliqueCNN significantly improves over state-of-the-art approaches for posture analysis and retrieval by learning a general feature representation for human pose.The method organizes reliable local similarities into compact cliques and trains on batches of mutually distant cliques, leaving unreliable pairs out of the same batch.Source: §1 · p. 2

1.1 Exemplar Based Methods for Similarity Learning

  • Supervised pair- and triplet-based similarity learning requires supervisory information that scales quadratically or cubically, while clique-based batches define a multi-class task with lower training cost.The related methods use pairs [36] or triplets [32]; the cited context-based approach requires all pairwise image relationships [6].Source: §1.1 · pp. 2–3

2 Approach

The approach turns weak local similarity estimates into compact cliques and batches of mutually consistent cliques, then trains a CNN to learn a shared representation. Repeatedly updating similarities, cliques, and batches lets the representation extend relations to samples whose initial similarities were unreliable.

2 Approach

  • With only one positive exemplar, CNN training faces unmodeled within-class variance, a highly imbalanced positive-to-negative ratio, and potentially inconsistent labels across similar samples.The proposed approach addresses these issues through compact cliques and batches that maximize within-clique similarity while minimizing inter-clique similarity.Source: §2 · p. 3
  • The proposed method outperforms 1-sample CNN and NN-CNN for Olympic Sports posture retrieval; iterative updates increase reliable within- and between-clique similarities and dissimilarities per batch by a factor of 2.34 over initialization.The learned similarity matrix also becomes more structured, with noisy relations disappearing and the matrix representable using fewer basis vectors.Source: §2 · p. 3, §2.5 · p. 6

2.1 Initialization

  • Only the most similar and most distant initial HOG-LDA relationships are treated as reliable; most samples cannot be ranked reliably against an exemplar.The initial similarity s_ij is the dot product of the HOG-LDA representations of samples d_i and d_j.Source: §2.1 · p. 4

2.2 Compact Cliques

  • Complete-linkage clustering builds mutually similar cliques around each exemplar, and farthest-neighbor clustering merges highly overlapping cliques while retaining compactness.The merge process stops when a cluster's intra-clique similarity falls below half that of its constituent cliques.Source: §2.2 · p. 4

2.3 Selecting Batches of Mutually Consistent Cliques

  • The batch optimization assigns cliques to batches that are mutually distant while maximizing sample coverage within each batch and across all batches.The number of cliques per batch is controlled by r; the binary assignment problem is relaxed toward boundary-valued solutions because the original problem is NP-hard.Source: §2.3 · pp. 4–5

2.4 CNN Training

  • CNN training treats each clique's samples as one class and alternates among batches, making clique categorization a set of smaller pretext tasks.The model computes similarities from correlations between learned CNN feature representations.Source: §2.4 · pp. 5–6

2.5 Similarity Imputation

  • Alternating across batches lets the CNN consolidate clique transitivity and generalize from initial cliques to previously unreliable relations between samples.After each training round, learned similarities support larger cliques and batches; the process alternates CNN training with recomputation of cliques and batches.Source: §2.5 · p. 6

3 Experimental Evaluation

The evaluation tests unsupervised visual similarity learning on Olympic Sports posture analysis, Leeds pose estimation, and PASCAL VOC object classification. Results include improved Olympic Sports performance, competitive Leeds pose estimation, and a PASCAL gain without PASCAL supervision or fine-tuning.

3 Experimental Evaluation

  • Olympic Sports average AuC improves by at least 10% over other methods; on Leeds, the unsupervised method is competitive with the ground-truth-similarity upper bound; PASCAL classification gains 3% without supervision or fine-tuning.These results span posture analysis, pose estimation, and object classification.Source: §3.1 · p. 7, Table 2 · p. 8, §3.2 · p. 8

3.1 Olympic Sports Dataset: Posture Analysis

  • The 1-sample CNN fails under positive-negative imbalance; NN-CNN gains 3% over it, while compact-clique grouping yields a 12% relative increase over NN-CNN.Olympic Sports testing uses manually labeled positive and negative examples solely for evaluation.Source: §3.1 · p. 7
  • Compact-clique neighbors produce sharper average postures that retain more detail and resemble the original exemplar more closely than the other methods.The comparison blends each method's top 50 nearest neighbors for a randomly chosen Olympic Sports exemplar.Source: §3.1 · p. 7

3.2 Leeds Sports Dataset: Pose Estimation

  • For Leeds pose estimation, the method retrieves a similar training frame and uses its pose to estimate the query pose despite having no joint labels during training.Training uses 1,000 Leeds images and 4,000 extended-dataset images, each annotated with 14 joint locations.Source: §3.2 · pp. 7–8
  • Pose matching can be invariant to front-back flips, transferring pose across facing directions but also confusing people facing toward and away from the camera.One successful example receives a PCP score of 0.6; the authors suggest additional training data or face-detection features could resolve the confusion.Source: §3.2 · p. 8, Figure 6 · p. 8

3.3 PASCAL VOC 2007: Object Classification

  • On PASCAL VOC 2007, the method improves initial unsupervised similarities by 3% without using PASCAL labels or fine-tuning on PASCAL.The evaluation classifies object bounding boxes using k = 5 nearest neighbors.Source: §3.3 · pp. 8–9

4 Conclusion

CliqueCNN makes unsupervised CNN similarity learning applicable by grouping samples into compact, mutually dissimilar cliques and treating each batch as a categorization task. It reports competitive state-of-the-art performance and finer similarity structure, particularly for detailed posture analysis.

4 Conclusion

  • CliqueCNN learns similarities among large numbers of exemplars with CNNs without supervision.Source: §4 · p. 9
  • A single cost function produces SGD batches of compact, mutually dissimilar cliques for categorization.Source: §4 · p. 9
  • The approach shows competitive state-of-the-art performance and finer similarity structure, particularly for detailed posture analysis.Source: §4 · p. 9

A ROCs for Olympic Sports

Across Olympic Sports categories, CliqueCNN improves on the compared CNN variants. The passage attributes the gains to better handling of within-class variability and the fine-grained structure of negative samples.

A ROCs for Olympic Sports

  • 12% relative performance increase over NN-CNN is reported for CliqueCNN.The passage attributes the gain to computing sets of mutually distant compact cliques.Source: A ROCs for Olympic Sports · p. 12
  • 3% performance increase over the 1-sample CNN is reported for NN-CNN.Additional nearest neighbours better model within-class variability of the exemplar.Source: A ROCs for Olympic Sports · p. 12
  • The 1-sample CNN fails to model the positive distribution under the imbalance of positives and negatives, while NN-CNN's unstructured negative set limits fine-grained similarity learning.Source: A ROCs for Olympic Sports · p. 12
  • CliqueCNN's gains are reported as consistent across all Olympic Sports classes.Source: A ROCs for Olympic Sports · p. 12

B Qualitative Results

Qualitative retrieval examples are presented for Olympic Sports, Leeds Sports, and VOC2007. The figures show nearest neighbours retrieved by CliqueCNN for representative queries.

B Qualitative Results

Loading 1608.08792v1…