PaperlayerView exact revision on arXiv

Source-linked AI summary

CNN-Based Joint Clustering and Representation Learning with Feature Drift Compensation for Large-Scale Image Data

Chih-Chung Hsu, Chia-Wen Lin

arXiv:1705.07091v2cs.CV

TL;DR

Large-scale clustering of unlabeled images must be efficient while preserving useful visual representations. The paper jointly learns representations and clusters with a CNN using mini-batch k-means and compensates for feature drift; experiments report strong clustering performance and scalability on image datasets containing millions of images.

  • Problem

    Large unlabeled image sets are difficult to cluster efficiently using visual representations, while feature encoding can reduce representational power and deep networks lack the labeled data usually used to learn features.

    Source: §I · p. 1, B. Contribution of Proposed Method · p. 2

  • Method

    The proposed CCNN jointly learns image representations and clusters through iterative mini-batch k-means, with centroid updates that compensate for feature drift across iterations.

    Source: B. Contribution of Proposed Method · pp. 2–3, D. Compensation of Feature Drift · p. 5

  • Results

    The method outperforms state-of-the-art clustering schemes in accuracy and storage complexity on large-scale image sets containing millions of images.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The mini-batch framework handles image clustering at million-image scale on a personal computer with reasonable memory and computation costs.

    Source: §5 · p. 8

Abstract

from arXiv · show

Given a large unlabeled set of images, how to efficiently and effectively group them into clusters based on extracted visual representations remains a challenging problem. To address this problem, we propose a convolutional neural network (CNN) to jointly solve clustering and representation learning in an iterative manner. In the proposed method, given an input image set, we first randomly pick k samples and extract their features as initial cluster centroids using the proposed CNN with an initial model pre-trained from the ImageNet dataset. Mini-batch k-means is then performed to assign cluster labels to individual input samples for a mini-batch of images randomly sampled from the input image set until all images are processed. Subsequently, the proposed CNN simultaneously updates the parameters of the proposed CNN and the centroids of image clusters iteratively based on stochastic gradient descent. We also proposed a feature drift compensation scheme to mitigate the drift error caused by feature mismatch in representation learning. Experimental results demonstrate the proposed method outperforms start-of-the-art clustering schemes in terms of accuracy and storage complexity on large-scale image sets containing millions of images.

I. INTRODUCTION

Large-scale image clustering must balance computational cost with useful visual representations. Deep networks offer a route to stronger features, but unlabeled clustering lacks the training data they typically rely on, and existing approaches have scale or representation constraints.

I. INTRODUCTION

  • Large-scale image clustering is important for image-processing and vision applications, but organizing the growing volume of cloud images efficiently remains challenging.Source: §I · p. 1
  • Feature encoding such as hashing reduces dimensionality and makes large-scale clustering possible, but may weaken representational power and require redesign when feature dimensions or ranges change.Source: §I · p. 1
  • Hierarchical clustering is computationally expensive on large datasets, while centroid-based methods use less memory and computation but depend strongly on feature representational power.Source: §I · p. 1

A. Deep-Learning Based Image Clustering

  • Deep networks learn discriminative visual representations with large labeled datasets, but such labels are unavailable for unsupervised clustering and pretrained models may not fit the input data.Source: A. Deep-Learning Based Image Clustering · pp. 1–2
  • Prior deep clustering methods face image-feature or scaling constraints: AutoEncoders struggle with high-dimensional images, CNN-CM uses costly full-set k-means, and CNN-RC requires a high-cost affinity matrix.Source: A. Deep-Learning Based Image Clustering · p. 2

B. Contribution of Proposed Method

The proposed CCNN combines salient-region feature extraction with mini-batch clustering and iterative representation learning. Feature-drift compensation addresses mismatches between successive representations during centroid updates.

B. Contribution of Proposed Method

  • The framework integrates mini-batch k-means with CNNs for large-scale clustering and introduces iterative centroid updating to address feature drift between representation-learning iterations.Source: B. Contribution of Proposed Method · pp. 2–3

II. PROPOSED CLUSTERING CNN ARCHITECTURE

  • CCNN uses AlexNet’s first five convolutional layers followed by three adaptation layers; global max-pooling on Conv8 yields k features from salient image regions without bounding-box training labels.Source: §II · p. 3

III. JOINT CLUSTERING AND REPRESENTATION LEARNING BASED ON MINI-BATCH K-MEANS

  • The method initializes centroids from features of k randomly selected images, then alternates mini-batch k-means label assignment with SGD updates to network parameters and centroids.Source: §III · pp. 3–4
  • Because network updates change extracted features across iterations, the method tracks successive features backward to compensate for mismatch and reduce gradient drift during centroid updates.Source: §III · p. 4

A. Initialization of CCNN

The method initializes CCNN from ImageNet-pretrained AlexNet layers, then learns its FC9 and Softmax parameters using mini-batch cluster labels as pseudo-ground truth.

A. Initialization of CCNN

  • Conv1–Conv5 are initialized from AlexNet pretrained on ImageNet, then concatenated with the remaining CCNN layers to provide the initial model for clustering tasks.Source: A. Initialization of CCNN · p. 4

B. Representation Learning

  • FC9 converts local salient features into clustering features, while SGD updates FC9 and Softmax weights using mini-batch k-means labels as pseudo-ground truth.Source: B. Representation Learning · p. 4
  • The network’s representation-learning update is defined through gradients of the squared-error objective, with ReLU activations and chain-rule derivatives used to calculate weight updates.Source: B. Representation Learning · p. 4

C. Cluster Centroid Updating

Cluster centroids are initialized to encourage diversity, then updated online from mini-batch assignments using per-centroid learning rates.

C. Cluster Centroid Updating

D. Compensation of Feature Drift

Feature representations can change between iterations as CCNN parameters update, making centroid updates unreliable and potentially degrading clustering. The method backward-tracks features to maintain consistency during centroid updates.

D. Compensation of Feature Drift

  • Changing CCNN parameters make the same image’s features time-varying, which can make centroid updates unreliable and degrade clustering.The proposed approach addresses this mismatch by maintaining feature consistency across successive iterations.Source: D. Compensation of Feature Drift · p. 5
  • Backward-tracked features replace newly extracted features in the centroid update to maintain consistency across successive iterations.The method derives the earlier-iteration feature from the weights obtained in the current iteration.Source: D. Compensation of Feature Drift · p. 5
  • The mini-batch scheme is intended to support large-scale clustering on a single personal computer with reasonable computational and memory complexity.After several epochs, cluster labels are expected to converge more reliably, while fine-tuned network parameters provide improved visual representations.Source: D. Compensation of Feature Drift · p. 5

E. Top-𝑘𝑚 Based Parameter Updating

CCNN parameter updates use selected mini-batch samples, balancing update speed against clustering performance and training time.

E. Top-𝑘𝑚 Based Parameter Updating

  • Only the top-k_m mini-batch samples closest to their assigned centroids are selected to update CCNN parameters.This selection addresses the possibility that some predicted cluster labels are unreliable when network parameters are inaccurate.Source: E. Top-𝑘𝑚 Based Parameter Updating · p. 5
  • CCNN parameters are updated after collecting N_m samples across N_m/k_m mini-batch clustering iterations.The mini-batch size is N_m.Source: E. Top-𝑘𝑚 Based Parameter Updating · p. 5
  • A higher k_m speeds parameter updates but can lower clustering performance, while a smaller value improves performance but increases update count and training time.The experiments empirically set k_m to 10.Source: E. Top-𝑘𝑚 Based Parameter Updating · p. 5

A. Experiment Setup

The experiments compare the proposed method with deep-learning-based clustering schemes and report NMI, runtime, and memory comparisons across image datasets. The setup also includes MNIST and specifies image preprocessing and computing hardware.

A. Experiment Setup

  • The comparison includes DEC, CNN-CM, and CNN-RC as deep-learning-based image clustering schemes.The source notes that these three schemes cannot deal with…; the supplied passage does not complete that statement.Source: A. Experiment Setup · p. 5
  • Table II compares NMI performance across ILSVRC-Val, Places-Val, and Places-Train.The supplied table entries include results for DEC, CNN-CM, and CNN-RC.Source: Table II · p. 6, A. Experiment Setup · p. 6
  • Table III compares runtime and memory costs across three image datasets.The caption identifies the comparison scope but the supplied passages do not provide its values.Source: Table III · p. 6
  • Test images were center-cropped to 256×256, while the 60,000 MNIST images were not cropped because they are 28×28 grayscale images.The reported implementation used TensorFlow on an Intel Core i7-4770 PC with 32 GB RAM and an NVIDIA Titan X GPU with 12 GB GPU RAM.Source: A. Experiment Setup · p. 6

B. Performance Evaluation

CCNN improves clustering against the stated baselines, handles a million-image dataset on a personal computer, and performs competitively on MNIST.

B. Performance Evaluation

  • Joint optimization raises NMI by 0.13–0.14 over Baseline-II and 0.08–0.10 over Baseline-III, while outperforming CNN-CM and DCC on ILSVRC-Val and Places-Val.Its NMI is comparable to CNN-RC on those datasets.Source: B. Performance Evaluation · p. 6
  • CCNN clustered Places-Train using about 8GB GPU memory and 43 hours on Titan X, while DEC, CNN-CM, and CNN-RC failed at this task.The paper attributes other methods' difficulty to high computation or memory requirements, including full-set k-means and affinity-matrix construction.Source: B. Performance Evaluation · pp. 6–7
  • On MNIST, the proposed method significantly outperforms k-means and achieves performance comparable to CNN-SF/CNN-RC.The evaluation uses a simplified CCNN with some convolution layers removed for MNIST's smaller image resolution.Source: B. Performance Evaluation · p. 7, Table IV · p. 7
  • CCNN outperforms AlexNet across all three tested clustering schemes, and Conv8 feature maps roughly localize salient regions for subsequent representation learning.The authors identify salient-region learning as one reason for the clustering advantage.Source: B. Performance Evaluation · p. 7, Table V · p. 7, Figure 5 · p. 7

C. Impacts of 𝑘𝑚 and the number of Epochs

The choice of k_m trades clustering performance against computation, while pretraining reaches saturation sooner than random initialization; the authors recommend no more than 10 fine-tuning epochs.

C. Impacts of 𝑘𝑚 and the number of Epochs

D. Convergence Analysis of CCNN

Experiments show steady NMI increases with training epochs, although the method lacks a theoretical convergence guarantee; the conclusion emphasizes salient-feature learning, scalable joint updating, and feature-drift compensation.

D. Convergence Analysis of CCNN

  • The method has no theoretical convergence guarantee, but experiments show NMI increasing with epochs for both pretrained and random initialization, with pretrained performance higher.The authors describe the empirical convergence as reasonable for either initialization.Source: D. Convergence Analysis of CCNN · p. 8

5. CONCLUSION

  • The conclusion presents CCNN as extracting salient features and combining mini-batch representation learning with centroid updates and feature-drift compensation for clustering up to millions of images.The stated aim is reasonable memory and computation costs on several public image datasets.Source: §5 · p. 8

Related papers

Loading 1705.07091v2…