PaperlayerView exact revision on arXiv

Source-linked AI summary

Multiview Hessian Discriminative Sparse Coding for Image Annotation

Weifeng Liu, Dacheng Tao, Jun Cheng, Yuanyan Tang

arXiv:1307.3811v1cs.MMcs.CVcs.IT

TL;DR

Image annotation requires methods that can handle multiview features while avoiding the limited extrapolation associated with Laplacian regularization. The paper proposes mHDSC, which combines discriminative sparse coding with Hessian regularization and treats labels as an additional feature view. Experiments on PASCAL VOC’07 demonstrate mHDSC’s effectiveness compared with related sparse coding baselines.

  • Problem

    Existing discriminative sparse coding methods do not naturally handle multiview visual data, while Laplacian regularization can bias solutions toward constants and limit extrapolation.

  • Method

    mHDSC integrates Hessian regularization with discriminative multiview sparse coding and treats label information as an additional feature view.

  • Results

    mHDSC outperforms mLDSC and other related sparse coding methods in experiments on PASCAL VOC’07 image annotation.

  • Takeaways & Limitations

    The method provides an image-annotation framework that combines complementary multiview information, discriminative labels, and manifold-smooth Hessian regularization.

  • Takeaways & Limitations

    The paper identifies Laplacian regularization’s constant-function bias and resulting poor extrapolation capability as a limitation of existing approaches.

Abstract

from arXiv · show

Sparse coding represents a signal sparsely by using an overcomplete dictionary, and obtains promising performance in practical computer vision applications, especially for signal restoration tasks such as image denoising and image inpainting. In recent years, many discriminative sparse coding algorithms have been developed for classification problems, but they cannot naturally handle visual data represented by multiview features. In addition, existing sparse coding algorithms use graph Laplacian to model the local geometry of the data distribution. It has been identified that Laplacian regularization biases the solution towards a constant function which possibly leads to poor extrapolating power. In this paper, we present multiview Hessian discriminative sparse coding (mHDSC) which seamlessly integrates Hessian regularization with discriminative sparse coding for multiview learning problems. In particular, mHDSC exploits Hessian regularization to steer the solution which varies smoothly along geodesics in the manifold, and treats the label information as an additional view of feature for incorporating the discriminative power for image annotation. We conduct extensive experiments on PASCAL VOC'07 dataset and demonstrate the effectiveness of mHDSC for image annotation.

1. Introduction

Existing sparse coding methods support several computer-vision tasks but face limitations for image annotation: Laplacian regularization can weaken extrapolation, and single-view methods do not naturally exploit multiview features. The paper introduces mHDSC, combining multiview sparse coding, discrimination, and Hessian regularization, and evaluates it on PASCAL VOC’07.

  • Existing sparse coding: Sparse coding methods span reconstructive, supervised, discriminative, structured, and graph-regularized approaches for applications including clustering, classification, and dimensional reduction.These approaches respectively emphasize reconstruction, labels, class separability, structured coefficients, or local data geometry.
  • Limitations: Graph Laplacian regularization can bias solutions toward a constant function because of its constant null space, limiting extrapolation capability.The introduction identifies this as a major problem for image annotation despite Laplacian regularization’s strong performance.
  • Limitations: Existing sparse coding methods are mainly designed for single-view images, whereas image annotation commonly uses complementary multiview features.Examples of views include colour histograms, edge sketches, and local binary patterns.
  • Limitations: Feature concatenation does not efficiently exploit complementary views because it treats features with different physical characteristics improperly.The paper contrasts concatenation with methods that explicitly model multiple views.
  • Proposed method: mHDSC integrates Hessian regularization with discriminative multiview sparse coding and treats label information as an additional feature view.Hessian regularization is intended to steer solutions smoothly along manifold geodesics while preserving local similarity.
  • Evaluation: Experiments on PASCAL VOC’07 compare mHDSC with DSC, LDSC, HDSC, mSC, mDSC, and mLDSC, demonstrating effectiveness against these baselines.The introduction reports the comparison without providing numerical metrics.

2. multiview Hessian discriminative sparse coding

mHDSC integrates multiview sparse coding, Hessian regularization, and label information to support discriminative image annotation. It addresses limitations of Laplacian geometry modeling while exploiting complementary visual features and classifier-free label inference.

  • Multiview formulation: mHDSC learns an integrated sparse representation and multiview dictionary from observations across multiple feature views.Each view contributes dictionary atoms, while regularizers control sparsity, dictionary structure, and local similarity.
  • Motivation: Laplacian regularization is rejected because it biases solutions toward constant functions, limiting its suitability for local-geometry encoding.Accurate local geometry matters because images sharing semantic concepts should have nearby sparse codes.
  • Method: The proposed mHDSC uses Hessian regularization to encode local geometry in multiview feature space.The paper describes Hessian regularization as steering solutions to vary smoothly along the coordinate system.
  • Method: Labels are treated as an additional feature view, boosting dictionary discrimination and enabling label inference directly from sparse codes without classifiers.For a test image, mHDSC first obtains a sparse code and then estimates the label view from that representation.
  • Advantages: mHDSC combines complementary multiview features, second-order local-geometry information, and discriminative label modeling within sparse coding.The objective uses traditional least-squares sparse-coding loss for efficiency and simplicity.

3. Algorithms

The optimization procedure alternates among sparse-code learning, dictionary updating, and graph-coefficient learning until convergence. The paper organizes these updates into an overall alternating-optimization algorithm and supporting notation.

  • Alternating optimization: mHDSC optimization has three stages: learning sparse codes, updating the dictionary, and learning graph coefficients.Each stage optimizes one variable while the others remain fixed.
  • Notation: Table 1 lists important notation for labelled and unlabelled data, views, dictionary atoms, sparse codes, and graph coefficients.The notation distinguishes labelled-data views from unlabelled-data views.
  • Alternating optimization: The alternating procedure initializes the dictionary, then repeatedly updates dictionary and graph coefficients around sparse-code learning until convergence.Algorithm 1 summarizes this iterative procedure.

2:repeat

The subproblems use convex optimization, closed-form updates, and a weighting trick that preserves contributions from multiple views. The paper also analyzes computational cost and parallelization opportunities.

  • Subproblem optimization: Sparse-code and dictionary subproblems are decomposed by rows or views and solved using convex optimization methods based on Nesterov’s first-order framework.The relevant objectives combine convex components, with differentiability and Lipschitz continuity enabling efficient optimization.
  • Graph coefficients: The graph-coefficient subproblem is solved with a Lagrange multiplier and produces a closed-form global optimum when the other variables are fixed.The positive semidefinite Hessian matrix supports global optimality for the resulting update.
  • Graph coefficients: A direct graph-coefficient solution would select only the view with the smallest objective value, failing to exploit complementary views.The paper identifies this as a limitation of the unmodified formulation.
  • Graph coefficients: Replacing the problematic weighting term ensures that each view makes a particular contribution to the final sparse coding.This modification yields the new objective function for graph-coefficient learning.
  • Complexity analysis: The total computational cost depends on view dimensions, dictionary size, iterations, alternating iterations, and cross-validation candidates.Because matrix products dominate runtime, MapReduce and GPU computing can reduce time cost.

4. Related work

The related work covers supervised, discriminative, structured, graph-regularized, and multirepresentation sparse coding. mHDSC is positioned as combining multiview complementarity, label-based discrimination, and Hessian-based geometry modeling.

  • Discriminative sparse coding: Supervised sparse coding encodes label information through a linear predictive classifier, whereas discriminative sparse coding uses specialized discriminative terms.Examples include softmax, Fisher, and max-margin formulations.
  • Structured sparse coding: Structured sparse coding imposes structure on codes through regularizers such as group sparsity and tree-structured sparse regularization.These methods use different norms or selection structures to shape sparsity.
  • Graph regularization: Graph-regularized sparse coding models local data geometry using graph regularizations including Laplacian and hypergraph Laplacian matrices.These approaches differ in the graph regularization chosen.
  • Positioning mHDSC: mHDSC is presented as exploiting complementary multiview features, incorporating labels into sparse coding, and encoding local geometry with Hessian regularization.These design choices distinguish it from the aforementioned sparse-coding approaches.

5. Experiments

Experiments evaluate mHDSC for image annotation on PASCAL VOC’07 using multiple visual views, sparse-code classifiers, and comparisons with single-view, concatenation, and multiview methods. Results show that visual representations differ in performance, multiview methods help most with few labels, and mHDSC outperforms other multiview methods.

  • Experimental setup: The experiments use 15 visual features, including GIST, color, Hue, SIFT, and Harris-based features, with spatially decomposed histogram representations.Features are normalized per view, and experiments use uniform weights for the normalized views.
  • Experimental setup: Sparse codes are supplied to linear SVM or least-squares classifiers, while mHDSC is compared with single-view, concatenation, and multiview sparse-coding methods.Compared methods include DSC, LDSC, HDSC, mSC, mDSC, and mLDSC; AP and mAP measure annotation performance.
  • Single-view performance: Different visual representations achieve different mAP and AP performances, so the best single view is selected for comparison with multiview methods.The experiments evaluate both mAP across views and AP on selected visual object classes.
  • Multiview performance: Inference performance is compared with classifier-based methods, and the reported methods are comparable to least-squares methods.The figures include SVM, least-squares, and inference evaluation methods.
  • Multiview performance: Multiview methods significantly boost performance when labelled data are scarce, and mHDSC outperforms other multiview methods.The comparison includes multiple labelled-data settings and evaluates mAP and AP across selected classes.

6. Conclusion

The conclusion positions mHDSC as a multiview image-annotation method that addresses limitations of Laplacian-based and single-view sparse coding. Experiments on PASCAL VOC’07 report stronger performance than mLDSC and related sparse-coding methods, while future work targets additional practical applications.

  • Motivation: Laplacian-regularized sparse coding can generalize poorly because Laplacian regularization biases the solution toward a constant function.Existing sparse-coding methods also primarily handle single views despite images often having multiple visual features.
  • Proposed method: mHDSC combines multiview representation with Hessian regularization and discriminative sparse coding for image annotation.The method represents images using multiple visual features and uses Hessian regularization to explore local data geometry.
  • Proposed method: Hessian regularization helps explore local data geometry, while the method utilizes complementary information from different visual features.The conclusion identifies both properties as central components of the proposed approach.
  • Conclusion: Experiments on PASCAL VOC’07 show that mHDSC outperforms mLDSC and other related sparse-coding methods for image annotation.The method is applied with linear SVM and least-squares regression.
  • Future work: Future work will apply mHDSC to practical tasks including cartoon retrieval and cartoon classification.These applications are identified as directions for future implementation.

Figure Captions

The captions identify the mHDSC framework, example PASCAL VOC’07 images, and figures comparing mAP or AP across views and methods.

  • Framework: Figure 1 presents the mHDSC framework.The figure is captioned as the framework of the proposed method.
  • View comparisons: Figure 3 compares the mAP of different views.The experiments use this comparison to assess view-specific performance.
  • View comparisons: Figure 4 compares the AP of different views on selected classes.The figure focuses on average precision across selected visual object classes.
Loading 1307.3811v1…