PaperlayerView exact revision on arXiv

Source-linked AI summary

Gabor Convolutional Networks

Shangzhen Luan, Baochang Zhang, Chen Chen, Xianbin Cao, Jungong Han, Jianzhuang Liu

arXiv:1705.01450v4cs.CV

TL;DR

DCNNs have not well explored the steerable properties of traditional filters for handling orientation and scale changes. The paper introduces GCNs, which modulate learned convolution filters with Gabor filters, and reports improved benchmark performance with fewer learnable parameters.

  • Problem

    DCNNs have limited use of traditional filters' steerable properties for handling orientation and scale changes, while their learned models can be costly and parameter-heavy.

  • Method

    GCNs create Gabor orientation filters by modulating learned convolution filters with Gabor filter banks across orientations and scales, replacing the basic convolution element.

  • Results

    GCNs are reported to improve classification performance across benchmarks while using fewer parameters, including 3.04% versus 3.16% Top-5 error and 11.46% versus 11.94% Top-1 error with 35.85M versus 44.54M parameters.

  • Takeaways & Limitations

    The proposed GCNs are compatible with different network architectures and provide compact deep models with enhanced robustness to orientation and scale variations.

Abstract

from arXiv · show

Steerable properties dominate the design of traditional filters, e.g., Gabor filters, and endow features the capability of dealing with spatial transformations. However, such excellent properties have not been well explored in the popular deep convolutional neural networks (DCNNs). In this paper, we propose a new deep model, termed Gabor Convolutional Networks (GCNs or Gabor CNNs), which incorporates Gabor filters into DCNNs to enhance the resistance of deep learned features to the orientation and scale changes. By only manipulating the basic element of DCNNs based on Gabor filters, i.e., the convolution operator, GCNs can be easily implemented and are compatible with any popular deep learning architecture. Experimental results demonstrate the super capability of our algorithm in recognizing objects, where the scale and rotation changes occur frequently. The proposed GCNs have much fewer learnable network parameters, and thus is easier to train with an end-to-end pipeline.

I. INTRODUCTION

The paper identifies limited transformation modeling and costly redundancy in DCNNs, then proposes modulating learned convolution filters with Gabor filters to improve robustness while reducing parameters.

  • Gabor filters characterize spatial frequency while preserving spatial relations, providing orientation-dependent frequency contents with steerable and scalable kernels.
  • DCNNs learn powerful representations from raw pixels but typically require millions of parameters, extensive computation, and augmentation for geometric transformations.
  • GCNs replace ordinary convolution filters with Gabor orientation filters formed by modulating learned filters with Gabor filters at multiple orientations and scales.
  • The modulation adds spatial localization, orientation selectivity, and spatial frequency selectivity to output feature maps while inheriting steerable properties.
  • GoFs can be incorporated into different network architectures, and GCNs are reported to achieve state-of-the-art performance across various benchmarks with CNN and ResNet backbones.
  • GCNs reduce learnable filters through predefined Gabor scales and orientations, producing more compact networks while maintaining high feature representation capacity.

A. Gabor Filters

Gabor wavelets provide a fixed, steerable basis for analyzing spatial-frequency structure, while transformation-based augmentation can increase network cost and overfitting risk.

  • Gabor wavelets use complex functions as a basis for Fourier transforms and have minimized standard-deviation products in time and frequency domains.
  • Gabor filters model receptive fields of visual-cortex simple cells and are used to characterize spatial frequency while preserving spatial relations.
  • The formulation defines v as frequency and u as orientation, with Gabor filters indexed across scales and orientations.
  • Learning all transformations through data augmentation can require many network parameters, increasing training cost and over-fitting risk.

1) Spatial Transformer Networks:

Prior approaches improve transformation handling through feature-map manipulation, deformable sampling, fixed bases, or Gabor preprocessing; GCNs instead modulate learned filters directly.

  • 1) Spatial Transformer Networks:: Spatial Transformer Networks manipulate feature maps using transform matrices estimated by a localization sub-CNN, but complex transformation parameters may be difficult to estimate precisely.
  • ORN encodes hierarchical orientation responses, but its orientation invariance is not guaranteed for larger filters and its reported gains over TI-Pooling are marginal.
  • Deformable filters enhance geometric-transformation modeling by learning offsets for free-form sampling-grid deformation, but their operation is complicated and favors small filters.
  • Wavelet scattering networks use weighted sums over fixed bases, whereas GCNs change convolution filters in a steerable way using Gabor filters.
  • Gabor-CNN preprocessing methods apply Gabor filters to images first and then feed the resulting feature images into CNNs, unlike GCN filter modulation.

III. GABOR CONVOLUTIONAL NETWORKS

GCNs replace standard convolution filters with steerable Gabor orientation filters, encoding orientation in learned filters and scale across layers. This design reduces learnable filter parameters while enhancing feature maps with orientation and scale information.

  • Convolutional Gabor Orientation Filters: GCNs use Gabor orientation filters (GoFs), created by modulating learned convolution filters with Gabor filter banks.GoFs are steerable filters used throughout the network to produce enhanced feature maps.
  • Orientation and Scale Encoding: Gabor filters provide U orientations and V scales; GCNs encode orientation in learned filters and embed scale information across layers.The resulting convolution features explicitly capture orientation and scale information.
  • Learned Filter Representation: GCN learned weights have dimensions Cout × Cin × N × W × W, with N set to U to preserve feature-map channel consistency.U is the number of Gabor orientations used to modulate each learned filter.
  • Filter Modulation: A GoF is formed by applying U Gabor filters at a given scale to each learned filter through element-by-element modulation.The modulated filters are indexed by orientation and scale.
  • Multiple Feature Maps: The forward convolution process uses 10 input feature maps and produces 20 output feature maps, with reconstructed filters divided into 20 groups of 10.This grouping organizes the reconstructed filters for multiple feature maps.

B. GCN Convolution

GCN convolution applies GoFs to input feature maps and retains orientation responses in the output. With four Gabor orientations, 10 GoFs transform a 1×4×32×32 input into a 10 × 4 × 30 × 30 output.

  • Forward Convolution: GCNs use GoFs to produce feature maps that explicitly enhance scale and orientation information in deep features.The input feature map is convolved with each GoF.
  • Orientation Responses: Each output GoF produces orientation-specific responses, with the kth output channel representing the kth orientation response.The convolution sums responses across the input channels for the corresponding GoF.
  • Output Dimensions: A 1×4×32×32 input convolved with 10 GoFs having 4 Gabor orientations produces a 10 × 4 × 30 × 30 output feature map.The example preserves four orientation channels for each of the ten GoFs.
  • Visualization: In the first convolution layer, each GoF group and its corresponding feature map are visualized with matching colors across four orientation channels.The blue-rectangle examples show that GoFs carry varied orientation information.

C. Updating GoF

GCNs perform forward computation with GoFs but save and update only the underlying learned filters. Back-propagation sums sub-filter gradients into those learned filters, producing a compact update procedure.

  • Parameter Updates: GCNs use GoFs during forward calculation, while only the underlying learned filters are stored and updated.This separates the filters used for computation from the parameters retained by the model.
  • Gradient Accumulation: Back-propagation sums gradients from GoF sub-filters into their corresponding learned filters before applying the parameter update.The learned filter is updated using the loss gradient and learning rate.
  • Model Properties: Updating only learned convolution filters makes the GCN model more compact and efficient, according to the paper.The authors also state that this procedure is more robust to orientation and scale variations.

IV. IMPLEMENTATION AND EXPERIMENTS

The experiments evaluate GCNs across rotated-digit and image benchmarks, varying scales, orientations, model sizes, and training efficiency. Results favor multiple Gabor scales and moderate orientation counts, while GCNs reduce complexity and improve error rates against baselines.

  • Scale and orientation settings: Using V = 4 Gabor scales yields lower MNIST error rates than using a single scale across different convolution filter sizes.Larger Gabor filters are used in shallow layers, while deeper ResNet experiments also use V = 4.
  • Scale and orientation settings: GCNs perform better with 3 to 6 orientations when V = 4, while U = 4 or 5 produces more stable results.Too few orientations may extract insufficient features, whereas too many may make the network overly complex.
  • Efficiency and accuracy: GCNs achieve better performance with 3x3 kernels while using 1/12 or 1/4 as many parameters as CNNs.The parameter size is linear in orientation channels but quadratic in layer width, enabling complexity reduction.
  • Efficiency and accuracy: GCNs with 5x5 and 7x7 kernels achieve 1.10% error on MNIST-rot and 0.42% error on MNIST, respectively, outperforming ORNs.The passage attributes this result to larger kernels carrying more Gabor-orientation information.

B. SVHN

On SVHN and CIFAR datasets, GCNs improve or match baseline performance while using comparable or smaller models. Improvements are especially associated with scale variation and larger-kernel configurations.

  • SVHN: SVHN contains 32x32 digit images with illumination changes, rotations, complex backgrounds, and 600000 total images.The dataset includes 73257 training digits, 26032 test digits, and 531131 additional images, though the additional images are not used by all methods.
  • SVHN: GCN-ResNet achieves 1.1% and 0.66% improvements over ResNet and ORNs, respectively, with similar parameter sizes.GCNs also outperform the Gaussian-mask variant GCN4-Gau, supporting the contribution of Gabor filters.
  • CIFAR datasets: On CIFAR-10, GCNs consistently improve performance versus baseline ResNet across different parameter counts.The experiments compare GCN-ResNet with NIN, VGG, ORN, and ResNet.
  • CIFAR datasets: GCNs reach 3.88% versus 4% error on CIFAR-10 while using half the WRN model size.CIFAR-100 also shows improvement at similar parameter sizes.
  • CIFAR datasets: Increasing kernel configurations from 3x3 to 5x5 improves CIFAR error rate from 6.34% to 5.62%.The model size increases with the larger-kernel configuration.

D. Large Size Image Classification

The large-image experiments test GCNs on ImageNet, where convergence is initially slower than ResNet but later performance is better. GCNs also outperform the baseline with fewer parameters.

  • ImageNet: The ImageNet-100 experiment trains a 34-layer GCN with four orientation channels against a ResNet-101 baseline.Both models are trained for 120 epochs, using Top-1 and Top-5 error as evaluation metrics.
  • ImageNet: GCNs converge slightly more slowly initially than ResNet but achieve better performance in later epochs.ResNet performs better early but saturates quickly; the ImageNet test-error curves omit the first 60 epochs for clarity.
  • ImageNet: GCNs achieve 3.04% versus 3.16% Top-5 error and 11.46% versus 11.94% Top-1 error with 35.85M versus 44.54M parameters.These comparisons are reported against the ImageNet-100 ResNet baseline.
  • ImageNet: On full ImageNet, the 34-layer GCN reaches 73.2% performance compared with 71.6% for ResNet-34.The passage presents this as further validation of the method.

E. Experiment on Food-101 Dataset

The Food-101 experiment evaluates GCNs on high-resolution food images that challenge existing methods. GCNs outperform a state-of-the-art kernel-pooling method, with steerable filters suggested as a reason for improved noisy-image recognition.

  • Food-101: Food-101 contains high-resolution food-recognition images that remain challenging for existing methods.The comparison includes state-of-the-art ResNets and kernel-pooling methods.
  • Food-101: A 28-layer GCN with stages 16-16-32-64 achieves substantially better error-rate performance than the state-of-the-art kernel-pooling method.The experiment uses the architecture specified in the passage.
  • Food-101: The authors suggest that steerable filters may help GCNs capture nonlinear features for food recognition under noise.This is presented as a possible explanation rather than a demonstrated causal mechanism.
  • Conclusion: The paper concludes that GCNs improve baseline networks across several benchmarks and can be implemented with popular architectures.It also identifies object tracking, detection, and segmentation as future application areas.
Loading 1705.01450v4…