PaperlayerView exact revision on arXiv

Source-linked AI summary

Oriented Response Networks

Yanzhao Zhou, Qixiang Ye, Qiang Qiu, Jianbin Jiao

arXiv:1701.01833v2cs.CV

TL;DR

DCNNs remain limited in handling significant local and global rotations, motivating a rotation-aware alternative. The paper introduces actively rotating filters and oriented convolutions that encode location and orientation, and reports improved performance with substantially fewer parameters across architectures and benchmarks.

  • Problem

    DCNNs have limited ability to handle significant local and global image rotations, restricting performance on several visual tasks.

  • Method

    Active Rotating Filters rotate during convolution, while Oriented Response Convolution produces orientation channels that encode hierarchical location and orientation information.

  • Results

    ORNs consistently improve classification performance while using significantly fewer parameters across upgraded VGG, ResNet, TI-Pooling, and STN architectures.

  • Takeaways & Limitations

    Model-level rotation priors support compact and effective deep networks that explicitly encode hierarchical orientation information.

  • Takeaways & Limitations

    Smooth processing of all rotation angles requires considerable orientation channels, although multi-layer pooling permits using a limited number in practice.

Abstract

from arXiv · show

Deep Convolution Neural Networks (DCNNs) are capable of learning unprecedentedly effective image representations. However, their ability in handling significant local and global image rotations remains limited. In this paper, we propose Active Rotating Filters (ARFs) that actively rotate during convolution and produce feature maps with location and orientation explicitly encoded. An ARF acts as a virtual filter bank containing the filter itself and its multiple unmaterialised rotated versions. During back-propagation, an ARF is collectively updated using errors from all its rotated versions. DCNNs using ARFs, referred to as Oriented Response Networks (ORNs), can produce within-class rotation-invariant deep features while maintaining inter-class discrimination for classification tasks. The oriented response produced by ORNs can also be used for image and object orientation estimation tasks. Over multiple state-of-the-art DCNN architectures, such as VGG, ResNet, and STN, we consistently observe that replacing regular filters with the proposed ARFs leads to significant reduction in the number of network parameters and improvement in classification performance. We report the best results on several commonly used benchmarks.

1. Introduction

DCNNs handle moderate transformations but remain limited on significant local and global rotations. The paper addresses this by embedding rotation-aware filters into convolution to encode orientation explicitly while reducing redundancy and parameters.

  • Significant local and global rotations limit DCNN performance in boundary detection, multi-oriented object detection, and image classification.
  • DCNN filters often redundantly learn rotated versions of identical structures across multiple network levels.Relative rotations of object parts can require additional filter combinations, while augmentation increases training cost and parameter requirements.
  • Active Rotating Filters and Oriented Response Convolution generate orientation channels that explicitly encode discriminative patterns’ location and orientation.Each ARF rotates during convolution to capture receptive-field responses from multiple orientations.
  • ORNs incorporate rotation prior directly into convolution, allowing integration with modern DCNN architectures without extra functional modules or new topologies.The resulting oriented responses support rotation-invariant feature alignment and image or object orientation estimation.
  • Upgraded VGG, ResNet, TI-Pooling, and STN architectures achieve state-of-the-art benchmark performance with significantly fewer parameters.

2. Related Works

Prior work addresses orientation through hand-crafted descriptors, augmentation, spatial transformation, and rotated filters. These approaches provide rotation handling but leave limitations in cost, transform estimation, or hierarchical orientation encoding.

  • SIFT achieves rotation invariance by aligning descriptors to a dominant orientation before accumulating local gradient statistics.
  • LBP achieves rotation invariance by minimizing its code value under cyclic bit shifts.
  • DCNN convolution, redundant filters, and hierarchical pooling handle moderate transitions, scale changes, and small rotations.More general pooling addresses local deformation but does not encode specific prior knowledge about the transformation.
  • Data augmentation can provide local and global transform invariance, but learning augmented transformations increases parameters, training cost, and over-fitting risk.
  • STN offers a general spatial-transform framework, but precise estimation of complex transform parameters remains unresolved.
  • Rotating conventional filters supports rotation-invariant classification, but without multi-oriented filters and orientation channels, hierarchical fine-grained orientation capture remains limited.

3. Oriented Response Networks

ORNs replace conventional filters with actively rotating filters that encode location and orientation in feature maps. They support orientation-aware processing, collective filter updates, and top-layer rotation-invariant encoding through alignment or pooling.

  • 3.1. Active Rotating Filters: ORNs use ARFs, which materialize one filter while representing its rotated versions as a virtual filter bank.An ARF has size W × W × N and produces N orientation channels during convolution.
  • 3.1. Active Rotating Filters: ARFs rotate through coordinate rotation and orientation spin, with bilinear interpolation and Fourier-domain circular shifts enabling efficient processing.Points outside the inscribed circle are padded with zero, while orientation spin quantizes a shifted periodic response.
  • 3.2. Oriented Response Convolution: ORConv produces feature maps with N orientation channels, explicitly encoding receptive-field location and orientation while combining responses from rotated filters.Each ARF orientation channel contributes separately to the final response, capturing richer fine-detailed patterns than regular CNNs.
  • 3.2. Oriented Response Convolution: Feature visualizations show similar deep responses across rotated digits at different orientations, while the final ORAlign layer performs alignment for rotation invariance.Texture visualizations show ARFs defining patterns through combinations of multi-oriented edges.
  • 3.3. Updating Filters: Back-propagation aligns errors from all rotated ARF versions and collectively updates the single materialized filter.This aggregates training errors from appearance-like samples that differ in orientation.
  • 3.4. Rotation Invariant Feature Encoding: ORNs preserve orientation information rather than discarding it, then use ORAlign or ORPooling at the top layer when within-class rotation invariance is required.ORAlign applies SIFT-like dominant-orientation alignment, whereas ORPooling reduces feature dimension but loses feature arrangement information.

4. Experiments

Across rotated-digit, orientation-estimation, and natural-image benchmarks, ORNs consistently improve rotation handling while reducing parameters and, in several settings, training cost.

  • Rotation Invariance: ORN-4 uses 50% of baseline training time, whereas ORN-8 has similar training time to the baseline CNN and TIPooling increases time about eightfold.TIPooling requires augmenting each sample to eight orientations.
  • Rotation Invariance: ORNs use 7.95%-31.4% of baseline parameters while consistently improving performance across MNIST variants.On the original dataset, error decreases from 0.73% to 0.57%; against augmentation, ORN on rot achieves 1.37% versus 2.19%.
  • Rotation Invariance: Cross-generalization from MNIST-original to randomly rotated MNIST shows a 71% improvement for ORNs over the baseline.The evaluation trains on unrotated images and tests on randomly rotated images.
  • Rotation Invariance: ORN-8(ORAlign) produces within-class rotation-invariant features while preserving discrimination between digit classes such as 6 and 9.CNN and STN features show large within-class variation and 180° symmetry, whereas ORN-8(ORAlign) forms clearer feature manifolds.
  • Orientation Estimation: Upgrading the localization network to ORN significantly improves orientation estimation, while OR-STN with ORAlign performs worse because aligned features lose orientation information.The experiments use MNIST-half-rot and compare STN, OR-STN, and OR-STN(ORAlign).
  • Natural Image Classification: On natural-image classification, ORNs consistently improve baselines with fewer parameters; OR-WideResNet-40-2 uses 12% of parameters and outperforms WideResNet-28-10 on CIFAR10.The comparison is 4.5M versus 36.5M parameters, and rotation-heavy classes show larger gains.

5. Conclusions

ORNs explicitly encode hierarchical orientation information to handle global and local rotations, outperforming baseline DCNNs with substantially fewer parameters.

  • 12%-50% fewer network parameters are used by ORNs while outperforming baseline DCNNs.The paper presents this as evidence that model-level rotation priors support compact and effective deep networks.
Loading 1701.01833v2…