Source-linked AI summary
TI-POOLING: transformation-invariant pooling for feature learning in Convolutional Neural Networks
Dmitry Laptev, Nikolay Savinov, Joachim M. Buhmann, Marc Pollefeys
TL;DR
Deep networks need to handle known nuisance variations, but augmentation increases data, model, and training requirements while learning separate representations for transformed inputs. The paper introduces TI-POOLING, which pools responses from shared paths over predefined transformations to learn invariant features and canonical instances. Across three datasets, it matches or exceeds state-of-the-art baselines, often with faster convergence and smaller models.
Problem
Known nuisance variations motivate transformation-invariant features, but augmentation requires more data, longer training, greater model flexibility, and separate representations for each variation.
Method
TI-POOLING passes original and transformed inputs through parallel shared-weight network paths and takes the maximum response to form transformation-invariant features.
Results
Across three datasets, TI-POOLING significantly outperforms or matches state-of-the-art baselines, often with faster convergence and smaller models.
Takeaways & Limitations
The approach learns from representative canonical instances and avoids learning separate features for every possible data variation.
Takeaways & Limitations
The approach depends on specifying an appropriate transformation set; in one biomedical experiment, rotations were sampled every 15 degrees while scale was excluded because imaging fixed it and object size could be informative.
Abstract
from arXiv · showhide
In this paper we present a deep neural network topology that incorporates a simple to implement transformation invariant pooling operator (TI-POOLING). This operator is able to efficiently handle prior knowledge on nuisance variations in the data, such as rotation or scale changes. Most current methods usually make use of dataset augmentation to address this issue, but this requires larger number of model parameters and more training data, and results in significantly increased training time and larger chance of under- or overfitting. The main reason for these drawbacks is that the learned model needs to capture adequate features for all the possible transformations of the input. On the other hand, we formulate features in convolutional neural networks to be transformation-invariant. We achieve that using parallel siamese architectures for the considered transformation set and applying the TI-POOLING operator on their outputs before the fully-connected layers. We show that this topology internally finds the most optimal "canonical" instance of the input image for training and therefore limits the redundancy in learned features. This more efficient use of training data results in better performance on popular benchmark datasets with smaller number of parameters when comparing to standard convolutional neural networks with dataset augmentation and to other baselines.
1. Introduction
Known nuisance variations such as rotation and scale can improve computer-vision models, but augmentation increases learning and training burdens. TI-POOLING instead forms transformation-invariant features from transformed inputs and shows strong benchmark performance with efficient data use.
- Motivation: Known nuisance variations, including rotation, shifts, scale, and illumination, can be incorporated during training when experts identify them in advance.Their relevance is domain-dependent: rotation may be irrelevant in medical imaging, while scale can remain informative.
- Limitations of augmentation: Data augmentation trains on transformed samples, but requires separate feature representations for different variations of the original data.For rotation-invariant edge detection, vertical and horizontal edge detectors may be learned along separate activation paths.
- Limitations of augmentation: More considered variations require greater model flexibility, more data, longer training, less control over complexity, and greater overfitting potential.Some transformations can also produce noise samples or wrong labels, such as crops that remove or poorly represent the object.
- Proposed approach: TI-POOLING feeds original and transformed inputs through shared network paths, accumulates their responses, and takes the maximum to produce transformation-invariant features.The maximum response makes the representation independent of the considered variations and selects a canonical instance for learning.
- Evaluation: Across three datasets, the method significantly outperforms or matches state-of-the-art baselines, often with faster training or similar performance using fewer parameters.The experiments include transformed MNIST variants and a rotation-invariant biomedical segmentation dataset.
2. Related works
Prior approaches achieve transformation invariance through handcrafted features, learned representations, model ensembles, or input transformers, each with specific limitations. TI-POOLING instead learns invariant features through pooling over predefined transformations and differs from multiple-instance learning by optimizing each feature separately.
- Transformation-invariant features: Handcrafted invariant features such as SIFT, RIFT, and domain-specific line filters reduce the transformation burden but are costly to design, task-inspecific, and limited to particular variations.Their invariance is built into the input features rather than adapted during task-specific learning.
- Feature learning: Learned alternatives include bag-of-visual-words representations and transformation-invariant decision jungles, but decision jungles learn greedily and use a less expressive combining algorithm.Bag-of-visual-words methods can provide shift invariance and, with modifications, rotation invariance.
- Deep neural networks: Multi-column networks average outputs from separate models trained on original and transformed datasets, while spatial transformer networks learn transformation parameters from the data.Spatial transformers search for a canonical appearance using a parameterized transformation class without additional supervision.
- Deep neural networks: TI-POOLING resembles spatial transformers in finding a canonical input position but uses a stricter predefined transformation set and reports similar benchmark results with simpler models and shorter training.The comparison is made on a benchmark introduced by the spatial-transformer work.
- Multiple instance learning: Maximum pooling is the operator emphasized in this work, although alternatives such as averaging can also be used.The paper states that maximum pooling works best in most applications considered.
- Multiple instance learning: Unlike general multiple-instance learning, TI-POOLING learns individual features to be invariant, allowing different features to rely on different canonical instances.The paper reports that this distinction leads to significantly better performance than standard multiple-instance-learning models.
- Related approaches: Rolling feature maps pool over transformations without guaranteeing invariant learned features, whereas multi-view networks address view invariance rather than an expert-defined transformation set.These methods therefore target different invariance properties from TI-POOLING.
3. Method description
TI-POOLING transforms each input through shared-weight parallel branches, then max-pools their feature responses so learned features become less dependent on known transformations. The method can identify representative or canonical transformed instances while retaining standard fully connected processing and backpropagation.
- Network topology: Parallel siamese convolutional and subsampling branches process the transformation set Φ with shared weights before pooling and fully connected layers.Weight sharing means the model requires the same memory as one convolutional neural network.
- TI-POOLING: TI-POOLING applies element-wise maximum pooling across feature vectors computed from transformed versions of the same input.The pooled feature is g_k(x) = max_{φ∈Φ} f_k(φ(x)).
- Canonical instances: Each feature can select its own optimal transformation φ = arg max_{φ∈Φ} f_k(φ(x)), rather than using one transformation choice for all features.This selection identifies the most representative instance for learning.
- Theoretical properties: When Φ forms a group, the resulting features are transformation-invariant; local shifts are an example that does not satisfy this condition.The stated group conditions are closure, associativity, invertibility, and identity.
- Canonical instances: Even without group structure, the method often maps transformed images toward canonical appearances, preserving approximate invariance in practical cases.The paper illustrates canonical orientations for neuronal structures.
- Performance and efficiency: Larger transformation sets usually improve results because the learning algorithm handles fewer canonical positions, while training uses more representative examples than simple augmentation.The paper also reports better performance and convergence, smaller models, and more than two-times-faster processing than standard augmentation.
4. Experiments
Experiments on rotated MNIST and neuronal segmentation evaluate TI-POOLING under artificial and real-world nuisance variations. Across these settings, the method improves performance or matches state-of-the-art results with faster training and comparable topology or parameter counts.
- Experimental setup: The experiments cover two rotated-MNIST variants and a neuronal-structure segmentation dataset with rotation invariance.The datasets test artificially introduced variations and a real-world biomedical setting.
- Rotated MNIST: On mnist-rot-12k, TI-POOLING uses 24 uniformly sampled rotations and achieves 1.2% test error versus 4.2% for the best published approach.The network is trained for 1200 epochs on a single GPU.
- Half-rotated MNIST: On half-rotated MNIST, TI-POOLING reaches 0.8% error within 360 epochs, matching the general STN model trained for 1280 epochs.The transformation set contains 13 angles sampled uniformly from half a circle; performance is slightly worse than a manually selected affine-transformation comparison by 0.1%.
- Neuronal structures segmentation: For neuronal segmentation, expert knowledge supports treating membrane appearance as rotation-invariant while retaining a fixed set of 24 rotations sampled every 15 degrees.The method classifies 32 × 32 rotated input patches using the central pixel label.
- Neuronal structures segmentation: Using the same underlying topology, TI-POOLING significantly improves over both augmented CNN and MIL baselines on the neuronal segmentation task.The comparison controls topology and, for augmentation, equalizes the number of images seen by training for 16 × 24 epochs.
5. Conclusions
The paper concludes that TI-POOLING incorporates expert knowledge about nuisance variations into trainable, transformation-invariant CNN features while retaining expressive representations. Across three datasets, it matches or surpasses state-of-the-art baselines, often with faster convergence and smaller data-aware models.
- TI-POOLING passes transformed image instances through initial network layers and pools their responses to form fully trainable features independent of nuisance variations.The operator can be incorporated as a separate neuronal unit with little effort across most network architectures.
- The network learns from representative “canonical” instances rather than separately learning features for every possible data variation.This reduces redundancy and lets all transformed samples contribute to shared transformation-invariant features such as edge detectors.
- Across three datasets with explicitly defined variability, the method significantly outperforms or matches state-of-the-art baselines.The reported evaluations include substantially better performance on one MNIST variation and faster training while matching state of the art on another.
- TI-POOLING offers a practical route for combining prior knowledge about nuisance factors with the expressive power of deep learning.The paper suggests that the operator may also accommodate broader types of prior information about data.