Source-linked AI summary
Batch-Instance Normalization for Adaptively Style-Invariant Neural Networks
Hyeonseob Nam, Hyo-Eun Kim
TL;DR
Real-world recognition must handle style variations that may be irrelevant or discriminative depending on the task. The paper introduces Batch-Instance Normalization, which learns per-channel gates to balance Batch and Instance Normalization. BIN improves recognition across object and multi-domain settings and also supports image stylization, while its style representation assumes feature-map mean and variance encode style attributes.
Problem
Image-style variation complicates real-world recognition, but directly removing styles can discard features that are useful for discrimination.
Method
BIN uses learnable per-channel gates to preserve important styles with Batch Normalization and suppress unnecessary styles with Instance Normalization.
Results
BIN improves performance over Batch Normalization across object classification and multi-domain tasks, and successfully replaces Instance Normalization in style transfer.
Takeaways & Limitations
A single gated normalization module adapts style preservation and removal across recognition and stylization tasks.
Takeaways & Limitations
BIN assumes that a convolutional feature map’s mean and variance encode a style attribute, with remaining spatial configuration representing shape.
Abstract
from arXiv · showhide
Real-world image recognition is often challenged by the variability of visual styles including object textures, lighting conditions, filter effects, etc. Although these variations have been deemed to be implicitly handled by more training data and deeper networks, recent advances in image style transfer suggest that it is also possible to explicitly manipulate the style information. Extending this idea to general visual recognition problems, we present Batch-Instance Normalization (BIN) to explicitly normalize unnecessary styles from images. Considering certain style features play an essential role in discriminative tasks, BIN learns to selectively normalize only disturbing styles while preserving useful styles. The proposed normalization module is easily incorporated into existing network architectures such as Residual Networks, and surprisingly improves the recognition performance in various scenarios. Furthermore, experiments verify that BIN effectively adapts to completely different tasks like object classification and style transfer, by controlling the trade-off between preserving and removing style variations. BIN can be implemented with only a few lines of code using popular deep learning frameworks.
1 Introduction
Real-world recognition is disrupted by image-style variation, yet style information can also be task-relevant. BIN addresses this tension by selectively normalizing disturbing styles while preserving useful ones.
- Image information combines spatial content, such as object shape, with style details, such as texture, lighting, and camera-induced appearance changes.
- Style variability can disturb recognition, although explicit style manipulation has shown value in neural image generation.
- Directly applying Instance Normalization to classification degrades performance because some styles provide discriminative features.
- The usefulness of style depends on the task: brightness may support weather prediction, while texture may distinguish fashion attributes but confuse clothing categories.
- BIN uses a learnable gate to suppress irrelevant styles through Instance Normalization and preserve important styles through Batch Normalization.
- Experiments report that BIN outperforms Batch Normalization in object classification and multi-domain problems and substitutes for Instance Normalization in style transfer.
2 Related Work
Prior work links convolutional feature statistics to image style and develops normalization methods for controlling internal representations. Instance Normalization is distinguished by its per-instance adjustment of feature statistics for style manipulation.
- Feature statistics from convolutional networks support style-related tasks including texture recognition, segmentation, and synthesis.
- Batch Normalization normalizes activations using statistics computed across examples in a minibatch, while several variants reduce its minibatch dependencies.
- Instance Normalization adjusts per-instance feature statistics and has been associated with image-style normalization.
3 Batch-Instance Normalization
BIN combines batch and instance normalization to adaptively preserve useful style information while suppressing disturbing styles on a channel-by-channel basis.
- Motivation: The paper models each feature map as style, represented by activation mean and variance, plus shape, represented by spatial configuration.This assumption motivates normalizing style statistics while retaining spatial structure.
- Batch and instance normalization: BN computes channel statistics across the minibatch and preserves instance-level style variation, whereas IN computes statistics independently per example and removes that variation.The two normalizations therefore provide complementary treatments of style.
- Batch-Instance Normalization: BIN combines BN and IN using learnable channel-wise parameters ρ ∈ [0, 1]^C to balance style preservation and normalization.The output also applies affine transformation parameters γ and β, while ρ is constrained to the stated range.
- Adaptive gating: A gate value approaching 1 selects BN to preserve important style, while a value approaching 0 selects IN to suppress unnecessary or disturbing style.The gate is updated during training, and the paper notes that increasing its learning rate is beneficial because its loss gradient can be small.
- Evaluation context: Figure 1 compares top-1 training and testing accuracy curves for ResNet-110 on CIFAR-100 across normalization methods.Solid lines show averages and shaded areas show 95% confidence intervals over 10 repetitions.
4 Experiments
Across object classification, multi-domain learning, and style transfer, BIN improves or matches relevant normalization baselines while adapting its style-selection behavior to each task. Its learned gates favor BN for classification and IN more strongly for style transfer, supporting selective style normalization.
- Object Classification: BIN improves classification over BN across CIFAR-10, CIFAR-100, and ImageNet, while using nearly the same parameter count.ResNet-110 uses 1.73M parameters with BN and 1.74M with BIN.
- Object Classification: The learned classification gates are mostly near 0 or 1, with substantially more gates near 1, showing a preference for selecting IN or BN rather than mixing them uniformly.Gates near 1 correspond to BN and occupy a greater fraction than gates near 0, which correspond to IN.
- Object Classification: BIN outperforms BN across all tested CNN architectures on CIFAR-100, indicating scalability beyond a single network design.Architectures include AlexNet, VGGNet, ResNet, PreResNet, WRN, ResNeXt, and DenseNet.
- Multi-Domain Learning: BIN improves mixed-domain classification on all four Office-Home domains, especially Art, where exploiting other domains is more valuable because it has fewer images.The evaluation averages accuracy over 5-fold cross validation and tests a network trained on the combined domains.
- Multi-Domain Learning: On Office-Home domain adaptation, BIN surpasses BN on 11 of 12 transfer tasks and normalizes styles associated with domain labels.The experiments use DANN with ResNet-18 under a fully-transductive protocol, incorporating unlabeled target-domain training data.
- Image Style Transfer: In style transfer, BIN achieves slightly lower content loss than IN while mitigating the style-loss degradation of naïvely combining BN and IN.BIN preserves styles relevant to content while selectively discarding impeditive styles; its gates show a higher fraction near 0 than in classification.
5 Conclusion
BIN adaptively balances Batch Normalization and Instance Normalization to remove task-irrelevant styles while preserving useful style information. It improves BN-based recognition models and also supports image stylization as an alternative to IN.
- 5 Conclusion: BIN binds BN and IN with learnable gates that select normalization according to the importance of style features.The method suppresses irrelevant styles through IN and preserves important styles through BN.
- 5 Conclusion: Replacing BN with BIN considerably improves performance across various image recognition tasks.
- 5 Conclusion: BIN also applies successfully to image stylization, providing an alternative to IN for controlling style preservation and removal.Style-transfer examples show both similar and distinct results relative to other normalization methods.
- 5 Conclusion: Normalization research has largely focused on reducing BN’s minibatch dependencies, while BIN addresses adaptive style normalization across applications.
Appendix A Experiments on Character Recognition
Subsidiary experiments evaluated BIN on character-recognition datasets with substantial style variation. BIN consistently outperformed BN, with results reported as repeated average Top-1 accuracies and 95% confidence intervals.
- Appendix A Experiments on Character Recognition: BIN consistently outperforms BN across ICDAR2003, ICDAR2005, and Chars74K character-recognition experiments.The experiments used relatively shallow residual networks and repeated training with the same core hyper-parameters as Section 4.1.
- Appendix A Experiments on Character Recognition: The character-recognition results report average Top-1 accuracy (%) over five repetitions with 95% confidence intervals.