PaperlayerView exact revision on arXiv

Source-linked AI summary

Omni-Scale Feature Learning for Person Re-Identification

Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, Tao Xiang

arXiv:1905.00953v6cs.CV

TL;DR

Person re-identification needs features that represent both individual spatial scales and arbitrary mixtures of scales, but existing models did not explicitly learn such omni-scale representations. OSNet addresses this with lightweight multi-stream residual blocks and an input-dependent unified aggregation gate, achieving state-of-the-art results across six re-ID datasets and strong performance beyond re-ID.

  • Problem

    Person re-identification requires discriminative features that capture both homogeneous spatial scales and heterogeneous combinations of scales, a capability existing re-ID models did not address explicitly.

  • Method

    OSNet stacks lightweight residual blocks with multiple receptive-field streams and a unified aggregation gate that dynamically fuses their features using input-dependent channel-wise weights.

  • Results

    State-of-the-art performance on six person re-ID datasets was achieved despite OSNet's lightweight design, with strong results also reported for object categorization and person attribute recognition.

  • Takeaways & Limitations

    Omni-scale feature learning is useful beyond person re-identification and can be considered for a broad range of visual recognition tasks.

Abstract

from arXiv · show

As an instance-level recognition problem, person re-identification (ReID) relies on discriminative features, which not only capture different spatial scales but also encapsulate an arbitrary combination of multiple scales. We call features of both homogeneous and heterogeneous scales omni-scale features. In this paper, a novel deep ReID CNN is designed, termed Omni-Scale Network (OSNet), for omni-scale feature learning. This is achieved by designing a residual block composed of multiple convolutional streams, each detecting features at a certain scale. Importantly, a novel unified aggregation gate is introduced to dynamically fuse multi-scale features with input-dependent channel-wise weights. To efficiently learn spatial-channel correlations and avoid overfitting, the building block uses pointwise and depthwise convolutions. By stacking such block layer-by-layer, our OSNet is extremely lightweight and can be trained from scratch on existing ReID benchmarks. Despite its small model size, OSNet achieves state-of-the-art performance on six person ReID datasets, outperforming most large-sized models, often by a clear margin. Code and models are available at: \url{https://github.com/KaiyangZhou/deep-person-reid}.

1. Introduction

Person re-identification must handle large intraclass variation and small interclass variation, motivating omni-scale features that combine local, global, and heterogeneous spatial information. OSNet addresses this with lightweight multi-stream blocks and input-dependent aggregation, achieving strong performance across re-ID benchmarks.

  • Motivation: Person re-identification matches people across non-overlapping camera views despite large intraclass and small interclass appearance variations.View changes can substantially alter the same person's appearance, while different people may wear similar clothes.
  • Motivation: Omni-scale features combine variable homogeneous scales with heterogeneous mixtures of scales to capture both local details and global person context.Examples include shoes or logos alongside whole-body clothing combinations.
  • Research gap: Existing re-ID models did not explicitly learn omni-scale features, while many were adapted from CNNs designed for object category recognition.Prior multi-scale approaches were described as relying on external pose models or hand-picked layers.
  • Approach: OSNet uses multiple convolutional streams with different receptive field sizes and a unified aggregation gate that dynamically assigns input-dependent channel-wise weights.The gate can emphasize one scale or combine multiple scales for a given input.
  • Efficiency: Pointwise and depthwise convolutions make OSNet lightweight, reducing overfitting risk for moderate-sized re-ID datasets and supporting camera-end feature extraction.The paper states that the resulting model is more than one order of magnitude smaller than popular ResNet50-based models.
  • Results: OSNet achieves state-of-the-art performance on six person re-ID datasets despite its lightweight design and also performs strongly on category and attribute recognition tasks.These results suggest applicability beyond instance recognition.

2. Related Work

Prior deep re-ID models largely borrowed architectures from generic object categorization, while later methods added pose-aware, attention, branched, or multi-level mechanisms. OSNet differs by explicitly learning multi-scale features at every layer and dynamically combining heterogeneous scales.

  • Deep re-ID architectures: Most deep re-ID CNNs borrow architectures originally designed for generic object categorization, such as ImageNet object classification.Some later modifications exploit upright body pose through auxiliary supervision on horizontally pooled features.
  • Prior multi-scale approaches: Existing approaches also use attention mechanisms, branches for global and local regions, or combinations of features extracted from different network layers.These strategies target re-ID structure but do not constitute OSNet's per-layer omni-scale design.
  • Distinction: OSNet explicitly learns multi-scale features at each network layer rather than relying on an external pose model or hand-picked layers.It also uses a unified aggregation gate to combine scale-specific streams dynamically.

3. Omni-Scale Feature Learning

OSNet learns omni-scale features through lightweight residual bottlenecks that combine incremental receptive fields with input-dependent, channel-wise aggregation. Its shared aggregation gate and factorised convolutions support scalable multi-stream learning while reducing computation and parameters.

  • 3.1. Depthwise Separable Convolutions: OSNet factorises standard convolutions into pointwise and depthwise convolutions to reduce computation and parameters.The implementation uses pointwise-to-depthwise ordering, called Lite 3 × 3 convolution.
  • 3.2. Omni-Scale Residual Block: Each omni-scale residual block stacks Lite 3 × 3 layers to create streams with progressively larger receptive fields.For exponent t, the receptive field is (2t + 1) × (2t + 1), while the residual sums incremental scales up to T.
  • 3.2. Omni-Scale Residual Block: The proposed bottleneck preserves smaller-scale features through a shortcut while using T = 4 to reach a largest receptive field of 9 × 9.When T = 1, the formulation reduces to the baseline residual bottleneck.
  • 3.2. Omni-Scale Residual Block: A unified aggregation gate dynamically fuses stream outputs using input-dependent channel-wise weights rather than fixed or scalar scale weights.The gate uses global average pooling and an MLP with sigmoid activation; channel-wise weighting enables finer-grained fusion.
  • 3.2. Omni-Scale Residual Block: Sharing one aggregation gate across streams keeps parameter count independent of the number of streams and gathers supervision signals from all streams.The shared gate also makes the architecture more scalable as streams are added.
  • 3.3. Network Architecture: OSNet stacks the lightweight bottleneck without stage-specific customization, and its Lite 3 × 3 design is about three times smaller than the corresponding standard-convolution network.The standard-convolution counterpart has 6.9 million parameters and 3,384.9 million mult-add operations.

4. Experiments

Experiments evaluate OSNet across person re-ID, object recognition, and attribute recognition settings, including large and small datasets, ablations, and model shrinking. OSNet generally maintains strong performance with substantially fewer parameters than larger alternatives.

  • Person re-identification: OSNet achieves state-of-the-art performance on all big re-ID datasets with 2.2 million parameters, versus more than 24 million for many ResNet50-based competitors.The reported improvements are obtained on benchmarks whose performance has recently become saturated.
  • Person re-identification: OSNet outperforms HAN and BraidNet by 2.4%/9.9% on Market1501 and 4.2%/8.3% on Duke at Rank-1 when trained from scratch.The passage also reports larger margins at mAP and advantages over MobileNetV2 and ShuffleNet across datasets.
  • Person re-identification: On VIPeR, OSNet exceeds alternatives by more than 11.4% Rank-1 accuracy, while on GRID it is marginally better than JLML.VIPeR and GRID contain very limited training data, with GRID additionally including distractors and operational analogue CCTV imagery.
  • Ablation studies: Ablations show that omni-scale design and unified aggregation improve performance, while channel-wise gates outperform stream-wise gates by 1% in both Rank-1 and mAP.The primary model also exceeds concatenation and addition fusion baselines by more than 1.6%/2.8% at Rank-1/mAP.
  • Efficiency: 92.2% Rank-1 accuracy is obtained with a shrunken OSNet using 0.2M parameters and 82M mult-adds.The width and resolution multipliers provide a smooth trade-off among model size, computation, and performance.
  • Analysis: OSNet captures local discriminative regions more effectively than the single-scale baseline in activation-map comparisons.The visualization examines where the final convolutional feature maps focus when extracting features.
  • Attribute recognition: OSNet outperforms two alternatives on all five PA-100K metrics and is particularly strong on heterogeneous-scale attributes such as age and gender.The experiments extend evaluation beyond person re-identification to attribute recognition.
  • Object recognition: On ImageNet, OSNet×1.0 surpasses MobileNetV2×1.0 by 3.5% and MobileNetV2×1.4 by 0.8%, despite the latter being around 2.5× larger.OSNet×0.75 performs on par with ShuffleNet×2.0 and exceeds ShuffleNet×1.5/×1.0 by 2.0%/5.9%.

5. Conclusion

The paper concludes that OSNet learns omni-scale representations in a lightweight CNN and achieves strong performance across re-ID and broader visual-recognition tasks.

  • Conclusion: OSNet achieves state-of-the-art performance on six person re-ID datasets despite its lightweight design.The authors also report superior performance on object categorisation and a multi-label attribute recognition task.

Supplementary

The supplementary material presents additional results intended to further demonstrate OSNet’s strength.

  • Supplementary: The supplementary material reports additional results to further demonstrate OSNet’s strength.The main-paper results were presented at ICCV’19.

A. A Strong Backbone for Cross-Domain ReID

OSNet-IBN extends OSNet with instance normalisation in lower layers to build a strong cross-domain re-ID backbone. Without target data, it achieves competitive results against unsupervised domain adaptation methods.

  • Method: OSNet-IBN inserts instance normalisation into the lower OSNet layers to improve cross-domain re-ID generalisation.Instance normalisation is added to conv1 and conv2, after the residual connection and before ReLU.
  • Evaluation setting: Market1501 and Duke serve as target datasets, with Market1501, Duke, or MSMT17 used as source datasets for direct cross-domain testing.Models are trained on labelled source data and tested directly on target data.
  • Results: OSNet-IBN achieves competitive performance with state-of-the-art unsupervised domain adaptation methods without using target data.The comparison is reported in Table 11, whose highlighted OSNet-IBN rows use no target data.
  • Results: On Market1501→Duke at rank-1, OSNet-IBN beats every listed UDA method except ECN.It performs on par with MAR on MSMT17→Duke and obtains comparable results with MAR and PAUL on MSMT17→Market1501.

B. Training Recipes for Practitioners

The paper evaluates practical training choices for OSNet and recommends combining cross-entropy, a carefully weighted triplet loss, deep mutual learning, and model ensembling. Several other choices provide only marginal or little benefit.

  • Normalisation: Dataset-specific pixel normalisation changes performance only subtly, so practitioners are encouraged to try both dataset-specific and ImageNet statistics.Re-ID images can differ substantially from ImageNet images in quality and blur, but the measured performance difference is small.
  • Input size: Larger input size improves performance only marginally, while increasing computation from 978.9M to 1,529.3M FLOPs; 256 × 128 is therefore recommended.The authors attribute the limited gain to OSNet learning omni-scale features that are insensitive to input size.
  • Experimental setup: Three-random-seed experiments evaluate training methods and implementation choices for improving OSNet.The reported implementation uses cosine annealing for learning-rate decay and benchmarks Market1501 and Duke.
  • Regularisation: Entropy maximisation has little effect on performance across the tested balancing weights.The term is introduced as a regulariser for small-scale re-ID datasets that penalises confident predictions.
  • Deep mutual learning: Deep mutual learning improves mAP, and ensembling its two networks further improves both rank-1 and mAP.The paper reports these changes in Table 12d.
  • Auxiliary loss: Triplet loss improves performance when its balancing weight is carefully tuned, with cross-entropy retained as the main objective.The authors recommend triplet loss as an auxiliary loss rather than the primary training objective.
  • Combination: Combining deep mutual learning with auxiliary triplet-loss training yields larger improvements than either technique alone.The best performance comes from fusing the two DML-trained models.
  • Recommendation: The practical recipe is cross-entropy plus triplet loss with λ_t = 0.5, deep mutual learning, and model-ensemble testing.This is presented as the authors’ rule of thumb for training OSNet.
Loading 1905.00953v6…