PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Generalisable Omni-Scale Representations for Person Re-Identification

Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, Tao Xiang

arXiv:1910.06827v5cs.CV

TL;DR

Person re-ID needs features that distinguish similar-looking people while generalising across datasets without adaptation. The paper introduces OSNet, which learns dynamically fused omni-scale features, and OSNet-AIN, which adds searched instance-normalisation placement. OSNet achieves strong same-domain performance with a much smaller model, while OSNet-AIN beats most recent UDA methods on unseen targets without target data.

  • Problem

    Re-ID must distinguish similar-looking people despite intraclass variation and inter-class ambiguity, while direct transfer across datasets suffers from domain gaps and performance drops.

  • Method

    OSNet uses multi-scale convolutional streams with an input-dependent unified aggregation gate, while OSNet-AIN adds instance normalisation whose placement is selected by differentiable architecture search.

  • Results

    OSNet achieves state-of-the-art same-domain performance with 2.2 million parameters, and OSNet-AIN beats most recent UDA methods on unseen target datasets without target data.

  • Takeaways & Limitations

    The architectures provide a lightweight same-domain re-ID model and a cross-domain variant that generalises without per-domain target adaptation.

  • Takeaways & Limitations

    Setting λ < 1 makes the training unstable.

Abstract

from arXiv · show

An effective person re-identification (re-ID) model should learn feature representations that are both discriminative, for distinguishing similar-looking people, and generalisable, for deployment across datasets without any adaptation. In this paper, we develop novel CNN architectures to address both challenges. First, we present a re-ID CNN termed omni-scale network (OSNet) to learn features that not only capture different spatial scales but also encapsulate a synergistic combination of multiple scales, namely omni-scale features. The basic building block consists of multiple convolutional streams, each detecting features at a certain scale. For omni-scale feature learning, a unified aggregation gate is introduced to dynamically fuse multi-scale features with channel-wise weights. OSNet is lightweight as its building blocks comprise factorised convolutions. Second, to improve generalisable feature learning, we introduce instance normalisation (IN) layers into OSNet to cope with cross-dataset discrepancies. Further, to determine the optimal placements of these IN layers in the architecture, we formulate an efficient differentiable architecture search algorithm. Extensive experiments show that, in the conventional same-dataset setting, OSNet achieves state-of-the-art performance, despite being much smaller than existing re-ID models. In the more challenging yet practical cross-dataset setting, OSNet beats most recent unsupervised domain adaptation methods without using any target data. Our code and models are released at \texttt{https://github.com/KaiyangZhou/deep-person-reid}.

1 INTRODUCTION

The paper targets two re-ID challenges: discriminative features for distinguishing similar people and generalisable features for deployment across datasets. It proposes OSNet, which learns omni-scale representations with dynamic multi-scale fusion, and incorporates instance normalisation for cross-domain robustness.

  • Motivation: Person re-ID must handle intraclass appearance changes and inter-class ambiguity across disjoint camera views.Viewpoint changes and visually similar people make matching difficult.
  • Motivation: Directly applying source-trained models to unseen datasets typically causes large performance drops because of domain gaps in lighting, backgrounds, and viewpoints.These gaps reflect differences in image styles and camera environments.
  • Omni-scale learning: Omni-scale features combine variable homogeneous scales with heterogeneous features spanning multiple scales, such as a logo with its upper-body context.Both local details and global or contextual regions can be needed to reject distractors.
  • Omni-scale learning: OSNet uses multiple convolutional streams with different receptive fields and a unified aggregation gate that dynamically fuses their feature maps using channel-wise weights.The gate can emphasize one scale or mix several scales depending on the input image.
  • Architecture: Factorised convolutions make OSNet lightweight, with 2.2 million parameters—more than one order of magnitude smaller than common ResNet50-based re-ID models.The lightweight design is intended to support efficient implementation and deployment.
  • Domain generalisation: OSNet-AIN incorporates instance normalisation into OSNet through differentiable architecture search to improve cross-dataset feature generalisation.Instance normalisation uses within-sample statistics to reduce instance-specific contrast and style affected by domain environments.

2 RELATED WORK

Prior re-ID architectures adapt generic CNNs and existing multi-stream designs, while domain adaptation commonly requires target data. This work distinguishes OSNet through scale-diverse streams, dynamic aggregation, and searched instance-normalisation placement for cross-domain re-ID.

  • CNN Architectures for Person Re-ID: Most deep re-ID methods use CNN architectures originally designed for generic object classification, then modify them for re-ID-specific challenges.Examples of targeted modifications address misalignment and pose variations.
  • Multi-Scale and Multi-Stream CNNs: OSNet differs from ResNeXt-based multi-stream designs because its streams learn at different scales rather than at one shared scale.A scale-controlling factor diversifies the spatial scales covered by the streams.
  • Multi-Scale and Multi-Stream CNNs: OSNet differs from Inception-style designs by using a scale-controlling factor and a unified aggregation gate instead of hand-crafted mixed operations and fixed feature aggregation.The gate provides dynamic, input-adaptive fusion across scales.
  • Lightweight Network Design: Existing lightweight CNNs rely on compression, group, depthwise-separable, or shuffled convolutions, whereas OSNet uses factorised convolutions in its lightweight design.The related designs include SqueezeNet, ResNeXt, Xception, MobileNet, and ShuffleNet.
  • Domain Generalisation: Unsupervised domain adaptation methods require unlabelled target data, whereas this work seeks cross-domain re-ID without target-data collection or per-domain model updates.The paper frames this as a domain-generalisable alternative to target-specific adaptation.
  • Domain Generalisation: Neural architecture search automatically selects instance-normalisation placements instead of relying on an unclear hand-picked definition of shallow layers.The paper argues that rules derived from semantic segmentation may not transfer to person re-ID.

3 OMNI-SCALE NETWORK FOR PERSON RE-ID

OSNet builds lightweight omni-scale representations by combining multiple receptive-field streams with input-adaptive channel-wise fusion. Its extension uses instance normalisation and differentiable architecture search to select how IN is integrated for domain generalisation.

  • Lightweight network design: OSNet stacks lightweight residual bottlenecks built from depthwise separable convolutions to reduce computation and parameters.Depthwise separable convolution decomposes spatial and channel mixing into depthwise and pointwise layers.
  • Omni-scale feature learning: Multiple streams capture homogeneous and heterogeneous feature scales by stacking Lite 3×3 layers with different exponents.The paper uses T = 4, producing a largest receptive field of 9×9.
  • Omni-scale residual block: The OSNet bottleneck uses first and last 1×1 layers to reduce and restore feature dimensions around the multi-stream residual transformation.The bottleneck is illustrated alongside a baseline bottleneck in Fig. 4.
  • Dynamic feature fusion: A shared aggregation gate dynamically fuses stream outputs using data-conditioned channel-wise vectors rather than fixed scalar weights.The gate uses global average pooling and an MLP with a reduction ratio of 16.
  • Instance normalisation and architecture search: Instance normalisation calibrates each sample using its own statistics, targeting style discrepancies caused by domain-specific environments.Unlike batch normalisation, IN removes instance-specific contrast and style using per-sample mean and standard deviation.
  • Instance normalisation and architecture search: The architecture search space contains four OS block variants with different IN placements, optimized through continuous Gumbel-Softmax relaxation of discrete choices.After search, the compact architecture selects the candidate with the largest architecture parameter π for each layer.

4 EXPERIMENTS

Experiments evaluate OSNet and OSNet-AIN across same-domain and cross-domain person re-ID settings, showing strong accuracy with compact models and adaptation-free transfer.

  • Same-Domain Person Re-Identification: OSNet achieves the best overall performance on big re-ID datasets while using only 2.2 million parameters.It outperforms most recently published methods and is substantially smaller than common ResNet50-based models.
  • Same-Domain Person Re-Identification: OSNet outperforms all alternatives on VIPeR by more than 11% and exceeds Spindle and JLML on CUHK01 by 6.7% and 16.8%, respectively.It remains marginally above JLML on GRID despite the dataset’s limited training data and additional observation noise.
  • Same-Domain Person Re-Identification: Factorised convolutions shrink the model size by more than 3× without significantly harming performance.The ablation reports a positive effect on large datasets such as MSMT17.
  • Same-Domain Person Re-Identification: A shrunken OSNet reaches 92.2% R1 with 0.2M parameters and 82.3M mult-adds.Performance decreases smoothly as width or image-resolution multipliers are reduced, while the compact model remains competitive with much larger methods.
  • Cross-Domain Person Re-Identification: OSNet-AIN improves over OSNet by 7.7% R1 and 4.6% mAP on Market1501→Duke, plus 8.8% R1 and 6.6% mAP on Duke→Market1501.The results support instance normalisation for cross-domain re-ID.
  • Cross-Domain Person Re-Identification: OSNet-AIN slightly decreases same-domain performance because instance normalisation removes dataset-specific features that can benefit same-domain recognition.The model remains competitive with state-of-the-art alternatives in the same-domain setting.
  • Cross-Domain Person Re-Identification: OSNet-AIN outperforms hand-engineered OSNet-IBN by 4.5% R1 and 2.9% mAP on Market1501→Duke, and by 3.2% R1 and 3.2% mAP on Duke→Market1501.The architecture-search space contains 4096 configurations, making learned configuration selection more efficient than exhaustive evaluation.
  • Cross-Domain Person Re-Identification: OSNet-AIN outperforms most UDA methods using only source data, improving R1 from 52.4% to 71.1% on Duke and from 61.0% to 70.1% on Market1501 when trained on MSMT17.In Market1501→MSMT17, it performs on par with ECN despite ECN using unlabelled target data.

5 CONCLUSION

The paper presents OSNet as a lightweight CNN for omni-scale person re-identification and OSNet-AIN as its domain-adaptive variant. OSNet achieves state-of-the-art same-domain performance, while OSNet-AIN generalises to unseen target datasets without target-domain adaptation.

  • OSNet learns omni-scale representations by explicitly modeling multiple scales within each building block.A unified aggregation gate dynamically fuses multi-scale features to produce omni-scale features.
  • OSNet-AIN integrates instance normalisation into OSNet through differentiable architecture search for cross-domain generalisation.
  • OSNet achieves state-of-the-art performance in the same-domain person re-identification setting.
  • OSNet is a lightweight architecture that is much smaller than ResNet-based competitors.
  • OSNet-AIN beats most recent unsupervised domain adaptation methods on unseen target datasets without per-domain target-data adaptation.

APPENDIX A MULTI-SOURCE DOMAIN GENERALISATION

The appendix evaluates OSNet-AIN for multi-source domain generalisation by training on three re-identification datasets and testing on the remaining dataset. Instance normalisation improves OSNet, while architecture search further improves performance over hand-engineered integration.

  • MULTI-SOURCE DOMAIN GENERALISATION: Multi-source domain generalisation trains on multiple source datasets and tests on the remaining dataset.
  • Datasets: The experiments use Market1501, Duke, CUHK03, and MSMT17, with three datasets for training and one for testing.Only the training splits of the source datasets are used during training.
  • Training Details: All models use cross-entropy loss with label smoothing and a single classification layer covering identities from all datasets.
  • Results: Instance normalisation improves plain OSNet’s performance, especially on the challenging MSMT17 test domain.
  • Results: Architecture search further improves performance, with OSNet-AIN outperforming hand-engineered OSNet-IBN.
Loading 1910.06827v5…