Source-linked AI summary
Looking Beyond Appearances: Synthetic Training Data for Deep CNNs in Re-identification
Igor Barros Barbosa, Marco Cristani, Barbara Caputo, Aleksander Rognhaugen, Theoharis Theoharis
TL;DR
Existing re-identification methods often depend on apparel and paired cross-camera training examples, limiting clothing-change recognition and complicating data preparation. The paper introduces SOMAnet, an Inception-based single-path network trained from scratch on the synthetic 100K-image SOMAset and fine-tuned on benchmarks. It reports state-of-the-art performance across four benchmarks and recognition across clothing changes using RGB data.
Problem
Re-identification commonly models apparel and assumes unchanged clothing, while siamese training requires costly preassembled positive and negative image pairs.
Method
SOMAnet is an Inception-based single-path feature extractor trained from scratch on synthetic SOMAset data and then fine-tuned on task-specific datasets.
Results
SOMAnet achieves state-of-the-art performance on CUHK03, Market-1501, RAiD, and RGBD-ID, and recognizes people across clothing changes using RGB data.
Takeaways & Limitations
Synthetic training data and single-path Inception architectures support learning structural body cues beyond apparel and enable probing the semantics captured by network neurons.
Takeaways & Limitations
SOMAset uses a single outdoor scene, which lacks the background variability present in other datasets and may also provide a reference for comparing individual sizes.
Abstract
from arXiv · showhide
Re-identification is generally carried out by encoding the appearance of a subject in terms of outfit, suggesting scenarios where people do not change their attire. In this paper we overcome this restriction, by proposing a framework based on a deep convolutional neural network, SOMAnet, that additionally models other discriminative aspects, namely, structural attributes of the human figure (e.g. height, obesity, gender). Our method is unique in many respects. First, SOMAnet is based on the Inception architecture, departing from the usual siamese framework. This spares expensive data preparation (pairing images across cameras) and allows the understanding of what the network learned. Second, and most notably, the training data consists of a synthetic 100K instance dataset, SOMAset, created by photorealistic human body generation software. Synthetic data represents a good compromise between realistic imagery, usually not required in re-identification since surveillance cameras capture low-resolution silhouettes, and complete control of the samples, which is useful in order to customize the data w.r.t. the surveillance scenario at-hand, e.g. ethnicity. SOMAnet, trained on SOMAset and fine-tuned on recent re-identification benchmarks, outperforms all competitors, matching subjects even with different apparel. The combination of synthetic data with Inception architectures opens up new research avenues in re-identification.
1 INTRODUCTION
Re-identification commonly relies on apparel and siamese networks, which limits recognition when clothing changes and requires costly image-pair preparation. The paper introduces SOMAnet, an Inception-based feature extractor trained with synthetic human imagery to capture clothing-independent structural cues.
- Siamese architectures learn cross-camera appearance differences from labeled positive and negative image pairs, but preparing those pairs increases training complexity.
- Re-identification systems traditionally encode identity mainly through apparel, so changing clothes between camera acquisitions degrades recognition.
- SOMAnet uses an Inception-based feature extractor instead of a siamese setup, making internal neuron probing easier.
- SOMAnet captures structural human-body attributes, including somatotype and gender, alongside clothing information.
- SOMAset is a completely synthetic dataset of 100K images used to train SOMAnet from scratch, combining 50 human prototypes, clothing, poses, ethnicities, and rendered outdoor scenes.
- When trained on SOMAset and fine-tuned on other datasets, SOMAnet achieves state-of-the-art performance on CUHK03, Market-1501, RAiD, and RGBD-ID.
2 RELATED LITERATURE
Related work frames re-identification as matching people across cameras, with deep methods commonly using paired or triplet inputs. The paper positions single-path descriptors trained on synthetic data as an alternative that can probe learned characteristics and address clothing variation using RGB imagery.
- 2.1 Re-identification: Traditional re-identification assumes people retain the same clothes between camera acquisitions, although jackets or backpacks can change appearance within hours.
- 2.1 Re-identification: Few clothing-change approaches exist, and prior methods in this setting rely on RGB-D data rather than RGB alone.
- Deep re-identification methods typically feed cross-camera image pairs into siamese or pseudo-siamese networks, while triplet-loss methods compare three or more images.
- The proposed alternative learns a descriptor with a single-path network, first using synthetic training data and then fine-tuning on a specific dataset.
- Synthetic-data research has expanded toward automatically producing large collections spanning visual category semantics, while prior re-identification augmentation mainly altered background scenes.
3 THE SOMASET DATASET
SOMAset is a controlled synthetic re-identification collection designed to expose networks to body morphology, clothing changes, ethnicity variation, pose diversity, and changing viewpoints. It contains 100K rendered subject-clothing-pose instances generated from 50 human prototypes.
- SOMAset is introduced as a synthetic re-identification collection with a dedicated creation protocol and distinctive design features.
- The dataset models three somatotypes by varying height, weight, and body proportions across 25 male and 25 female 3D subjects.
- Each subject is rendered with multiple clothing sets to relax the usual assumption that identity is tied to one apparel appearance.
- Different skin colors are assigned across the prototypes to model ethnicity variation.
- The collection includes 250 motion-capture poses for each subject-clothing combination, covering forward, backward, and sideways walking.
- SOMAset represents 100K subject-clothing-pose combinations, calculated as 50 subjects × 8 clothing sets × 250 poses.
- Its rendered outdoor scene includes trees, buildings, pavement, grass, and a vehicle while viewpoints generate varied occupancy, lighting, and relative pose.
4 THE SOMANET ARCHITECTURE
SOMAnet uses an Inception-based feature-extraction architecture to learn compact descriptors from RGB images, avoiding the pairing requirements of siamese networks. Its design is adapted for re-identification through reduced depth, bounded embeddings, and fine-tuning for new tasks.
- SOMAnet computes a concise RGB representation of high-level individual features for similarity-based re-identification.
- The network follows Inception V3 and uses convolutional, pooling, and cascading Inception modules.
- Difference to GoogLeNet: A SOMAset experiment found no performance gain from GoogLeNet’s deeper classification stages, motivating a four-module redesign without auxiliary outputs.
- SOMAnet’s fully connected layer produces a 256-dimensional vector bounded within [−1, 1], reducing the embedding from 2048 to 256 dimensions.
- Unlike siamese networks, SOMAnet uses a compact R256 descriptor with linear space and time requirements for feature extraction and distance evaluation.
- After fine-tuning, mirrored image descriptors are concatenated to form a 512-dimensional representation for more invariant matching.
5 EXPERIMENTS
The experiments evaluate SOMAnet and SOMAset through quantitative and qualitative analyses of network performance, learned human features, and synthetic training data.
- The experiments investigate SOMAnet and SOMAset through quantitative and qualitative analyses.
- The evaluation compares SOMAnet with other deep architectures.
- Additional experiments examine neuron-encoded human features and the contribution of synthetic training data.
5.1 Datasets
The study evaluates re-identification across four datasets representing camera, illumination, clothing, and modality challenges. It uses CMC and mAP comparisons, with dataset-specific protocols and fine-tuning setups.
- The evaluation compares peer-reviewed methods using Cumulative Matching Characteristic curves and mean Average Precision.
- CUHK03: CUHK03 contains 13,164 images of 1,467 identities captured by five cameras with different acquisition settings.
- Market-1501: Market-1501 contains 32,668 images of 1,501 identities from five high-resolution and one low-resolution camera.
- RAiD: RAiD contains 6,920 images of 41 identities across four cameras, including indoor and outdoor views with large illumination variations.
- RGBD-ID: RGBD-ID contains four data groups from 79 identities and was originally designed to explore depth data in re-identification.
- RGBD-ID: For RGBD-ID, SOMAnet is fine-tuned on Collaborative and Backwards groups while Walking2 serves as probe and Walking1 as gallery.
5.2 Analysis of SOMAnet
SOMAnet achieves strong re-identification performance with an Inception-based architecture and reveals neurons encoding structural human attributes beyond clothing. Experiments show state-of-the-art results across benchmarks, including clothing-independent recognition from RGB data.
- Architecture and representation: A discernibility score combines fire-rate and activation scores to rank neurons by sensitivity to a target characteristic.The score measures both how selectively and how strongly a neuron responds to images in set C rather than set R.
- Neuron probing: On SOMAset, the top 10 discernibility-ranked neurons frequently identified obese subjects in the exploration set.The analysis used 64,000 localization images and 36,000 exploration images, with obesity defined using two obese subjects versus 30 others.
- Neuron probing: All female subjects in the exploration set were detected correctly, while gender discernibility decreased faster than obesity discernibility.The authors interpret the faster decrease as potentially indicating that gender requires fewer neurons to detect.
- Clothing-independent recognition: On RGBD-ID, SOMAnet recognized subjects using RGB data alone despite clothing changes, while competing approaches used silhouettes or RGB-D information.The SOMAset-trained model achieved a Market-1501 single-shot mAP of 47.89%.
5.3 Analysis of SOMAset
SOMAset provides effective synthetic pretraining for SOMAnet, with benefits from cross-dataset learning and broader subject diversity. The experiments also indicate that adding subjects is more useful than adding poses at comparable dataset sizes.
- Training from scratch: Cross-dataset learning generally improves performance, except for single-shot testing on CUHK03, where performance essentially does not change.Fine-tuning Market-1501 after training from scratch on CUHK03 performs better than the reverse, possibly because Market-1501 is larger.
- Training from scratch: SOMAset pretraining produces systematically very significant improvements after fine-tuning.The authors describe synthetic data as an economic and effective proxy for real data that supports later specialization.
- Number of subjects: Adding SOMAset subjects yields a roughly linear, but mild, increase in rank 1 and mAP performance.The reduced-dataset study compares the original 50-subject SOMAset with versions containing 32, 16, and 8 subjects, using Market-1501 for fine-tuning and testing.
- Number of subjects: A 1% performance increase corresponds to 7.5 additional subjects matched correctly at rank 1 on Market-1501's 750-subject testing set.The authors emphasize that fine-tuning is more crucial than training from scratch in deep-network practice.
- Number of poses: At matched image counts, using more subjects is more advantageous than using more poses, because different subjects provide greater visual intraclass variability.The comparison uses reduced datasets formed by varying either subjects or poses.
5.4 Effects of Illumination, poses and camera viewpoints
The rendering-factor experiments compare balanced controls with datasets degraded by restricted poses, poor viewpoints, or poor illumination. Restricted poses are reported as the strongest degradation factor for rank 1 performance.
- Performance effects: Restricted poses degrade Rank 1 performance more than bad viewpoints or bad illumination at both tested dataset sizes.The comparison uses 16,000- and 32,000-image SOMAset variants, all fine-tuned and tested on Market-1501.
- Performance effects: The Balanced Control Group performs best across both 16,000- and 32,000-image datasets.mAP generally follows the Rank 1 pattern, except that bad viewpoints produce a paradoxical mAP drop when increasing from 16,000 to 32,000 images.
- Interpretation: Overfitting to a degraded dataset is easier when its size is smaller.This statement is offered to account for the observed behavior of the degraded variants.
6 CONCLUSIONS
The paper concludes that synthetic data can initialize single-path Inception networks that capture bodily structure in addition to apparel. SOMAnet can therefore match people across clothing changes, while the scope of synthetic-data scaling and background variability remains open.
- Conclusions: Synthetic training data can greatly help initialize deep networks for re-identification.The authors argue that re-identification need not rely exclusively on siamese architectures.
- Conclusions: SOMAnet captures structural human-body aspects such as somatotype beyond clothing appearance.The conclusion identifies synthetic data as a proxy for real data and reports that training from scratch supports later fine-tuning.
- Conclusions: SOMAnet trained on SOMAset2 can match people even when they change apparel between camera acquisitions.The stated capability uses the proposed Inception-based network and synthetic training dataset.
- Open questions: The performance plateau as the number of synthetic subjects increases remains an open issue.The authors report a somewhat linear performance increase with added diverse subjects but do not identify where saturation occurs.
- Open questions: SOMAset uses a single scene, limiting background variability; the authors report worse recognition when the background is replaced by a homogeneous grey area.The paper also leaves the effect of training-image realism for high-resolution re-identification settings unresolved.