PaperlayerView exact revision on arXiv

Source-linked AI summary

Luggage re-identification: A study of strategies towards a real-world application

Quentin Dhôte, Romaric Audigier

arXiv:2610.40643v1cs.CVcs.AI

TL;DR

Real-world luggage ReID raises challenges beyond clean, cropped-image matching, including occlusion and incomplete visibility. This paper studies an end-to-end multi-camera pipeline and finds that filtering unreliable observations before aggregation improves representations.

  • Problem

    Existing luggage ReID approaches focus on cropped-image appearance matching under clean detections, while practical surveillance raises unresolved questions about cameras, observations, and multi-view aggregation.

    Source: §I · p. 1

  • Method

    The paper proposes an end-to-end multi-camera pipeline that uses scene-specific gallery and query representations, including confidence-based filtering and feature aggregation.

    Source: §I · p. 1, C. Feature Extraction and Aggregation · p. 3

  • Results

    Confidence-based filtering combined with feature aggregation performs best, while aggregating all frames without filtering degrades performance and cross-environment transfer still requires dataset-specific fine-tuning.

    Source: C. Results · p. 4, 3) Dataset and Domain Analysis · p. 6

  • Takeaways & Limitations

    Multi-camera observations and feature aggregation are important for robustness to occlusions and missed detections in realistic luggage surveillance.

    Source: §V · p. 6

    Cross-environment generalization remains limited: fine-tuning on the target dataset leads to the best performance.

    Source: 3) Dataset and Domain Analysis · p. 6

Abstract

from arXiv · show

Luggage re-identification (ReID) is a critical task in video surveillance for forensics, security or border management (suspicious or abandoned luggage recognition), as well as luggage management (lost luggage recovery). It enables the association of observations of the same luggage across disjoint camera views in complex environments such as airports, train stations and border crossing. While existing approaches primarily focus on appearance-based cropped image matching under relatively clean conditions, real-world scenarios are characterized by frequent occlusions, cluttered background and crowded scenes which significantly degrade performance. Yet, strategies leveraging the complementarity of multiple shots and views from different video streams is overlooked. In this paper, we study strategies for an end-to-end pipeline that operates from multiple video streams to detect and re-identify luggage. We evaluate different variants on a benchmark dataset acquired in real-world environments to address the questions of selecting and merging different views and shots of the same luggage as well as training on different environment datasets for an overall ReID performance improvement in practical applications.

I. INTRODUCTION

The introduction frames luggage ReID as a practical surveillance problem that remains underexplored beyond clean, cropped-image matching, and proposes a multi-camera pipeline for asymmetric enrollment and monitoring. It positions this work against luggage-ReID methods – and whole-pipeline approaches [9]–.

I. INTRODUCTION

  • The proposed end-to-end pipeline targets asymmetric enrollment and monitoring, combining multi-view and temporal aggregation to handle occlusions, missed detections, and fragmented observations.The introduction also identifies camera configuration, shot selection, aggregation, and cross-dataset adaptation as experimental questions.Source: §I · p. 1

A. Object and Luggage Re-Identification

  • Prior luggage ReID methods – mainly learn appearance features from precropped images and assume clean detections, limiting their fit to real surveillance.QuadNet, for example, uses a quadruplet loss for multi-view representations robust to viewpoint variation.Source: A. Object and Luggage Re-Identification · p. 1

B. Whole ReID Pipelines for Real-World Applications

  • Existing tracking and unified detection-ReID pipelines [9]– do not explicitly address cross-scene, multi-camera luggage ReID under fragmented visibility.Commercial baggage systems use controlled enrollment and query acquisitions with isolated luggage, unlike the asymmetric scenario here.Source: B. Whole ReID Pipelines for Real-World… · pp. 1–2

C. Occlusion and Multi-View Robust Re-Identification

Prior occlusion and multi-view methods offer relevant strategies, but often assume structured motion or complete, synchronized observations. The paper frames luggage ReID across controlled gallery and unconstrained query scenes, whose observations differ in visibility and track continuity.

C. Occlusion and Multi-View Robust Re-Identification

  • Occlusion methods include part-based representations, attention, temporal modeling, and pose-guided or part-discovery strategies,, but mostly target person ReID with structured motion.Source: C. Occlusion and Multi-View Robust… · p. 2
  • Multi-view approaches,, combine complementary viewpoints but typically assume synchronized or fully observed data, unlike asymmetric surveillance with fragmented observations.Source: C. Occlusion and Multi-View Robust… · p. 2

III. METHOD

  • The paper's task matches luggage enrolled in a controlled, conveyor-belt gallery to query observations from a complex scene with partial visibility and fragmented tracks.The query scene also involves irregular motion, preventing consistent cross-camera track association.Source: §III · p. 2

A. Pipeline Overview

The pipeline processes video streams through detection and tracking, scene-specific representation construction, feature extraction, and cross-scene matching. It exploits structured gallery views alongside temporal redundancy in query tracks.

A. Pipeline Overview

2) Scene-Specific Representation Construction:

3) Feature Extraction:

  • A deep network extracts appearance features from each detected instance for subsequent aggregation according to the selected multi-shot strategy.Source: 3) Feature Extraction · p. 2

4) Cross-Scene Matching:

  • Each query-track representation is compared with gallery tracks using a similarity metric, and gallery candidates are ranked.Source: 4) Cross-Scene Matching · p. 2

A. Pipeline Overview

  • The design pairs structured multi-view gallery information with temporal redundancy in query observations to mitigate occlusions and viewpoint variation.Source: 4) Cross-Scene Matching · p. 2

B. Detection and Multi-Object Tracking

Detection and tracking produce luggage tracks for the pipeline, with different association strategies reflecting the controlled gallery and unconstrained surveillance settings.

B. Detection and Multi-Object Tracking

1) Detection:

  • The detector returns bounding boxes and confidence scores, retaining only detections above a threshold to reduce false positives.Source: 1) Detection · p. 2

2) Tracking in the Gallery Scene:

3) Tracking in the Query Scene:

Query luggage is tracked independently in each camera because motion is unpredictable and occlusions make reliable cross-camera association infeasible. The method filters low-confidence detections, averages the remaining features for each track, and uses structured multi-camera gallery views without temporal aggregation.

3) Tracking in the Query Scene:

  • Unpredictable motion and frequent occlusions prevent reliable cross-camera association in the query scene, so tracking runs independently per camera and produces potentially fragmented tracks.Source: 3) Tracking in the Query Scene · p. 3

C. Feature Extraction and Aggregation

  • Each detection is cropped, resized, and passed through a convolutional network to produce a feature vector.The vector dimension d is the network's output dimension.Source: C. Feature Extraction and Aggregation · p. 3
  • For each query track, the method filters detections by confidence and averages the remaining feature vectors into a multi-shot representation.The retained detections satisfy sci ≥ θc; filtering before averaging reduces the impact of occlusions and noise without added weighting.Source: C. Feature Extraction and Aggregation · p. 3
  • Gallery instances use one observation from each camera, with no temporal aggregation because the gallery provides structured views with limited occlusion.Source: C. Feature Extraction and Aggregation · p. 3

D. Cross-Scene Matching

The pipeline matches one aggregated query-track representation against multiple gallery-camera features, ranking luggage identities by their best-matched view. It is evaluated on a real-airport dataset with separate conveyor-belt gallery and passenger-corridor query scenes.

D. Cross-Scene Matching

  • Each query feature is compared independently with every gallery-camera feature, and each luggage identity is ranked by its best-ranked occurrence.This matching formulation uses the complementary gallery viewpoints when some views are less informative or occluded.Source: D. Cross-Scene Matching · p. 3

A. Dataset Construction

  • The dataset pairs a conveyor-belt gallery scene with a query scene where passengers carry luggage through a customs corridor.Source: A. Dataset Construction · pp. 3–4
  • The real-airport dataset contains 116 bags, 579 gallery tracks, and 745 query tracks, with approximately 18 images per track on average.It uses five gallery cameras and six query cameras; matching instances were manually annotated across scenes.Source: A. Dataset Construction · p. 4

B. Implementation Details

Confidence-filtered feature aggregation achieves the best reported performance among the tested variants, although the score-based single-frame baseline is slightly better at rank-10. The evaluation uses a pretrained and fine-tuned ReID model and reports bidirectional matching metrics.

B. Implementation Details

  • The implementation uses TransReID pretrained on MVB and fine-tuned on the constructed dataset, with Euclidean distance for query-to-gallery comparisons.Gallery instances use five camera images, while each query track is represented by an aggregated feature vector.Source: B. Implementation Details · p. 4
  • Evaluation reports CMC at ranks 1, 5, and 10 and mAP in both query-to-gallery and gallery-to-query directions.Source: C. Results · p. 4

C. Results

  • The proposed confidence-filtered aggregation achieves the best performance, while using all frames without filtering degrades performance relative to the score-based single-frame baseline.The score-based baseline selects the highest-confidence frame; the proposed variant aggregates the five most confident detections.Source: Table I · p. 4, C. Results · p. 4
  • The score-based single-frame baseline is slightly better at rank-10, but the proposed method outperforms it at rank-1 and mAP.The authors identify rank-1 and mAP as more important for practical deployment.Source: C. Results · pp. 4–5
  • Detection and tracking take 6.1 seconds for synchronized images from one scene's cameras, while ReID takes 30 ms per query.Source: C. Results · p. 5

D. Ablation Study

The ablation study examines multi-camera configurations, temporal aggregation strategies, and dataset characteristics.

D. Ablation Study

  • Ablations assess multi-camera configurations, temporal aggregation strategies, and dataset characteristics.Source: D. Ablation Study · p. 5

1) Impact of Multi-Camera Configurations:

Camera choice matters, and single-camera restrictions affect query and gallery directions differently. Multi-camera views help recover matches missed by one view, although difficult detections remain a challenge.

1) Impact of Multi-Camera Configurations:

2) Impact of Temporal Aggregation:

The study compares temporal aggregation across different numbers and selections of query observations. Aggregating features improves same-K performance, while selection strategy and inference time shape the trade-offs.

2) Impact of Temporal Aggregation:

  • The study varies K and compares top-K observations selected by detection confidence or bounding-box size, with and without feature aggregation.Source: 2) Impact of Temporal Aggregation · pp. 5–6
  • Feature aggregation consistently improves performance at the same K when observations are selected by detection confidence, whereas increasing K without aggregation gradually decreases it.Table IV reports the K-shot study.Source: Table IV · p. 6, 2) Impact of Temporal Aggregation · p. 6
  • Using all available observations significantly increases inference time, creating an accuracy–computational-cost trade-off.Source: 2) Impact of Temporal Aggregation · p. 6
  • BBox-size selection generally performs worse than confidence selection at low K but can become comparable or slightly better at larger K.The text identifies confidence as a good trade-off for selecting a few reliable observations and size-based selection as potentially retaining complementary appearance information at larger K.Source: 2) Impact of Temporal Aggregation · p. 6

3) Dataset and Domain Analysis:

The experiments find domain gaps both between an external dataset and the target environment and between similar acquisition sites. Fine-tuning improves results, and the conclusion identifies unsupervised domain adaptation as future work for deployment without additional annotation.

3) Dataset and Domain Analysis:

  • The experiments compare MVB, CD-1, and CD-2 train/test splits; CD-2 appears easier, likely because its more uniform background reduces visual clutter.Source: 3) Dataset and Domain Analysis · p. 6
  • CD-1 fine-tuning substantially improves on direct MVB-pretrained testing, and CD-2-specific fine-tuning yields the best CD-2 performance after MVB or CD-1 transfer.Table V presents the dataset analysis results.Source: Table V · p. 6, 3) Dataset and Domain Analysis · p. 6

V. CONCLUSION AND FUTURE WORK

  • The conclusion identifies unsupervised domain adaptation as future work to maintain performance in new environments without additional annotation cost.Source: §V · p. 6

Related papers

Loading 2610.40643v1…