Bag of Tricks and A Strong baseline for Image Copy Detection

1. Introduction

• Image copy detection asks whether a query is a modified copy of an image in a reference dataset, a task with social-media integrity applications and difficult real-life distractors and edits.The paper notes that many state-of-the-art solutions do not deliver satisfying results in real-life scenarios [8].Source: §1 · p. 1
• The approach replaces commonly used supervised pre-training with unsupervised pre-training and combines a training bag of tricks with descriptor stretching at test time.The baseline is trained on augmented images; Figure 2 names Barlow-Twins [36] for pre-training and lists GeM [26], WaveBlock [34], a high-dimension projector, a learnable matrix, two losses, and cosine-annealing warm-up.Source: §1 · p. 1, Figure 1 · p. 1, Figure 2 · p. 2
• Descriptor stretching rescales descriptors to stabilize scores across queries, and the authors report that it significantly improves retrieval accuracy.It approximates score normalization by stretching descriptors, addressing the Descriptor Track's use of Euclidean distance as the final matching score.Source: §1 · p. 1
• The paper presents its bag of tricks and strong baseline as effective for real-life social-media copy detection scenarios.These are listed among the paper's main contributions.Source: §1 · p. 1
• 3rd out of 526 participants: the baseline ranked third in the Image Similarity Challenge: Descriptor Track at NeurIPS’21.The challenge includes distractor images and diverse transformations intended to mimic social-media cases.Source: §1 · pp. 1–2

2.1. Copy Detection

• Classical global and local descriptors and CNN-extracted features are reviewed, but none gives satisfactory performance on challenging, large benchmarks like ISC2021.The paper also notes that copy-detection research is not widely known and that the task is sometimes considered easy [8].Source: §2.1 · p. 2

2.2. Unsupervised Pre-training

• For copy detection in ISC2021, unsupervised pre-trained models show superior performance to fully-supervised counterparts; this may be because their pre-training category is more similar to copy detection than the fully-supervised category.Source: §2.2 · p. 2

2.3. Deep Metric Learning

• Deep metric learning uses pair-based or proxy-based losses to learn discriminative features; the ISC2021 solution uses hard-mined triplet loss and cross-entropy loss.The paper characterizes these two losses as simple but effective.Source: §2.3 · p. 2

3.1. Unsupervised Pre-training

• The method uses Barlow-Twins pre-training on ImageNet because ISC2021’s tight copy categories differ from ImageNet classes, while direct self-supervised training is costly and had not yielded satisfying performance.The paper says Barlow-Twins was selected for the pre-trained model; it reports that unsupervised pre-training empirically outperformed some other unsupervised methods.Source: §3.1 · pp. 2–3

3.2. Training Tricks

• Training tricks combine GeM pooling, WaveBlock feature-level augmentation, and a high-dimension projector that becomes useful when the training set has many IDs.GeM’s learnable parameter shifts pooling from mean toward max pooling and tends to increase for copy detection; WaveBlocks are argued to improve ability to handle unseen augmentations.Source: §3.2 · p. 3
• The training design maps 2048-dimensional features through an 8192-dimensional projector and a learned 8192 × 256 matrix, with hard-mined triplet and cross-entropy losses.The 2048-dimensional feature is used for triplet loss; cross-entropy trains the 8192- and 256-dimensional features, with soft cross-entropy transferring 8192-dimensional predictions as labels.Source: §3.2 · pp. 3–4

3.3. Augmentation

• The augmentation set includes random crops, rotations, pixel and perspective changes, padding, color and blur changes, flips, overlays, and resizing.Figure 3 displays the set, with the first image identified as the resized original.Source: Figure 3 · p. 3, §3.3 · p. 4

3.4. Descriptor Stretching

• Descriptor stretching scales query features using a hyperparameter and the mean of their top-n training-set inner-product scores, then uses the stretched query feature as the final descriptor.The authors set α=2.5 and n=5, while noting that the strategy’s theoretical basis for stabilizing scores remains unclear.Source: §3.4 · p. 4

5. Experimental Settings

• The experimental setup uses Barlow-Twins pre-training with ResNet50, samples 100,000 training images, and applies 19 augmentations per image.Testing uses YOLOv5 overlay detection, four image scales, normalized feature fusion, and descriptor stretching on fused query features.Source: §5 · pp. 4–5

5.1. Comparison with State-of-the-Arts

• The proposed baseline shows promising performance against Phase 2 state-of-the-art leaderboard methods.The comparison results are presented in Table 1; the supplied passage does not provide its cell values.Source: §5.1 · p. 5, Table 1 · p. 5

5.2. Ablation Studies

• Over 14% improvement was obtained with unsupervised rather than supervised pre-training.The authors suggest this may reflect similarity between category definitions in unsupervised pre-training and copy detection.Source: §5.2 · p. 5
• 17.26% improvement in Micro-average Precision was reported with descriptor stretching, while Recall@Rank 1 changed from 76.18% to 76.10% and Recall@Rank 10 from 79.72% to 79.98%.The authors attribute the overall gain to comparable scores across queries, not improved per-query performance.Source: §5.2 · p. 5
• About 1% improvement was observed when cropping overlays, and multi-scale testing improved performance from 71.49% to 73.02%.The authors explain that an overlaid image may contain a true match while the underlying background does not.Source: §5.2 · p. 5

• The proposed baseline uses recent self-supervised pre-training methods instead of traditional supervised pre-training.Source: §6 · p. 5
• Descriptor stretching is proposed to stabilize scores across queries.Source: §6 · p. 5
• The authors hope the solution will benefit content tracing, copyright infringement, and misinformation applications.Source: §6 · p. 5