Source-linked AI summary
Crafting Better Contrastive Views for Siamese Representation Learning
Xiangyu Peng, Kai Wang, Zheng Zhu, Mang Wang, Yang You
TL;DR
Random cropping can overlook image semantics and yield object–background or overly similar positive views. ContrastiveCrop uses semantic-aware localization and center-suppressed sampling; it improves classification across four Siamese methods and reports superior downstream detection and segmentation results.
Problem
RandomCrop overlooks image semantics and may produce object–background false positives or views with insufficient variance for contrastive learning.
Source: §1 · p. 2Method
ContrastiveCrop combines unsupervised semantic-aware localization with center-suppressed sampling to generate contrastive views.
Source: §1 · p. 2, §3.2 · p. 4, §3.3 · p. 5Results
ContrastiveCrop improves SimCLR, MoCo, BYOL, and SimSiam by 0.4%–2.0% classification accuracy across four datasets, with superior downstream detection and segmentation results.
Source: §1 · p. 2Takeaways & Limitations
The authors conclude that ContrastiveCrop is effective across several state-of-the-art contrastive methods and considers both semantic information and crop variance.
Source: §5 · pp. 8–9
Abstract
from arXiv · showhide
Recent self-supervised contrastive learning methods greatly benefit from the Siamese structure that aims at minimizing distances between positive pairs. For high performance Siamese representation learning, one of the keys is to design good contrastive pairs. Most previous works simply apply random sampling to make different crops of the same image, which overlooks the semantic information that may degrade the quality of views. In this work, we propose ContrastiveCrop, which could effectively generate better crops for Siamese representation learning. Firstly, a semantic-aware object localization strategy is proposed within the training process in a fully unsupervised manner. This guides us to generate contrastive views which could avoid most false positives (i.e., object vs. background). Moreover, we empirically find that views with similar appearances are trivial for the Siamese model training. Thus, a center-suppressed sampling is further designed to enlarge the variance of crops. Remarkably, our method takes a careful consideration of positive pairs for contrastive learning with negligible extra training overhead. As a plug-and-play and framework-agnostic module, ContrastiveCrop consistently improves SimCLR, MoCo, BYOL, SimSiam by 0.4% ~ 2.0% classification accuracy on CIFAR-10, CIFAR-100, Tiny ImageNet and STL-10. Superior results are also achieved on downstream detection and segmentation tasks when pre-trained on ImageNet-1K.
1. Introduction
The introduction argues that random crops can create object–background false positives and overly similar views, weakening positive-pair selection. ContrastiveCrop combines semantic-aware localization with center-suppressed sampling and improves several Siamese methods across classification and downstream tasks.
1. Introduction
- RandomCrop ignores image semantics and can pair an object with background or produce views with insufficient variance.The authors argue that optimizing object–background pairs can mislead representation learning, while highly similar views are trivial for learning discriminative models.Source: §1 · p. 2
- ContrastiveCrop combines semantic-aware localization to avoid most false positives with center-suppressed sampling to reduce trivial positive pairs.The two strategies guide crops toward the object and toward different parts of it.Source: §1 · p. 2
- ContrastiveCrop improves SimCLR, MoCo, BYOL, and SimSiam by 0.4%–2.0% classification accuracy across CIFAR-10, CIFAR-100, Tiny ImageNet, and STL-10, with superior downstream detection and segmentation results.The method has negligible extra training overhead.Source: §1 · p. 2
- The paper identifies investigating RandomCrop for contrastive learning and crafting better views for Siamese representation learning as its contributions.The authors describe this as the first work to investigate the commonly used RandomCrop in contrastive learning.Source: §1 · p. 2
2. Related works
The related-work section introduces contrastive learning and reviews approaches to selecting positive views. It situates ContrastiveCrop among Siamese frameworks and crop-selection methods.
2. Related works
- Contrastive learning pulls positive pairs together and pushes negatives apart in the embedding space, supporting visual representation learning without annotation.Siamese-structure methods have achieved strong downstream performance, with some surpassing supervised models.Source: §2.1 · p. 2
2.1. Contrastive Learning
- SimCLR uses a nonlinear transformation head, MoCo a momentum-updated memory bank, BYOL online prediction of a target network, and SimSiam stop-gradient.BYOL and SimSiam learn without negative samples.Source: §2.1 · p. 2
2.2. Positives Selection
- The localization box starts as the whole image and is periodically updated using the latest heatmap to guide crop generation toward the object of interest.The stated goal is guidance for crop generation, not precise localization.Source: Figure 2 · p. 3
- Positive-view methods include augmentation pipelines, InfoMin Augmentation, unsupervised saliency-constrained crops, and object-scene crops requiring external proposal algorithms.The cited works commonly retain RandomCrop as their basic sampling method; ContrastiveCrop is proposed without external functions.Source: §2.2 · pp. 2–3
3. Method
ContrastiveCrop combines semantic-aware localization with center-suppressed sampling to form more informative, varied positive views for Siamese representation learning. Its method builds from RandomCrop, uses training heatmaps to guide crop placement, and evaluates how the combined design changes positive pairs.
3.1. Preliminary
- RandomCrop samples crop scale and aspect ratio, then selects a location equiprobably, which can miss objects and create false positives.The crop is represented by its center coordinates, height, and width.Source: §3.1 · p. 3
3.2. Semantic-aware Localization
- Semantic-aware localization uses training heatmaps to form an object bounding box, progressively updates it, and constrains crop centers within the box to reduce false positives.The box is guidance rather than precise localization; its activation threshold controls its size, and the method is reported to work across a wide threshold range.Source: §3.2 · pp. 4–5
3.3. Center-suppressed Sampling
- Center-suppressed sampling uses a U-shaped beta distribution to push crop centers toward the operable region’s borders, reducing overlap and enlarging crop variance.The center-suppressed distribution is combined with localization in ContrastiveCrop and can be integrated into general contrastive frameworks.Source: §3.3 · p. 5
3.4. Discussion
- ContrastiveCrop conveys more semantic information at equal variance and achieves larger variance at equal semantic information than RandomCrop, while producing less-similar positive pairs during training.Localization alone slightly increases positive-pair similarity; adding center-suppressed sampling reduces it.Source: Figure 4 · p. 5, Figure 5 · p. 5, §3.4 · p. 6
4. Experiments
ContrastiveCrop is evaluated across multiple datasets and Siamese methods, with linear-classification gains, downstream-task results, and ablations examining its localization and sampling choices. The experiments also test compatibility with other transformations.
4.1. Datasets & Baseline Approaches
- Experiments use CIFAR-10, CIFAR-100, Tiny ImageNet, STL-10, and ImageNet, with SimCLR, MoCo V1/V2, BYOL, and SimSiam as baselines.Small-dataset experiments use ResNet-18; ImageNet experiments use ResNet-50.Source: §4.1 · p. 6, §4.2 · p. 6
4.2. Implementation Details
- The comparison keeps each baseline’s training setting unchanged; localization boxes update every 20% of training, adding negligible overhead.For ContrastiveCrop, RandomCrop is used before the first update; the implementation sets k=0.1 and α=0.6.Source: §4.2 · p. 6
4.4. Ablation Studies
- For center-suppressed sampling with k=0.1, α<1 outperforms localization alone, while α>1 lowers accuracy, supporting larger crop variance for stronger contrast.Figure 6b compares ContrastiveCrop across α with localization fixed at k=0.1.Source: Figure 6 · p. 7, §4.4 · p. 8
4.3. Linear Classification
- At least 0.4%, ContrastiveCrop improves small-dataset baselines; on IN-1K, it beats RandomCrop by 0.25% with SimCLR, 1.09% with MoCo V1, 0.49% with MoCo V2, and 0.33% with SimSiam.The IN-1K comparison uses linear classification; Table 2 covers linear-classification results on IN-200 and IN-1K.Source: Table 2 · p. 7, §4.3 · p. 7
4.4. Ablation Studies
- The ablation finds that localization beats RandomCrop for k from 0.05 to 0.2, but performance falls quickly above 0.25 as smaller boxes reduce view variance.Larger k produces smaller localization boxes.Source: §4.4 · p. 7
- The method outperforms the RandomCrop baseline with one localization-box update at 50% of training, and shows a larger improvement with more updates in the 10%–30% range.Source: Table 3 · p. 7, §4.4 · p. 7
4.5. Downstream Tasks
- On VOC detection, ContrastiveCrop gains +0.2AP, +0.2AP50, and +0.4AP75 over MoCo V1; on COCO, it leads in all reported detection and segmentation metrics.The VOC comparison uses Faster R-CNN; the COCO evaluation uses Mask R-CNN.Source: Table 4 · p. 8, §4.5 · p. 8
4.4. Ablation Studies
- Without other transformations, ContrastiveCrop is 0.4% higher than RandomCrop; with transformations, its advantage ranges from 0.3%–0.8% for one extra transformation to 1.2% with all included.The tested transformations are Flip, ColorJitter, Grayscale, and Blur; the authors report compatibility and orthogonality to other transformations.Source: Table 5 · p. 8, §4.4 · p. 8
5. Discussion and Conclusion
The conclusion presents ContrastiveCrop as a method for crafting Siamese-learning views by incorporating semantic information and investigating suitable crop variance. Experiments with SimCLR, MoCo, BYOL, and SimSiam support its effectiveness and generality.
5. Discussion and Conclusion
- ContrastiveCrop is tailored to make better contrastive views for Siamese representation learning by incorporating semantic information and investigating suitable crop variance.The authors hope the work will inspire future research on positive-pair design, given its role in contrastive learning.Source: §5 · pp. 8–9