TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-captured Scenarios

• Drone images challenge detection through highly variable object scales, dense objects and occlusion, and confusing geographic elements across large coverage areas.Source: Figure 1 · p. 1, §1 · p. 2
• TPH-YOLOv5 adds a tiny-object head for scale variation, replaces YOLOv5 prediction heads with TPH for dense scenes, and uses CBAM to find regions of interest in wide-coverage images.Source: §1 · p. 2
• Training augmentation, multi-scale testing, and model ensembles support scale adaptation and stronger inference; a ResNet18 classifier for confusing categories adds 0.8%–1.0% AP, alongside filtering of ineffective tricks.Source: §1 · p. 2
• 39.18% AP on VisDrone2021 test-challenge outperforms DPNetV3 by 1.81%; TPH-YOLOv5 places fifth in the VisDrone2021 DET challenge, with a minor gap to first place.Source: §1 · p. 2

2.1. Data Augmentation

• Data augmentation combines photometric and geometric distortions with multi-image methods: MixUp blends samples, CutMix covers regions, and Mosaic stitches four images; TPH-YOLOv5 combines these with traditional methods.Photometric distortions adjust hue, saturation, and value, while geometric methods include scaling, cropping, translation, shearing, and rotation.Source: §2.1 · p. 3

2.2. Multi-Model Ensemble Method in Object Detection

• Ensembling reduces prediction variance; NMS keeps the highest-confidence box above an IoU threshold, Soft-NMS decays neighboring scores, and WBF merges boxes, which TPH-YOLOv5 uses over NMS.The passage reports that Soft-NMS improves over traditional NMS on standard benchmarks and that WBF performs much better than NMS.Source: §2.2 · p. 3

2.3. Object Detection

• Detectors combine backbone feature extraction, neck feature-map aggregation, and head localization and classification; one-stage models predict boxes and classes together, offering speed but lower accuracy than two-stage models.The passages distinguish one-stage and two-stage detectors and describe common backbones and neck aggregation structures.Source: §2.3 · pp. 3–4

3.1. Overview of YOLOv5

• YOLOv5x is selected for VisDrone2021 training because its AP exceeds the other three YOLOv5 variants by more than 1.5%, despite higher training computation cost.Training uses Mosaic and MixUp augmentation, with photometric and geometric distortion parameters adjusted for drone imagery.Source: §3.1 · p. 4

3.2. TPH-YOLOv5

• An added high-resolution prediction head gives TPH-YOLOv5 four heads to detect tiny objects and ease the effects of large scale variation.The added head increases computation and memory cost.Source: §3.2 · p. 4
• Transformer encoder blocks in the heads and backbone end capture contextual features for dense, occluded objects, while low-resolution placement limits computation and memory cost.Each encoder uses multi-head attention and an MLP with residual connections; the authors report better performance on occluded, high-density objects in VisDrone2021.Source: §3.2 · p. 5, Figure 4 · p. 5
• CBAM [54] sequentially refines feature maps along channel and spatial dimensions so TPH-YOLOv5 can focus on targets amid confusing geographical elements.The lightweight module multiplies the inferred attention maps with the input feature map.Source: §3.2 · p. 5, Figure 5 · p. 5
• Inference combines six image scales and horizontal flips per model with NMS, then fuses predictions from five models using WBF.The image is scaled to 1.3 times, then reduced to 1, 0.83, and 0.67 times before horizontal flipping.Source: §3.2 · p. 5
• 0.8%–1.0% AP improvement is reported with a self-trained ResNet18 classifier trained on ground-truth crops resized to 64×64.The classifier addresses poor classification performance, particularly for hard categories such as tricycle and awning-tricycle.Source: §3.2 · pp. 5–6

4. Experiments

• Evaluation used VisDrone2021 testset-challenge and testset-dev, reporting mAP averaged over 10 IoU thresholds from [0.5: 0.95] and AP50.The paper states that VisDrone2021-DET is the same dataset as VisDrone2019-DET and VisDrone2018-DET.Source: §4 · p. 6

4.1. Implementation Details

• Training used 65 epochs with two warm-up epochs, Adam, an initial learning rate of 3e-4, and 1536-pixel long-side images at batch size 2.The learning rate followed a cosine schedule and decayed to 0.12 of its initial value in the last epoch; training used transferred YOLOv5x weights.Source: §4.1 · p. 6
• Covering objects smaller than 3 pixels with gray squares improved mAP by 0.2 on the processed training data.At 1536-pixel input size, 622 of 342391 labels were smaller than 3 pixels and described as hard to recognize; Figure 7 illustrates this issue.Source: §4.1 · p. 6, Figure 7 · p. 6
• Multi-scale testing resized test images to three sizes and flipped each horizontally, then fused predictions from all six versions.The authors describe averaging predictions across augmented test images as a way to improve prediction performance.Source: §4.1 · p. 6

4.2. Comparisons with the State-of-the-art

• 39.18 on VisDrone2021 testset-challenge ranked fifth, 0.25 below first place’s 39.43 and above VisDrone2020’s best score of 37.37.The reported challenge-server results were limited to four models and the final ensemble of five models.Source: §4.2 · p. 6, Table 1 · p. 7

4.3. Ablation Studies

• On testset-dev, the extra tiny-object head increased model layers from 607 to 719 and GFLOPs from 219.0 to 259.0, while transformer encoders reduced them to 705 and 237.3.The paper reports that the extra head improves mAP and small-object detection, while transformer blocks increase mAP and aid dense- and large-object detection. Five models with varied input sizes, category weights, and one YOLOv5l backbone were ensembled; Figure 9 shows representative detections of large, tiny, and dense objects.Source: §4.3 · p. 7

Conclusion

• TPH-YOLOv5 achieved state-of-the-art performance on the VisDrone2021 dataset.The authors state that their experiments refreshed the dataset’s record.Source: §5 · p. 7
• The detector adds transformer encoder blocks, CBAM, and practical techniques to YOLOv5 for drone-captured object detection.The authors report trying many features and using some to improve detector accuracy.Source: §5 · p. 7
• Visualizations on testset-challenge show localization of tiny, dense, and motion-blurred objects.The figure describes different bounding-box colors for different categories.Source: Figure 9 · p. 8