Detecting and Tracking Small and Dense Moving Objects in Satellite Videos: A Benchmark

I. INTRODUCTION

• Satellite-video research also lacks high-quality, publicly accessible datasets and benchmarks for fair, comprehensive evaluation.Source: §I · p. 1
• VISO provides 47 Jilin-1 satellite videos with 1,646,038 labeled object instances for satellite-video detection and tracking research.The paper describes VISO as the first well-annotated satellite-video dataset for these tasks.Source: §I · p. 2
• MMB combines foreground detection and background modeling to detect moving objects, with experiments reporting higher detection rates and fewer false alarms.Source: §I · p. 2
• The authors benchmark existing detectors and trackers across moving-object detection, single-object tracking, and multi-object tracking.Source: §I · p. 2

A. Satellite-based Object Detection and Tracking

• Satellite objects have low resolution and limited appearance and texture information, challenging methods that rely heavily on appearance features.Source: A. Satellite-based Object Detection… · p. 2

1) Moving Object Detection in Satellite Videos:

• Frame differencing and background modeling are common satellite-video detection approaches, but both rely on consistent illumination and frame registration.Deep-learning approaches have also been explored, though the cited work notes difficulty generalizing them without large, well-annotated public datasets.Source: 1) Moving Object Detection in… · p. 2

2) Object Tracking in Satellite Videos:

• Existing satellite-video tracking work has limited deep-learning coverage, and a public dataset can support further development in the area.The passage notes that existing trackers are limited to large objects and that little literature applies deep learning to single-object tracking in satellite videos.Source: 2) Object Tracking in Satellite Videos · p. 3

B. Datasets for Moving Object Detection and Tracking

• VISO’s sequential data distinguishes it from other object-detection datasets, and the paper establishes the first satellite-video benchmark across detection and tracking tasks.Source: B. Datasets for Moving Object… · p. 3

1) Videos Collection:

• VISO comprises 47 Jilin-1 videos covering urban areas and diverse satellite-orbit positions, with real-world challenges including complex backgrounds, illumination variation, and dense lanes.The dataset includes true-color sequences at 10 frames per second and scenes spanning several square kilometers.Source: 1) Videos Collection · pp. 3–4

2) Labelling Process:

• Four vehicle categories—car, airplane, ship, and train—are manually annotated with unoriented horizontal bounding boxes.The boxes are described by center coordinates, width, and height; the paper links the unoriented choice to the vehicles’ rigidity and small size.Source: 2) Labelling Process · p. 4
• For tracking, each instance receives a consistent identity across frames, and sequences are labeled with seven visual attributes, including occlusion subtypes.The listed attributes are background clutter, color change, low resolution, out-of-view, occlusion, similar object, and motion blur; occlusion is divided into partial and full.Source: 2) Labelling Process · p. 4
• Annotators received timely feedback, and bounding boxes were cross-checked to improve label quality and consistency.Source: 2) Labelling Process · p. 4

1) Image size:

• Original 12,000×5,000 satellite frames are divided into 1,000×1,000 patches for annotation and preprocessing.The paper contrasts these frame dimensions with regular-dataset images, which are mostly under 1,000×1,000.Source: 1) Image size · p. 4

2) Instance size:

• 90% of VISO instances are smaller than 50 pixels, usually about 4×4, unlike the middle-size-dominated PASCAL VOC and NWPU VHR-10 datasets.The paper identifies this instance-size distribution as a challenge for detection and tracking.Source: 2) Instance size · p. 4, Table IV · p. 5

3) Instance density:

• Up to 200 instances occur in a VISO frame, compared with an average of 7.7 in MSCOCO.The paper notes that most VISO images contain relatively many instances and that densely packed targets challenge existing methods.Source: 3) Instance density · p. 5

4) Instance Aspect Ratio:

• The dataset statistics include horizontal-box aspect ratios, instance counts per frame, and instance pixel counts.Source: Figure 2 · p. 4, 4) Instance Aspect Ratio · p. 5

5) Dataset Split:

• Detection uses image-level train, validation, and test splits; SOT and MOT use video-level splits with tracklet counts reported for MOT.The detection split contains 13,470, 535, and 3,725 images, respectively; MOT has 3,159, 440, and 658 tracklets across its splits.Source: 5) Dataset Split · p. 5

C. Dataset Challenges

• Nearly 90% of instances have fewer than 50 pixels, while frames contain many similar objects with partial occlusion, motion blur, and illumination variation.The paper identifies these properties as challenges for methods relying on appearance information.Source: C. Dataset Challenges · p. 5

A. Overview

• MMB combines AMFD candidate pixels and regions, LRMC background subtraction, and a trajectory-based filter for moving-object detection and false-alarm reduction.The pipeline is designed for the dataset’s extremely small objects and limited appearance information.Source: A. Overview · p. 5

B. Accumulative Multi-Frame Differencing

• AMFD uses differences across three neighboring frames, accumulating and normalizing the masks to suppress noise and highlight slow-moving objects.The method is motivated by the difficulty of detecting slow motion and removing object-edge ghosts from frame pairs.Source: B. Accumulative Multi-Frame Differencing · pp. 5–6
• AMFD binarizes its accumulated response using a threshold determined from that response rather than a fixed threshold; k is empirically set to 4.The threshold formula uses the response image’s mean and standard deviation.Source: B. Accumulative Multi-Frame Differencing · p. 6
• Connected regions are screened by object size and aspect ratio to distinguish targets from false alarms, with hyperparameters set empirically from dataset statistics.Source: B. Accumulative Multi-Frame Differencing · p. 7

• LRMC models video frames as a low-rank background matrix plus sparse foreground perturbations, recovering the background subspace to detect moving objects.This branch complements AMFD, which can produce incomplete objects and false targets.Source: C. LRMC for Object Detection · p. 7
• The method uses groups of frames to balance observation-matrix quantity and background variation; too many matrices add computation without performance gain, while too few cannot model changing backgrounds.Source: C. LRMC for Object Detection · p. 7
• AMFD regions of interest can also reduce false alarms when the background changes dynamically.Source: C. LRMC for Object Detection · p. 7

• The filter uses candidate pixel blobs from AMFD and LRMC and reduces false alarms based on the expectation that real-object trajectories are continuous and regular.Noise trajectories are described as scattered and irregular.Source: D. Motion Trajectory based False Alarm… · p. 7

Image 2

• The pipeline uses length 5 and size 7×7, slightly larger than the usual 3×3 object size.All object-pixel locations are given as outputs.Source: Image 2 · pp. 8–9
• Candidate targets are checked across the current frame and its next five frames, with nearby object pixels matched by minimum Euclidean distance.The Hungarian algorithm finds optimal associations; unmatched frames are skipped.Source: Image 2 · pp. 8–9
• After five frames, a candidate is accepted when its occurrence count h exceeds H=3; candidates with h=3 or 4 are added to missed frames.The procedure repeats until all sequence images are checked.Source: Image 2 · p. 9

V. EVALUATION OF OBJECT DETECTION

• The evaluation section presents qualitative and quantitative tests of MMB against representative baselines.The stated evaluation covers the method and metrics introduced in the section.Source: §V · p. 9

1) Baseline methods:

• The comparison covers foreground detection, background modeling, low-rank decomposition, and deep-learning methods, including FD, GMM, and ClusterNet.Other listed methods include D&T, ABM, ViBe, and FRMC.Source: 1) Baseline methods · p. 9

2) Evaluation Metrics:

• Six measures are used: Precision, Recall, F1 score, PR curve, AP, and mAP.The PR curve describes the precision–recall tradeoff; AP is its area, and mAP is the mean AP.Source: 2) Evaluation Metrics · p. 9

3) The Evaluation Protocol:

• A predicted box counts as a true positive when it overlaps the ground truth, because IoU is sensitive to small shifts on objects occupying mostly 2–20 pixels.The section notes that a 1- or 2-pixel shift can markedly change IoU for an object around 4 pixels in size.Source: 3) The Evaluation Protocol · p. 9

B. Quantitative Results

• 10% higher average F1 than D&T, with the highest F1 on all seven videos; MMB also leads in mAP at 0.83.The authors report top average Precision and Recall as well.Source: B. Quantitative Results · pp. 9, 11, Table V · p. 10, Table VI · p. 10

C. Qualitative Results

• In video 1, illumination changes on the water surface coincide with considerable false alarms from FD, ABM, MGBS, GMM, AGMM, and ViBe.The objects of interest in that example move relatively slowly.Source: C. Qualitative Results · p. 11

E. Parameter Choice

• L=4 gives the best reported performance among tested values from 1 to 30, so the framework uses L=4.Source: E. Parameter Choice · p. 11

B. Quantitative Results

• MMB’s PR curves approach the upper-right corner across seven videos; it is less sensitive to changing backgrounds than ABM and GMM.FPCP and FRMC cannot accurately detect the changing area in video 7, while MMB retains satisfactory performance there.Source: B. Quantitative Results · p. 11, Figure 7 · p. 12

C. Qualitative Results

• Using AMFD, LRMC, and PF together improves precision, recall, and F1; PF also improves recall and reduces false alarms.The variants are summarized in Table VII.Source: C. Qualitative Results · p. 11, Table VII · p. 12

VI. EVALUATIONS OF OBJECT TRACKING

• VISO is used to evaluate existing satellite-video tracking methods, and the paper establishes its first SOT and MOT benchmarks for this setting.Source: §VI · pp. 11–12

1) Experimental Settings:

• The VISO single-object setup uses 3,159 tracklets spanning 1.12 million frames, with only the first-frame ground-truth box provided for initialization.The sequences include illumination variation, fast motion, similar appearance, full occlusion, and persistently low resolution.Source: 1) Experimental Settings · p. 12

2) Evaluation Metrics:

• DPR measures the percentage of frames with center-location error below 5 pixels, while OSR measures frames whose overlap exceeds a threshold.The 5-pixel DPR threshold reflects that most objects occupy only 10–50 pixels.Source: 2) Evaluation Metrics · p. 12

3) Experimental Analyses:

• 61.2% DPR (ECO) and 34.7% OSR (MCCT) are the best reported scores, while SiamRPN++ and SiamBAN drop significantly on satellite data.CFME also performs poorly, primarily because many objects are extremely small and low-resolution.Source: 3) Experimental Analyses · pp. 12–13, Table VIII · p. 13
• Trackers are prone to lose targets under occlusion, and fast motion challenges search regions sized in proportion to the target.The authors suggest instance-specific detection as a potential way to relocate targets after occlusion.Source: 3) Experimental Analyses · p. 13
• Future work could investigate incorporating motion and historical trajectories into trackers or applying super-resolution before tracking.The motivation is that satellite images inherently lack sufficient appearance and texture information.Source: 3) Experimental Analyses · p. 13

B. Multi-object Tracking Evaluation

• Six representative MOT algorithms are evaluated with the same per-frame detections, produced by the MMB method, for fair comparison.The tested methods include Kalman Filter, CMOT, SORT, FairMOT, DTTP, and D&T.Source: B. Multi-object Tracking Evaluation · p. 14

1) Experimental Settings:

• The seven-video MOT subset contains 658 tracklets and 89,509 bounding boxes, with objects that can enter or leave scenes and require stable identities.Scenes may contain dozens of objects, and their count can vary over time.Source: 1) Experimental Settings · p. 14

2) Evaluation Metrics:

• MOTA combines false positives, missed targets, and identity switches; negative MOTA means tracker errors exceed the number of objects in a video.The metric summarizes accumulated tracking errors.Source: 2) Evaluation Metrics · p. 15
• MOTP assesses tracker precision through bounding-box overlap and/or center distance, while MT, PT, and ML categorize trajectory coverage.MT requires coverage of more than 80% of a ground-truth trajectory; ML means less than 20%.Source: 2) Evaluation Metrics · p. 15
• Additional evaluation counts include false positives, false negatives, fragmented trajectories, and identity switches.FM counts interruptions in a ground-truth trajectory; IDs counts identity switches.Source: 2) Evaluation Metrics · p. 15

3) Experimental Analyses:

• 73.6% MOTA is the best result for Kalman Filter, and 28.6% MOTP is the best for SORT, while most sequences are classified as mostly lost.The authors conclude that existing generic-video MOT trackers do not generalize directly to satellite video.Source: 3) Experimental Analyses · p. 15, Table IX · p. 16
• Frequent identity switches in densely packed scenes degrade overall tracking performance, with small, similar-looking objects, occlusion, and illumination variation adding difficulty.The qualitative analysis attributes the challenge to objects’ similar appearance and texture in satellite video.Source: 3) Experimental Analyses · pp. 15–16

VII. DISCUSSION

• VISO’s 47 sequences average 325 frames over 32 seconds, making the dataset suitable for persistent-tracking evaluation, but frame differencing and background modeling are less effective on temporarily static objects.Persistent tracking requires tracking objects once they begin moving for as long as they remain visible.Source: §VII · p. 16

VIII. CONCLUSION

• The conclusion reports that the proposed detector combines frame differencing and background subtraction and achieves high detection performance with low false alarms on VISO.The paper also introduces an annotated 47-video Jilin-1 dataset and evaluates representative detectors and trackers.Source: §VIII · p. 16