Training with synthetic data for drone detection in thermal imagery

• LWIR G2A drone detection is difficult because reduced texture, sensor noise, radiometric variability, and weak target-background thermal contrast hinder small-target detection, while scarce costly annotations constrain supervised training.Source: §1 · p. 1
• The paper develops synthetic-first LWIR training with IR-like backgrounds, Blender-based scenes varying drone geometry, pose and environments, thermal-like targets, sensor-noise post-processing, automatic labels, and targeted real-data fine-tuning.Source: §1 · pp. 1–2
• Its dataset-analysis framework measures pixel-level radiometric properties, detector feature-space semantic alignment, and structural diversity, then relates these dimensions to downstream detection performance.Source: §1 · pp. 1–2
• The study evaluates synthetic pre-training plus limited real LWIR fine-tuning across detector families and examines bird imagery as hard negatives, which may help some configurations but risk domain mismatch.Source: §1 · p. 2

2.1 Object detection in LWIR

• CNN and Transformer advances enable real-time detection,, [12],, but LWIR constraints make sensor-matched training important and annotated data costly to scale across conditions,,.LWIR is limited by lower spatial resolution and signal-to-noise ratio, atmospheric scattering, background noise, limited texture, and sensor sensitivity variability.Source: §2.1 · p. 2

2.2 Small target and drone detection

• For small IR targets, data and annotation curation can outweigh dataset volume, while multi-sensor coverage, hard negatives,, architecture adaptations,, and temporal cues offer complementary gains.Birds are common hard negatives because their apparent size, silhouette, and trajectory resemble drones,,.Source: §2.2 · p. 2

2.3 Synthetic data for training object detection models

• Synthetic data offers controllable scenes and ground-truth labels, but transfer depends on training design, augmentation, domain randomization, architecture, dataset diversity, and real–synthetic mixing,,,, [28],,.Remote-sensing generation such as AeroGen must preserve object scale, layout, localization, and annotation consistency; LWIR additionally requires background diversity and thermal realism.Source: §2.3 · p. 2

2.4 Strategies for synthetic data generation

• The pipeline provides scalable IR-like data with controllable geometry, pose, background diversity, and automatic labels, but is not fully radiometric; real LWIR fine-tuning remains necessary for sensor-domain alignment.Javidnia also shows that 16-bit-to-8-bit thermal tone mapping can materially affect object detection performance.Source: §2.4 · p. 3

2.5 Evaluating the quality of synthetic data

• The synthetic-to-real gap can occur at pixel and spatial-frequency levels; EMD, GLCM statistics, and Sobel gradients measure image differences, while KID and FID measure semantic differences, –.The cited work links the gap to limited diversity, poor domain fit, low fidelity, and differences between rendering engines and real microbolometer sensors.Source: §2.5 · p. 3

3.1 Synthetic IR image generation

• The pipeline generated synthetic IR scenes, post-processed them, and trained object detectors for the G2A drone-detection task.The work adopted synthetic scenes to address the scarcity of large, diverse, consistently annotated IR datasets and generated them at reduced resolution to reflect IR cameras’ lower spatial resolution.Source: §3 · p. 3, §3.1 · p. 3
• 1,362 filtered false-color IR-like backgrounds were produced using Florence-2 and FLUX, but they were not physically accurate thermal renderings.The process began with 2,000 web images, of which 1,751 were retained after filtering; some generated outputs with irregularities were excluded.Source: §3.1 · p. 4
• 20K synthetic scenes contained 29,945 annotated quadcopter instances, created by placing and animating drones in Blender environments.Scenes were rendered at 512 x 512 pixels; randomized emission strength and increased intensity on darker backgrounds helped maintain target contrast.Source: §3.1 · p. 4
• Gaussian noise, Poisson noise, and 5 x 5 Gaussian filtering simulated sensor noise, photon-counting variation, and blur in generated scenes.The degradations were applied simultaneously or exclusively, and noise-based degradations were added to a random subset of scenes.Source: §3.1 · pp. 4–5

3.2 Models

• YOLOv13, RF-DETR, and Faster R-CNN represented one-stage, transformer-based, and two-stage detectors, evaluated across multiple scales.Training compared SIR pretraining followed by fine-tuning on 100 FTMIR images, SIR plus 19,875 RBRGB samples followed by fine-tuning on 200 real IR images, and a 100-image RMIR-only baseline. Validation used 3K real IR multicopter samples and mAP across IoU thresholds of 0.50 to 0.95; architecture-specific training recipes were used, so cross-family comparisons are indicative rather than strictly controlled.Source: §3.2 · p. 5

4.1 Datasets

• Fine-tuning benchmarks use a 100-sample subset of real data to reflect data scarcity across sensors and drone models, though it may underrepresent object-level diversity.Real data consisted of processed 8-bit images; birds served as hard negatives during training and were excluded from validation.Source: §4.1 · pp. 5–6

4.2 Dataset Analysis Methodology

• The analysis uses pixel-level radiometric statistics, Kernel DINO Distance for semantic alignment, and Vendi Score for structural diversity as diagnostic tools, not formal causal predictors.KDD estimates distance from the real-world validation baseline in DINOv2 feature space; Vendi Score estimates the effective number of unique visual concepts. Metrics were applied to full images and target crops.Source: §4.2 · p. 6

4.3 Pixel-level physics and hardware divergence

• Real imagery has higher full-image HTV than synthetic data (Cohen's d = 1.242) and over four times the target-crop Sobel Gradient Variance (d = 1.224).The authors associate full-image roughness with AGC-like contrast stretching; post-processing noise makes synthetic full-image texture metrics comparable, while real drones show structural complexity from uneven heating, sensor noise, and optics.Source: §4.3 · pp. 6–7, Figure 3 · p. 7

4.4 Visual Semantic Domain Gap and Structural Diversity

• Synthetic imagery matches real target-crop diversity more closely than full-image diversity, but its feature-space distance from real validation remains high, so morphological alignment alone does not close the domain gap.The excess full-image diversity is driven largely by complex synthetic backgrounds; the authors also report that the 100-sample fine-tuning subset roughly matches the full set's characteristics.Source: §4.4 · p. 7, Figure 4 · p. 8
• Appending RGB birds to the synthetic baseline increases object-level KDD, whereas IR birds shift their fine-tuning base set less; this is not an isolated RGB-versus-IR ablation.Visible-spectrum birds add morphological variety but introduce cross-modal divergence from LWIR; IR birds stay closer to the validation baseline.Source: §4.4 · p. 7

5. RESULTS

• Adding RGB bird data to synthetic pre-training improves mAP@50:95 over synthetic-only pre-training across architectures, despite increasing the KDD domain gap from real LWIR imagery.The authors attribute the gain primarily to morphological similarity between birds and drones, which may strengthen discrimination and robustness despite the modality mismatch.Source: §5 · p. 8
• Except for Faster R-CNN, no synthetic pre-training strategy beats the real-only baseline, indicating that real LWIR fine-tuning is needed for sensor-domain alignment.The authors hypothesize Faster R-CNN struggles with low-resolution thermal features, and that background-driven latent diversity does not provide accurate thermal signals.Source: §5 · p. 8
• Across evaluated detector families, synthetic pre-training followed by single-modality real-data fine-tuning improves few-shot (100 samples) mAP@50:95 over real-only training.Faster R-CNN rises from 0.012 with real-only training to 0.352 with single-modality fine-tuning and 0.278 with dual-modality fine-tuning.Source: §5 · p. 8, Figure 5 · p. 9
• Bird distractors help RF-DETR variants consistently but slightly and YOLOv13-s slightly, while YOLOv13-n, YOLOv13-l, YOLOv13-xl, and Faster R-CNN show no apparent benefit.The authors suggest that small thermal fine-tuning datasets may not overwrite or suppress irrelevant RGB bird features, limiting modality-specific representations.Source: §5 · pp. 8–9
• RF-DETR-L with multi-modality fine-tuning reaches 0.723 mAP@50:95, versus 0.713 for single-modality fine-tuning; the small difference is uncertain without repeated-seed experiments.The model-performance comparison is shown in Figure 5.Source: §5 · p. 8, Figure 5 · p. 9

6. DISCUSSION

• Synthetic pre-training followed by fine-tuning on a small real LWIR dataset improved mAP@50:95 over limited real-only training, with RF-DETR-L achieving the highest performance in this setup.The authors conclude that synthetic data benefits initialization and generalization but does not replace real-world LWIR data.Source: §6 · p. 9
• Synthetic imagery captures object morphology but misses real LWIR heat structure and sensor properties, so even a small amount of real data is needed to align learned representations.The identified gaps include sensor noise, optics, compression, and camera-side processing; morphological diversity alone does not close the thermal reality gap.Source: §6 · p. 9, §6.1 · p. 9
• Bird imagery is a conditional robustness aid: it can provide aerial-object hard negatives, but its effects vary by detector and scale and may be affected by RGB–LWIR divergence.Several RF-DETR variants benefited, but the modality gap could help or hinder adaptation to the thermal domain.Source: §6 · p. 9
• RF-DETR achieved the strongest absolute results, but attributing this to transformer context and long-range interactions remains tentative because detector training recipes were not harmonized.The proposed explanation draws on prior infrared small-target work, not a definitive architecture comparison.Source: §6 · p. 9

6.1 Limitations and future work

• The synthetic pipeline produces IR-like rather than physically accurate LWIR imagery, motivating future thermal rendering, target heat-distribution modeling, and sensor-specific AGC simulation.These limitations constrain how faithfully the synthetic data represents physical and sensor-dependent properties of real imagery.Source: §6.1 · p. 9
• Limited sensor, environment, range, target, and deployment diversity means the results do not establish operational robustness; broader real LWIR and MWIR evaluation is needed.Future evaluations should use independent sensors and varied conditions, and consider real-time processing constraints; performance and latency benchmarks are reported in Appendix H.Source: §6.1 · p. 10
• The architecture comparison lacked harmonized ablations and repeated seeds, while too few dataset configurations prevented robust correlations between dataset metrics and accuracy.Future work should add controlled within-architecture tests, repeated runs, real-data learning curves, better-aligned hard negatives, and more configurations; consistent within-architecture gains support the conclusion under the evaluated conditions.Source: §6.1 · p. 10

• The complete matrix of benchmarking-model hyperparameters is provided in Table 2 for reproducibility and future comparative studies.Source: Appendix A · p. 13, Table 2 · p. 13

APPENDIX B: DATASET CONFIGURATIONS

• Validation used LWIR sequences [18],,, and fine-tuning used,, [57], with controlled reuse, campaign or location separation, and some overlapping drone models.Frames were quality-filtered and manually checked; validation was balanced across target positions and backgrounds, and existing bounding-box annotations were reviewed. Public LWIR images were processed 8-bit rather than raw radiometric data.Source: Appendix B · pp. 13–14
• A 100-sample real-world fine-tuning subset was selected using K-means in DINOv2 feature space, while dataset splits isolated synthetic, thermal, and visible-spectrum effects.The subset was intended to preserve the broader dataset's structural diversity and domain characteristics; split compositions and sampling rules are detailed in Table 3.Source: Appendix B · p. 14, Table 3 · p. 14

APPENDIX C: DATASET ANALYSIS METHODOLOGY

• Dataset-alignment statistics were normalized or sampled to reduce dataset-volume bias where possible, though high-resolution RGB bird data could still cause resolution-related distribution shifts.Luminance histograms were normalized by pixel count, but not all measurements could be fully normalized.Source: Appendix C · p. 14

C.1 Level I: Pixel-Level Radiometric and Textural Alignment

• The pixel-level analysis targets synthetic-to-real gaps in radiometry and spatial frequency, computing target-region contrast and gradients alongside frame- and crop-level radiometric statistics.Crops under 400 pixels were excluded to reduce histogram bin-starvation bias; that threshold was chosen to provide about 1.5 times as many pixels as the 256 histogram bins.Source: §C.1 · pp. 14–15

C.1.1 Radiometric Intensity, Quantization, and Contrast Alignment

• Radiometric alignment measures compare normalized mean-luminance histograms with 1D Earth Mover’s Distance, histogram roughness with HTV, and target visibility with Local Weber Contrast.HTV measures adjacent-bin intensity changes to capture AGC-related quantization; contrast uses an adaptively padded background region matched to the target's area.Source: §C.1.1 · p. 15

C.1.2 Signal-to-Noise and Texture Analysis

• Texture analysis uses rotation-invariant GLCM Homogeneity, Correlation, and Entropy statistics averaged across four orientations.The analysis characterizes micro-textures such as fixed-pattern noise and microbolometer grain.Source: §C.1.2 · p. 15

C.1.3 Intensity Gradient Distribution

• Gradient analysis measures variance in target-region Sobel gradient magnitudes and masks near-zero gradients to avoid bias from empty regions.This examines boundary softening from lens blur and thermal diffusion alongside localized fluctuations from heat sources and sensor noise.Source: §C.1.3 · p. 15

C.1.4 Dynamic Range Utilization

• Dynamic range utilization is measured as the difference between the 1st and 99th intensity percentiles, excluding dead pixels and extreme noise spikes while capturing usable sensor bandwidth.The measure evaluates how well synthetic imagery matches thermal cameras' AGC intensity stretching.Source: §C.1.4 · p. 15

C.2 Transitioning to Semantic Evaluation: Feature Extraction

• DINOv2 ViT-L/14-reg, provides a model-agnostic feature space, with full-frame and isolated-target-crop embeddings used to separate background context from the object-level domain gap.Source: §C.2 · p. 15
• 56×56 crops match the validation targets’ 43.3-pixel median size, while larger inputs inflate KDD and Vendi through upscaling artifacts, especially for synthetic data.Source: §C.2 · pp. 15–16, Figure 6 · p. 16
• Images were ImageNet-normalized, and global [CLS] embeddings were L2-normalized so metrics reflect structural alignment rather than raw activation magnitudes.Source: §C.2 · p. 16

C.3 Level II: Representation-Space Alignment (Kernel DINO Distance)

• Unlike FID, KDD uses an unbiased MMD estimate rather than Gaussian-distribution assumptions and Inception features that can miss semantic similarity,.Source: §C.3 · p. 16
• KDD is squared MMD between target and reference DINOv2 embeddings using the KID polynomial kernel, estimated over 500 subsets of 50 samples without replacement.Source: §C.3 · p. 16

C.4 Level III: Structural Diversity

• Vendi Score [60] estimates effective sample size from feature-similarity eigenvalue entropy, using standardized DINOv2 embeddings with a linear kernel.Source: §C.4 · p. 16
• The dual score caps maximum Vendi at D=1,024, easing volume bias for larger datasets but constraining few-shot sets and not eliminating sample-size or modality bias; Appendix F tests downsampling stability.Source: §C.4 · p. 16

APPENDIX D: PIXEL-LEVEL METRIC RESULTS

• Real LWIR luminance histograms show AGC quantization spikes, unlike continuous synthetic histograms, and real datasets differ in intensity peaks and dynamic-range use.The passage links this variation to camera-specific dynamic AGC stretching and cautions that a model trained on one sensor profile may generalize less readily to another.Source: Appendix D · pp. 16–17, Figure 7 · p. 17
• EMD against validation was 29.88 for SIR, 52.80 for S + RBRGB, 18.77 for Real FTMIR, and 13.61 for Real FTMIR + FTBIR.Adding RGB birds worsened radiometric alignment, while adding IR birds to FTMIR reduced EMD, indicating a closer pixel-distribution match to validation.Source: Appendix D · p. 17
• Full-image HTV divergence was d = 1.242 and dynamic-range-utilization divergence was d = 0.899 between validation and pure synthetic imagery.Despite these differences, median full-image GLCM Correlation was 0.994 synthetic versus 0.977 real validation, and Entropy was 9.862 synthetic versus 9.786 real validation.Source: Appendix D · p. 17, Figure 8 · p. 17, Table 4 · p. 18
• Real IR bird targets below 400 pixels were excluded from target histogram and GLCM calculations, but included for Local Weber Contrast and Sobel Gradient Variance.The threshold avoids bin-starvation artifacts that would inflate Quantization HTV and distort GLCM profiles.Source: Appendix D · p. 18
• For target crops meeting the 400-pixel threshold, synthetic targets had lower texture entropy (d = 0.974) and Sobel Gradient Variance (d = 1.224) than real targets.The analysis attributes this difference to relatively uniform synthetic meshes versus real LWIR crops with internal texture and localized intensity variation.Source: Appendix D · p. 18, Table 5 · p. 19

APPENDIX E: SEMANTIC GAP AND DATASET DIVERSITY RESULTS

• KDD and Vendi Score were computed for full frames and object crops to assess domain gap and diversity.Table 6 defines lower KDD as better; higher Vendi indicates greater diversity, with desirable values depending on target-domain alignment.Source: Appendix E · p. 19, Table 6 · p. 19

APPENDIX F: VENDI SCORE SENSITIVITY ANALYSIS

• Matched-volume evaluation preserves the relative hierarchy of randomly sampled datasets across tiers, while the native 100-sample FTMIR set is an exception optimized for feature-space coverage.The passage reports native FTMIR full images at 27.63 versus 20.82 for a random N=100 drawing, and says K-Means sampling has a diversity advantage over random sampling.Source: Appendix F · pp. 19–20, Table 7 · p. 19

APPENDIX G: MODEL PERFORMANCE

• Across models, fine-tuning generally improves recall, precision, mAP@50, and mAP@50:95, supporting synthetic IR scenes as a model-agnostic pre-training resource for LWIR drone detection.The passage notes a few fine-tuning exceptions.Source: Appendix G · p. 20, Table 8 · p. 20

SIR [20 K]

• The cited passages list four Faster-RCNN score rows, followed in order by 0.443/0.658/0.658/0.331, 0.488/0.672/0.672/0.364, 0.518/0.511/0.511/0.290, and 0.348/0.035/0.035/0.012.Source: SIR [20 K] · p. 21

APPENDIX H: MODEL PARAMETER COUNT, COMPUTATIONAL COMPLEXITY AND INFERENCE SPEED COMPARISON

• Inference latency was benchmarked on an NVIDIA T4 using TensorRT 11 at FP16, averaged over 100 warmed-up runs; Faster R-CNN instead used PyTorch FP32 because its dynamic RoI pooling was incompatible with TensorRT.Figure 10 compares mAP@50:95 with parameter count, GFLOPs, latency, and frame rate across training configurations.Source: Appendix H · p. 21, Figure 10 · p. 22