PaperlayerView exact revision on arXiv

Source-linked AI summary

FPV-CRASH: A Dataset for Offline Crash Moment Localization in Drone First-Person-View Videos

Jifan Zhang, Tingyu Wang, Qian Liu, Zhedong Zheng

arXiv:2610.19785v1cs.CV

TL;DR

Existing crash-video datasets and annotation schemes do not target major-impact localization in onboard drone FPV footage. FPV-CRASH provides a fixed-protocol dataset and heatmap-based video-only timestamp model; its reported results improve over a zero-training timing prior, within a clip-disjoint but non-source-disjoint evaluation.

  • Problem

    Existing crash-video datasets and annotation schemes are not designed to localize the major physical impact in UAV onboard FPV videos, although that timestamp provides a temporal anchor for retrospective analysis.

    Source: §I · p. 1

  • Method

    FPV-CRASH pairs class-balanced FPV clips and single-frame Crash Moment references with a heatmap-based continuous-timestamp model that uses sparse sampling and video-only localization inference.

    Source: §I · p. 1

  • Results

    0.5585s MAE, 0.3310s MedAE, and 65.71% W@0.5; MAE is reduced 50% and W@0.5 gains 28.6 points over the strongest zero-training prior.

    Source: §VI · p. 8

  • Takeaways & Limitations

    FPV-CRASH provides a fixed-protocol reference evaluation for offline Crash Moment Localization, with video-only inference and auxiliary caption-based retrieval.

    Source: §I · p. 1, §VI · p. 7

    The clip-disjoint split is not source-disjoint, so the results do not establish unseen-source generalization.

    Source: §VI · p. 8

Abstract

from arXiv · show

Crash Moment Localization aims to localize the major physical impact that interrupts controlled flight in an onboard first-person-view (FPV) video recorded by a drone, thereby providing a temporal reference for retrospective crash analysis in aerial robotics. However, existing crash-video datasets predominantly target surveillance or dash-camera footage, and their annotation schemes and task formulations are not designed for temporal localization of the major physical impact in UAV onboard FPV videos. To address this gap, we present FPV-CRASH, a dataset and fixed-protocol reference evaluation for offline Crash Moment Localization in drone FPV videos. FPV-CRASH contains 1,052 drone FPV clips (526 Crash and 526 Normal); each Crash clip has a single-frame reference annotation for the Crash Moment, and every clip is paired with four English captions automatically generated from sampled frames and the known Crash/Normal label. Normal clips carry no timestamp target and provide training-time background supervision. The captions support an auxiliary Text-to-Video Retrieval task, whereas localization inference remains video-only. We further provide a heatmap-based reference model that estimates a continuous Crash Moment timestamp from sparsely sampled frames. Under the fixed, non-source-disjoint clip-level protocol, the reference model reduces MAE from 1.12 s for the strongest fixed timing-prior baseline to 0.56 s (a 50% relative reduction) and achieves a median absolute error (MedAE) of 0.33 s; the fraction of clips localized within 0.5 s rises from 37.14% to 65.71%. The release includes annotations, available source video identifiers and URLs, clipping metadata, fixed split lists, and preprocessing code so that the reportable portion can be reconstructed without redistributing source footage. At submission time, all source-compilation URLs remained accessible.

I. INTRODUCTION

FPV-CRASH addresses the lack of temporal labels for major impacts in drone FPV videos by defining offline Crash Moment Localization. It pairs a class-balanced dataset and fixed protocol with a heatmap-based continuous-timestamp reference model.

I. INTRODUCTION

  • Localizing the major physical impact provides a temporal anchor for retrospective FPV crash analysis and separates pre-impact flight from post-impact evidence.The passage contrasts this retrospective task with prior UAV work focused largely on online operation,,,.Source: §I · p. 1
  • The fixed protocol evaluates localization on Crash clips, while Normal clips provide background supervision and inference remains video-only.The task returns one continuous timestamp from a complete video, without a text query.Source: §I · p. 1
  • 1,052 class-balanced FPV clips include single-frame Crash Moment references and four automatically generated captions per clip.The dataset contains 526 Crash clips and 526 Normal clips, with 4,208 captions in total.Source: §I · p. 1
  • A heatmap-based reference model estimates continuous Crash Moment timestamps from sparsely sampled frames under the fixed evaluation protocol.The captions also support auxiliary video–text alignment and retrieval, while localization inference remains video-only.Source: §I · p. 1
  • Removing all heatmap BCE terms or slot encoding increases MAE from 0.5585s to 0.8508s and 1.4221s, respectively.These are the largest observed degradations among the controlled component ablations.Source: §I · p. 1

II. RELATED WORK

Prior work covers UAV flight, collision response, temporal localization, and multimodal alignment, but does not provide crash-centered frame-level annotations for onboard FPV footage. FPV-CRASH supplies such data with a defined impact criterion and single-frame references.

II. RELATED WORK

  • Prior UAV datasets and methods address flight, collision response, accident forensics, or other localization tasks, rather than retrospective localization of a physical crash impact.The related work distinguishes this task from online prediction, avoidance, and failure diagnosis,,,,,,.Source: §II · p. 2

III. FPV-CRASH DATASET

  • FPV-CRASH provides continuous onboard FPV clips with Crash/Normal labels, single-frame Crash Moment annotations for Crash clips, and English captions.Source: §III · p. 2

A. Crash Definition and Annotation Protocol

  • A Crash requires visible or highly credible physical contact causing loss of controlled flight or flight termination; the labeled moment is the major impact after which control is not recovered.Earlier light contact is not the Crash Moment; uncontrolled descents or abrupt endings without credible contact are excluded.Source: A. Crash Definition and Annotation… · p. 2
  • The annotation criteria may under-represent impacts when contact evidence is occluded, poorly illuminated, or missing.Source: A. Crash Definition and Annotation… · p. 3
  • Each Crash clip receives a single-frame reference timestamp after repeated playback and frame-level inspection, with ambiguous samples excluded.The timestamp is a temporal reference rather than a uniquely precise physical instant; independent re-annotation covered 53 clips, with agreement within 0.5s on 85.7% and Cohen’s κ = 0.88 for impact presence.Source: A. Crash Definition and Annotation… · p. 3

B. Data Collection and Curation

FPV-CRASH is curated from public FPV compilations into class-balanced Crash and Normal clips, with captions and fixed clip-level splits. Its statistics and split design bound how the reported evaluation should be interpreted.

B. Data Collection and Curation

  • The 1,052 clips come from approximately 40 crash compilations and three normal-flight compilations, with cross-source near-duplicate scanning identifying 13 pairs above the threshold.Clip boundaries retain continuous pre-impact flight, the major collision, and sufficient post-impact evidence.Source: B. Data Collection and Curation · p. 3
  • Each clip has four English captions generated from sampled frames and its known class, but captions receive no formal human correctness audit.Captions contain no Crash Moment timestamp and are unnecessary for video-only localization inference.Source: B. Data Collection and Curation · p. 3
  • Table I presents the FPV-CRASH dataset composition and fixed splits.No per-clip source URL appears in the paper body.Source: Table I · p. 3, B. Data Collection and Curation · p. 3

C. Dataset Statistics and Splits

  • The clip-disjoint 60/20/20 split is not source-disjoint, and source-disjoint evaluation changes Crash MAE from 0.5585 to 0.7239s.The results characterize this protocol, not generalization to unseen sources, scenes, flights, or drone platforms.Source: C. Dataset Statistics and Splits · p. 3
  • Crash clips average 10.61s in duration and 8.47s before impact, while Normal clips average approximately 8.71s.The classes are broadly, not strictly, duration-matched.Source: C. Dataset Statistics and Splits · p. 3

D. Comparison with Existing Datasets

FPV-CRASH combines balanced Crash/Normal FPV clips and single-frame impact references, addressing a gap in available UAV datasets. The reference model adds temporal video modeling and a task-specific heatmap branch, with a separate shared embedding pathway for alignment and retrieval.

D. Comparison with Existing Datasets

  • FPV-CRASH combines balanced Crash/Normal FPV clips, single-frame Crash Moment references, and automatically generated captions.The comparison table covers representative onboard UAV and temporal-video datasets; their scale units are not directly comparable across rows.Source: D. Comparison with Existing Datasets · p. 3, Table II · p. 4

IV. METHOD

  • Given a complete clip, the task returns one continuous timestamp and evaluates localization only on Crash videos.Normal videos have no timestamp target; frames are divided into K contiguous segments for sparse sampling.Source: §IV · p. 4

A. Model Architecture and Temporal Encoding

  • The temporal heatmap head adds slot-indexed sinusoidal positional encoding and a non-causal Temporal Transformer to projected frame features.A position-wise linear layer produces the Crash logits used for both heatmap supervision and timestamp decoding.Source: A. Model Architecture and Temporal… · p. 4
  • The temporal summary supports alignment and retrieval but does not directly enter Crash Moment decoding.The temporal summary and trainable BERT output are projected and normalized in a shared embedding space.Source: A. Model Architecture and Temporal… · p. 4

B. Crash Heatmap and Timestamp Decoding

Crash references are converted into Gaussian targets over sampled slots, while predicted logits yield both an interpretable heatmap and a continuous timestamp. The heatmap responses and timestamp-decoding distribution are distinct outputs with different normalization.

B. Crash Heatmap and Timestamp Decoding

  • A Gaussian target maps each Crash reference to sampled slots, with width scaled using the median positive gap between adjacent valid sampling times.Valid Normal positions receive zero; sigmoid responses form the supervised and visualized Crash heatmap.Source: B. Crash Heatmap and Timestamp Decoding · p. 4
  • The predicted Crash heatmap contains position-wise, uncalibrated responses, while a separate distribution over valid slots determines the continuous timestamp.Both outputs derive from the same Crash logits, but only the decoding distribution is normalized across sampling positions.Source: B. Crash Heatmap and Timestamp Decoding · p. 4
  • Masked softmax over valid slots and expectation decoding produce a continuous timestamp from the same Crash logits used for the heatmap.A Normal-video output is not a Crash Moment and is excluded from localization metrics.Source: B. Crash Heatmap and Timestamp Decoding · p. 4

C. Training Objective

The training objective combines heatmap and timestamp supervision with optional video–text alignment. Alignment supports bidirectional retrieval, while localization inference remains video-only.

C. Training Objective

  • For Crash videos, the timestamp loss is Smooth L1 between normalized predicted and reference timestamps; Normal clips have no timestamp target.The loss passage also describes heatmap supervision, with a separate weight controlling Normal background supervision.Source: C. Training Objective · p. 4
  • Text–video alignment is used only during training, while localization inference is video-only.The shared embedding space also supports Text-to-Video Retrieval.Source: Figure 3 · p. 5
  • Matched video–text pairs serve as positives in a symmetric contrastive objective, and the aligned embeddings support bidirectional retrieval.Video-to-text retrieval reverses the text-query ranking procedure.Source: C. Training Objective · p. 5

V. EXPERIMENTS

The experiments use a fixed clip-level protocol, sparse frame sampling, and a temporal model to evaluate continuous crash-timestamp localization. They define localization metrics and specify training and comparison settings, including timing-prior baselines.

Implementation details

  • Training samples one frame from each of 10 temporal segments, while validation and testing use segment-center frames.The frozen SigLIP2 ViT-B/16 backbone and cached frame features are used unless otherwise specified; the Temporal Transformer has two layers, dimension 512, and eight heads.Source: §V · p. 5

Protocol and metrics

  • The evaluation uses all 105 Crash test videos without classification filtering; Normal videos have no localization target.Results are reported across three seeds with bootstrap 95% confidence intervals over the test clips; differences below 0.12 s MAE or 9 points W@0.5 are treated as unresolved.Source: §V · pp. 5–6
  • The protocol reports MAE, MedAE, Within@δ, and P90, with lower P90 indicating better localization.The timing-prior baseline table covers 105 Crash test clips and reports errors in seconds.Source: §V · p. 5, Table III · p. 6

Model configurations

  • The train-relative-median, train-relative-mean, and video-center priors provide zero-training timing references without using visual evidence.The strongest prior by MAE is the train-relative-median prior; the video-center prior predicts the clip midpoint.Source: §V · p. 6

A. Reference Localization Results

On the fixed test split, SigLIP2 Full localizes Crash Moments more accurately than the strongest timing-prior baseline. Its predictions track reference timestamps across the observed range, though duration-related error patterns remain difficult to disentangle from the supervision design.

A. Reference Localization Results

  • MAE falls from 1.1159s for the train-relative-median prior to 0.5585s for Full, while MedAE falls from 0.6610s to 0.3310s.W@0.5 rises from 37.14% to 65.71%, and P90 falls from 2.88s to 1.17s.Source: A. Reference Localization Results · p. 6, Figure 4 · p. 6
  • Full predictions broadly follow the identity line, and their error distribution shifts toward smaller errors than the train-relative-median prior.The figure plots Full predictions against reference timestamps above empirical absolute-error CDFs for Full and the prior.Source: A. Reference Localization Results · p. 6, Figure 4 · p. 6
  • Absolute error has modest rank associations with clip duration (ρ = 0.348) and pre-impact duration (ρ = 0.279), but their effects cannot be separated from target construction.Because σi ≈ 0.05 Ti, longer clips receive proportionally wider supervision kernels; predicting τ/T with constant normalized σ would decouple these factors.Source: A. Reference Localization Results · p. 6

B. Backbone Comparison

The observed Full-model localization differences across frozen backbones are small and do not establish a reliable backbone advantage. The best observed metric varies by backbone.

B. Backbone Comparison

  • Full-model MAEs span 0.5516–0.5688s across the backbones, and the 0.0172-s spread does not establish a reliable difference.EVA01 yields the lowest MAE, SigLIP2 the highest W@0.5, and PE-Core the lowest MedAE among Full models.Source: B. Backbone Comparison · p. 6
  • EVA01-CLIP-g/14 is roughly an order of magnitude larger than the two ViT-B/16 backbones, complicating interpretation of the observed MAE spread.The comparison holds the split, temporal architecture, sampling protocol, and prediction head fixed.Source: B. Backbone Comparison · p. 6
  • The ablation tables cover SigLIP2 localization variants and an approximately step-matched Crash-only control, rather than additional backbone comparisons.Table V retains the Temporal Transformer across variants; Table VI compares 760 and 780 optimizer updates.Source: Table V · p. 7, Table VI · p. 7

C. Ablation Studies

The ablations show that localization outcomes depend on slot encoding, heatmap losses, and training composition, while caption-related effects are not consistent across settings. Auxiliary retrieval results are reported with explicit limits on their scope.

Effect of primary components

  • Removing slot encoding raises MAE from 0.5585s to 1.4221s in Crash-and-Normal training, while caption effects differ across training compositions.With PE, adding text changes MAE from 0.6012s to 0.5585s for Crash-and-Normal training, but from 0.6292s to 0.6494s for Crash-only training.Source: C. Ablation Studies · p. 7

Crash-only training control

  • The approximately step-matched Crash-only PE-only MAE is 0.5879s versus 0.6012s with Crash-and-Normal training, so Normal clips do not uniformly improve localization.With text, Crash-only training yields 0.7089s versus 0.5585s for Crash-and-Normal; this comparison cannot isolate Normal-background supervision.Source: C. Ablation Studies · p. 7

Ablation studies

  • For SigLIP2 full trained with crash-and-normal data, the w/o-all-heatmap-BCE variant removes both Crash and Normal heatmap terms; other variants remove their named component.Source: Table VII · p. 7

Training objectives

  • Removing Normal-background BCE changes MAE from 0.5585s to 0.5900s and W@0.5 from 65.71% to 57.14%, while Normal responses rise.With Normal-background BCE disabled, additionally removing Crash heatmap BCE raises MAE to 0.8508s; removing timestamp loss has limited, metric-dependent effects.Source: C. Ablation Studies · p. 7, Table VII · p. 7

Auxiliary Cross-modal Retrieval

  • SigLIP2 achieves T2V R@1 of 41.90% and the highest observed R@1 in both retrieval directions, while EVA01 leads V2T R@5 and R@10.Class-conditioned captions, one caption per video, and no external baseline or free-query relevance labels limit general-purpose retrieval claims.Source: C. Ablation Studies · p. 7, Table VIII · p. 7

VI. CONCLUSION AND LIMITATIONS

FPV-CRASH provides a fixed-protocol reference evaluation for offline Crash Moment Localization, with a heatmap-based model that infers timestamps from video alone. The reported results are bounded by dataset and evaluation scope, and the paper identifies validation and coverage gaps for future work.

VI. CONCLUSION AND LIMITATIONS

  • The heatmap-based model provides video-only inference for the fixed-protocol offline Crash Moment Localization evaluation.Predicted timestamps can anchor automatic extraction of short pre- and post-impact windows for offline crash-footage analysis.Source: §VI · pp. 7–8
  • SigLIP2 Full obtains 0.5585s MAE, 0.3310s MedAE, and 65.71% W@0.5, with a 50% MAE reduction and 28.6-point W@0.5 gain over the strongest zero-training prior.Three-seed comparisons do not show a reliable caption benefit across training compositions.Source: §VI · p. 8
  • Conclusions are limited by dataset scale and source/scene coverage, 53 independently reannotated Crash clips, unaudited class-conditioned captions, and a clip-disjoint but non-source-disjoint split.The results do not establish unseen-source generalization; sensitivity to sampling, heatmap settings, loss weights, and external domains is only partially characterized.Source: §VI · p. 8
  • Future work will broaden coverage, validate captions and temporal annotations, and explore denser temporal representations.The paper does not frame the system as automated incident reporting and notes privacy and legal obligations around evidentiary use.Source: §VI · p. 8

Related papers

Loading 2610.19785v1…