PaperlayerView exact revision on arXiv

Source-linked AI summary

Residual Kalman Dynamics for Event-Based UAV Forecasting

Per Nyblom, Hannes Ovrén, David Gustafsson

arXiv:2609.00839v1cs.CV

TL;DR

The paper studies short- and mid-horizon UAV bounding-box forecasting on FRED and asks how much performance reflects event evidence versus motion priors. It combines a full-box constant-velocity Kalman prior with learned residual corrections, then tests shortcut dependence through decorrelated subsets. Residual models improve the Kalman baselines, event-conditioned models retain useful signal after decorrelation, and canonical-split gains require cautious interpretation.

  • Problem

    FRED forecasting performance may reflect target appearance or event evidence, but future motion can also be predictable from position and velocity alone.

  • Method

    The paper learns acceleration-like residual corrections from a full center-size Kalman state, box history, filtered features, and local event representations, and evaluates decorrelated subsets.

  • Results

    Learned acceleration residuals consistently improve over full-box Kalman baselines, while event-conditioned residual models remain useful after position- and velocity-based shortcuts are weakened.

  • Takeaways & Limitations

    Canonical FRED gains combine useful event-conditioned signal with motion-prior structure, so reporting canonical and decorrelated results gives a more cautious picture of model behavior.

  • Takeaways & Limitations

    The decorrelation stress test weakens selected linear shortcuts but does not eliminate nonlinear priors, scene cues, target-scale effects, or other shortcuts.

Abstract

from arXiv · show

We study short- and mid-horizon UAV bounding-box forecasting on the FRED event-camera dataset. We use a constant-velocity Kalman filter over a full center-size box state as a strong physical baseline, and train a residual model to predict acceleration-like corrections from recent box history, filtered state features, and local event representations. This simple residual formulation consistently improves over the Kalman baseline, with event-conditioned models giving the strongest results among the evaluated methods. We further show that part of the residual target is predictable from anchor position and velocity alone, indicating that canonical FRED results can reflect both visual evidence and dataset-specific motion priors. To analyze this effect, we introduce decorrelated subsets as a diagnostic stress test, showing that event-conditioned residual models retain useful predictive signal even when measured position- and velocity-based shortcuts are weakened.

1 Introduction

Event cameras support low-latency UAV forecasting, but FRED results may combine visual evidence with dataset-specific motion priors. The paper addresses this with residual Kalman forecasting and decorrelation diagnostics.

  • Event-based UAV forecasting matters for anticipating rapid aerial maneuvers under low latency, especially in demanding perception settings.
  • Real event-camera UAV forecasting remains relatively new, with FRED providing annotated flights for detection, tracking, and trajectory forecasting.
  • A constant-velocity Kalman filter can miss acceleration and turns, while residual acceleration-like corrections better follow future UAV trajectories.
  • Future acceleration is partially predictable from anchor position and velocity, motivating decorrelated subsets that weaken measured motion-prior shortcuts.
  • The proposed residual model predicts acceleration corrections on top of a full center-size box state to improve short- and mid-horizon forecasts.

2 Problem Setup

The forecasting setup anchors each sample at a labeled time, represents boxes in normalized center-size coordinates, and predicts future boxes from history, event representations, and filtered features. Evaluation uses center and box metrics on canonical and decorrelated FRED splits.

  • 2.1 State and Forecast Target: Each sample is anchored at label time t0, with observed boxes through t0 and future boxes as the forecasting target.
  • 2.1 State and Forecast Target: Bounding boxes use normalized center-size coordinates [x_t, y_t, w_t, h_t], with centers, widths, and heights normalized by image dimensions.
  • 2.1 State and Forecast Target: The Kalman state augments all four box channels with first-order velocities, enabling full-box rather than center-only forecasting.
  • 2.2 Input Features: Inputs combine recent event representations, normalized box history, and features derived from the optimized Kalman filter.
  • 2.3 Metrics: Center ADE/FDE measure trajectory accuracy, while box ADE/FDE and mIoU additionally capture apparent target-size changes.
  • 2.4 Training and Test Split Policy: Evaluation uses the canonical FRED split and a decorrelated subset, with decorrelation removing 50% of selected training samples.

3 Method

The method uses a carefully tuned constant-velocity Kalman filter over full center-size box states, then learns acceleration-like residual corrections using history, filtered state, and event features. Motion-prior baselines and decorrelation diagnostics assess how much residual forecasting can be explained by position and velocity alone.

  • Kalman baseline: The constant-velocity Kalman filter models all four box channels—x, y, w, and h—with corresponding velocities as a competitive physical prior.It filters the observed history and propagates forward without future measurements.
  • Residual forecaster: The residual forecaster predicts acceleration-like corrections from event representation, recent box history, filtered rollout state, and forecast time.The residual acceleration is incorporated into box and velocity rollouts, while predicted box channels are clamped to [0, 1].
  • Residual formulation: The learned residual separates the constant-velocity component handled by the Kalman filter from remaining acceleration-like structure.This formulation is intended to improve over a tuned motion model rather than an under-optimized extrapolator.
  • Network structure: The architecture encodes 12 recent normalized box-history samples and combines their feature with image, Kalman-state, and time inputs in a two-layer residual head.The residual head outputs four acceleration channels for box center and size components.
  • Motion-prior diagnostics: A linear residual baseline maps filtered center position and velocity directly to acceleration to measure motion-prior predictability.Constant-acceleration fitting supplies anchor position, anchor velocity, and fitted future acceleration for correlation and ridge-linear R2 analysis.
  • Motion-prior diagnostics: The decorrelation procedure iteratively removes samples that reduce measured linear relationships between standardized prior features and fitted acceleration.Experiments retain 50% of samples, so the diagnostic weakens measured shortcuts by dropping data rather than reweighting the data-generating process.

4 Experiments

Experiments show that residual acceleration corrections improve full-box Kalman forecasting, while canonical FRED performance partly reflects position- and velocity-based motion priors. Decorrelated stress tests indicate that event-conditioned residual models retain useful signal when those measured shortcuts are weakened.

  • 4.2 Canonical Split Results: The optimized constant-velocity and constant-acceleration Kalman filters are strong full-box baselines, with constant acceleration slightly better across reported metrics.Compared with last-four linear extrapolation, filtering the full center-size state improves bounding-box ADE/FDE and mIoU stability.
  • 4.2 Canonical Split Results: Residual acceleration models improve over Kalman baselines at both 400 ms and 800 ms forecast horizons.The linear residual model already improves center accuracy using filtered motion features alone.
  • 4.2 Canonical Split Results: Adding box history and filter-state features improves residual forecasting further, while event-derived image representations produce the best results.The residual formulation combines a constant-velocity physical prior with learned acceleration corrections.
  • 4.2 Canonical Split Results: Event frames and CSTR perform similarly on the canonical split, with CSTR best at the short horizon and the representations close at the mid horizon.The main gain is associated with residual learning combined with event-conditioned local evidence rather than a decisive representation advantage.
  • 4.2 Canonical Split Results: Comparisons with previously reported FRED and RPM-modulated Kalman results require caution because evaluation details, splits, and forecast targets differ.The paper nevertheless reports the optimized full-box Kalman baseline with residuals as highly competitive against direct learned forecasting baselines.
  • 4.3 Motion-Prior Correlation: Position and velocity alone partly predict future acceleration, revealing motion-prior structure in the canonical FRED split.Figure 3 shows edge-to-center tendencies and velocity-dependent deceleration patterns; decorrelation greatly reduces the measured predictability.
  • 4.4 Decorrelated-Subset Results: Decorrelating training data can reduce performance on the canonical test split, whereas canonical-trained models often perform worse on decorrelated tests, especially at 800 ms.These results indicate train-test mismatch when canonical motion-prior correlations are weakened.
  • 4.4 Decorrelated-Subset Results: Event-conditioned models remain stronger than non-image residual models after decorrelation, although CSTR is more stable under mismatched decorrelation conditions.The remaining gap indicates useful local event evidence and box-history information beyond the measured position- and velocity-based priors.

5 Discussion

Residual forecasting improves strong full-box Kalman baselines, but canonical-split gains can combine event-conditioned evidence with dataset-specific motion priors. Decorrelated-subset results provide a cautious diagnostic view while retaining useful event-conditioned signal.

  • Model behavior: Learned acceleration residuals consistently improve over CV and CA Kalman filters over the full center-size state.The results support combining a stable physical prior with learned corrections for short-horizon UAV forecasting.
  • Motion-prior dependence: Part of future acceleration is predictable from anchor position and velocity, so canonical-split improvements may reflect both event evidence and motion-prior structure.
  • Diagnostic scope: The decorrelation procedure weakens selected linear motion-prior relationships but does not eliminate nonlinear priors, scene cues, target-scale effects, or other shortcuts.It is a diagnostic stress test rather than a new benchmark or evidence of out-of-distribution robustness.
  • Stress-test results: Event-conditioned residual models still perform well after decorrelation, whereas models relying more directly on position and velocity are more affected.This indicates that event evidence contributes useful information while part of canonical-split improvement comes from motion-prior structure.
  • Limitations: The experiments remain restricted to short- and mid-horizon box forecasting on FRED and analyze fitted center acceleration rather than full box dynamics.The deterministic model also omits forecast uncertainty, and compact event encoders with target-centered cutouts may limit extracted visual evidence.

6 Conclusion

The paper presents residual Kalman forecasting for UAV bounding boxes on FRED, extending a full center-size physical prior with learned corrections. It reports strong results over optimized full-box Kalman baselines and finds that event-conditioned residuals remain useful when measured motion shortcuts are weakened.

  • The residual Kalman approach achieves strong FRED results and improves over optimized full-box Kalman baselines under the reported protocols.
  • The method extends center-only forecasting to a full center-size Kalman state and learns acceleration-like residuals from events, box history, and filtered state features.The Kalman filter supplies a stable first-order physical prior, while residuals capture data-dependent acceleration and turning corrections.
  • The diagnostic analysis shows that part of the residual target is explainable from anchor position and velocity without event evidence.Greedy sample decorrelation reduces this predictability and stress-tests learned residual models.
  • Event-conditioned residuals remain useful when measured position- and velocity-based shortcuts are weakened, but canonical-split gains require cautious interpretation.
Loading 2609.00839v1…