PaperlayerView exact revision on arXiv

Source-linked AI summary

Event-Only Wingbeat Counting under Camera Motion: A Controlled MuJoCo Benchmark

Zhang Nengbo

arXiv:2609.37465v1cs.CV

TL;DR

Counting completed wingbeats requires identifying individual cycles, especially when rates change or pauses occur, while camera motion also contributes events. This controlled simulation benchmark separates motion training from event-only translation compensation and compares a fixed CNN with adapted baselines. Motion training improved the CNN in the tested protocol, but compensation worsened count MAE and Fourier was best in near moderate-motion MAE.

  • Problem

    Completed-cycle counting requires identifying individual wingbeats because changing rates and pauses make dominant-frequency estimates insufficient, while camera motion also generates events.

    Source: §1 · p. 1

  • Method

    A controlled simulation study holds a temporal CNN and training budget fixed while crossing motion augmentation with causal event-only background translation compensation and comparing adapted baselines.

    Source: §1 · p. 1

  • Results

    In the paired synthetic protocol, motion examples improved complete-wingbeat counting for the fixed CNN, tested compensation worsened count MAE, and Fourier was best in near moderate-motion MAE.

    Source: §7 · p. 8, §6 · p. 8

  • Takeaways & Limitations

    The evidence supports motion-aware training only in this paired synthetic protocol and motivates reporting cycle-time matching, count MAE, and exact totals.

    Source: §6 · p. 8, §7 · p. 8

    The benchmark uses an ideal deterministic sensor and does not establish real-sensor performance.

    Source: §3.3 · p. 3, §7 · p. 8

Abstract

from arXiv · show

Counting completed wingbeats requires identifying individual cycles, including during frequency changes and pauses; estimating a dominant frequency alone is insufficient. Camera motion further mixes target and background brightness changes in event observations. We present a controlled MuJoCo benchmark that separates motion training from event-only image translation compensation. The acquisition contains 324 streams from 24 independent scenes, three flapping geometries, two distances (1.5 and 3.0 m), and static, moderate-motion and stronger-motion views. Fifteen scenes are used for fitting, three for validation and six for held-out testing. A fixed causal temporal convolutional network is evaluated in a matched 2 x 2 ablation with three initialization seeds and compared with ridge, Fourier, autocorrelation and an adapted EEPPR baseline. Under moderate motion, paired motion training reduces count mean absolute error from 31.130 to 3.185 cycles at 1.5 m and from 42.019 to 5.444 at 3.0 m. Adding the tested compensation increases these errors to 4.630 and 10.185, respectively. A Fourier baseline achieves 0.944 cycles at 1.5 m under moderate motion, showing that the neural model is not uniformly best. We report exact-count accuracy and temporally matched cycle F1 alongside count error. These findings support motion-aware training in this small synthetic benchmark, while exposing limits of simple event-background stabilization. They do not establish real-sensor performance, aerodynamic flight, or generalization to unseen vehicle types.

1 Introduction

The paper frames wingbeat counting as identifying completed cycles, not estimating a dominant rate, and asks how camera motion affects event-based counting. It addresses these questions in a controlled simulation study that separates motion training from event-only compensation.

1 Introduction

  • A counter must identify completed cycles because changing rates and pauses make a single frequency-times-duration estimate unreliable.It must retain its state as observations arrive.Source: §1 · p. 1
  • Camera motion adds events alongside target motion, motivating separate tests of motion exposure during training and event-only background compensation.The effects can be confounded when training data, architecture, and evaluation scenes change together.Source: §1 · p. 1
  • A controlled simulation holds the temporal CNN and training budget fixed while crossing motion augmentation with causal, event-only background translation compensation.Camera pose and inertial measurements remain outside inference, and actual simulated wing-joint motion supplies cycle labels.Source: §1 · p. 1
  • The benchmark contributes scene-separated paired acquisition, complete-cycle labels, a matched factorial comparison, and classical and event-specific periodic-estimator baselines.The comparison includes startup penalties, unfavorable results, and validation-selection failures.Source: §1 · p. 1
  • With six test scenes, the intended contribution is an auditable controlled benchmark and conditional findings, not evidence of field deployment.Source: §1 · p. 2

2 Related work and relation to the earlier study

Prior work motivates periodic-event and repetition-counting approaches, but frequency estimation and completed-cycle counting are distinct outputs. This study uses an event-only temporal CNN and adapted periodic baselines in a new simulated acquisition rather than claiming a comprehensive method ranking.

Event-based periodic sensing

  • Frequency Cam estimates pixelwise periodicity [1], while EEPPR detects repeated spatiotemporal event patterns [2]; neither output is itself a completed-cycle count.The EEPPR comparison uses a new causal phase-and-count adapter and is not a reproduction of its original frequency benchmark; Frequency Cam was not implemented.Source: §2 · p. 2

Repetition counting and temporal models

  • RepNet uses temporal self-similarity for video repetition counting [3], and temporal convolutions provide a general sequence-modeling approach [4].The study tests augmentation and compensation with a small causal CNN, not a new architecture or a ranking against RepNet or video transformers.Source: §2 · p. 2

Simulation and previous wingbeat work

  • The benchmark uses new MuJoCo scene seeds, high-rate simulated events, long streams, complete-cycle labels, and an event-only motion factorial experiment.Unlike the earlier RGB optical-flow study [8], its results are not pooled with those data or directly comparable numerically; it reports no new SNN–CNN–transformer experiment.Source: §2 · p. 2

3 Benchmark and counting protocol

The benchmark pairs simulated vehicle geometries and camera views with controlled motion, ideal contrast events, and ground-truth completed-cycle labels. Its protocol keeps inference event-only and evaluates counts with multiple metrics and scene-level paired comparisons.

3.1 Paired geometry and acquisition

  • Three distinct flapping geometries are observed at 1.5 or 3.0 m with fixed camera optics and resolution.The acquisition has 24 scene seeds split by scene into 15 fitting, three validation, and six test scenes, totaling 324 streams.Source: §3.1 · p. 2
  • Streams alternate between 12 and 16 seconds and include constant-rate, chirp, and stop–restart profiles over balanced plain and checker-textured backgrounds.Each test profile occurs once per background, with the two backgrounds sharing plane geometry.Source: §3.1 · p. 2

3.2 Controlled camera motion

  • Camera motion is a controlled small angular jitter; the stronger condition triples both moderate-motion amplitudes and is held out from fitting and selection.Wing trajectories are identical across views, and camera commands and poses are excluded from tested estimators.Source: §3.2 · p. 3

3.3 Ideal contrast events and the input boundary

  • Rendered RGB is converted to signed contrast events accumulated in causal 60 Hz bins, but inference excludes RGB, pose, IMU, and simulator state.The deterministic sensor omits threshold mismatch, shot noise, refractory effects, finite analog bandwidth, and readout saturation.Source: §3.3 · p. 3

3.4 Completed-cycle ground truth

  • A ground-truth wingbeat ends at a qualifying positive-going joint crossing after both amplitude thresholds are visited within 0.75 seconds.All methods use a shared causal decoder that preserves counting state, distinguishing estimated crossing time from when the count becomes available.Source: §3.4 · p. 3

3.5 Metrics and independent experimental units

  • The primary interval is (0.5, Ti], and the benchmark reports count MAE, exact-count accuracy, and one-to-one cycle-endpoint F1 matching within 33.33 ms.MAE is in cycles per stream, not hertz; the primary metric retains startup failures, invalid estimates, and pauses.Source: §3.5 · p. 3
  • The paired 2 × 2 ablation varies motion training and event-background compensation while fixing the CNN, initialization pairing, and update budget.Differences are bootstrapped over six independent test scenes, with descriptive small-sample intervals and no multiple-testing correction.Source: Figure 1 · p. 4, §3.5 · p. 4

4 Methods

The methods use a fixed causal CNN in a matched motion-training and compensation ablation, with compensation estimated strictly from events. Classical and event-specific baselines use calibrated causal procedures that translate periodic-rate estimates into counts.

4.1 Fixed causal temporal CNN

  • The causal CNN maps 130 standardized event features through a 64-wide projection and four residual temporal blocks to displacement and auxiliary velocity outputs.It has 74,562 trainable parameters, uses the current bin and 30 preceding bins, and is trained separately for each target and distance.Source: §4.1 · p. 4

4.2 Matched motion-training and compensation ablation

  • The matched ablation crosses static-only versus paired moderate-motion training with raw versus compensated events, holding architecture, update budgets, windows, and initialization pairing fixed.The full pipeline selects checkpoints on validation error, then count MAE, then earlier epoch.Source: §4.2 · p. 4

4.3 Strictly event-only translation compensation

  • Compensation estimates background translation from event-derived activity surfaces, tracks candidate displacements, rejects unreliable updates, and warps both polarity images.It is a small-angle translation approximation, not full pose estimation, and interpolation can alter event-count mass or move events out of frame.Source: §4.3 · p. 5

4.4 Baselines and causal period-to-count adaptation

  • Ridge uses lagged event features, while Fourier and autocorrelation estimate causal frequency from a supervised displacement proxy and share a phase-based count adapter.All baselines use raw events, the same scene boundary, and calibration streams that include static and moderate motion.Source: §4.4 · p. 5
  • The adapted EEPPR baseline uses one-second event windows, fixed frontal patches, and validation-selected template settings before applying the shared causal phase/count adapter.Its first full window arrives after one second, and the comparison is not against the full original EEPPR software stack.Source: §4.4 · p. 5

5 Results

In this controlled benchmark, paired motion training lowers count error under camera motion, while the tested compensation increases it and a Fourier baseline wins one near, moderate-motion condition. These findings are qualified by validation-selection failures, descriptive background subgroups, and a six-scene held-out test set.

5.1 Motion training improves the matched CNN pipeline

  • Under moderate motion, paired motion training lowered MAE from 31.130 to 3.185 cycles at 1.5 m and from 42.019 to 5.444 at 3.0 m.Under stronger held-out jitter, MAE decreased from 37.593 to 6.889 and from 46.741 to 10.167 cycles, respectively; scene-paired estimates were negative in all four conditions but retained large uncertainty.Source: §5.1 · p. 5, Table 1 · p. 6, Table 2 · p. 6
  • Two static-trained far/BIRD_SIM configurations selected epoch-zero weights under the validation rule, and those checkpoints were retained without post-test replacement.The reported benefits therefore concern the specified fitting-and-selection pipeline.Source: §5.1 · pp. 5–6

5.2 The tested compensation does not improve count MAE

  • With motion training enabled, compensation increased moderate-motion MAE from 3.185 to 4.630 cycles at 1.5 m and from 5.444 to 10.185 at 3.0 m.Under stronger jitter it increased MAE from 6.889 to 10.815 and from 10.167 to 13.667; this implementation provided no added MAE benefit in the tested moving-camera settings.Source: §5.2 · p. 6
  • At 3.0 m under stronger motion, A1C1 had slightly higher exact-count accuracy and event F1 than A1C0 despite worse MAE.Exact total-count equality can coexist with incorrect individual cycle times, motivating reporting all three metrics.Source: §5.2 · p. 6

5.3 A classical baseline wins one important condition

  • Fourier achieved 0.944 cycles MAE versus A1C0’s 3.185 at 1.5 m under moderate motion, while A1C0 had higher exact-count accuracy (59.26% versus 33.33%).Fourier was less accurate in MAE than A1C0 at far distance and under stronger motion; the adapted EEPPR pipeline had larger count errors in every aggregate setting, but its adapter stages also contribute.Source: §5.3 · p. 6
  • After 2 s, near moderate-motion FFT MAE was 0.889 cycles versus A1C0 MAE of 3.148, with no hyperparameter changed for this analysis.The secondary interval used continuous decoder state and unchanged selected models; the far moderate-motion EEPPR-adapted MAE decreased from 33.556 to 28.500 and remained high.Source: §5.3 · p. 7, Table 4 · p. 7

5.4 Background-dependent failures and compensation availability

  • Accepted compensation updates were absent in static and plain-background views, while high acceptance on textured views measured availability, not pose accuracy, and did not guarantee better counting.Each background subgroup contained only three independent test scenes.Source: §5.4 · p. 7, Table 5 · p. 7

5.5 Integrity checks and preserved failures

  • All 1,728 held-out predictions passed separately implemented checks, but the work was not externally preregistered and its source-hash record followed the start of recording.The checks assess implementation consistency, not third-party human validation or sensor-hardware match.Source: §5.5 · p. 8

6 Discussion

In this paired synthetic protocol, motion examples improved complete-wingbeat counting for a fixed CNN, while the tested translation compensation performed adversely. The discussion retains classical periodic estimators as serious baselines and identifies limits on external validity and specific next steps.

6 Discussion

  • Motion examples improved complete-wingbeat counting for a fixed CNN and training budget, but the proposed explanations for compensation's adverse result remain untested.Separating pose error, motion-model limits and interpolation effects would require a pose-error audit, a richer motion model and interpolation controls.Source: §6 · p. 8
  • In a constrained, low-frequency task, the Fourier estimator can outperform a neural model on aggregate count error, supporting its retention as a serious baseline.Cycle-time matching, count MAE and exact totals complement aggregate error by showing when errors occur and whether totals are exact.Source: §6 · p. 8
  • External validity is limited by six independent test scenes, target- and distance-specific models, an idealized sensor, narrow camera motion and prescribed trajectories without validated aerodynamic control.Compute-matched exhaustive baseline searches, inference energy and hardware latency were also not measured.Source: §6 · p. 8
  • The next evidential steps are more diverse held-out scenes, calibrated or measured event streams, pose-ground-truth analysis and evaluation against physical wing-joint or high-speed-video references.Broader architecture comparisons would require retraining SNN and transformer models under the same frozen protocol.Source: §6 · p. 8

7 Conclusion

The conclusion presents a controlled MuJoCo benchmark separating completed-cycle counting from periodic-rate estimation and motion training from event-only translation compensation. Motion training improved a fixed CNN in the tested motion conditions, while compensation worsened count MAE and Fourier led on near moderate-motion MAE; the evidence remains bounded to a small ideal-sensor simulation.

7 Conclusion

  • Paired motion training improved a fixed CNN under moderate and stronger angular jitter, while the tested compensation worsened count MAE.A Fourier baseline was best in near moderate-motion MAE.Source: §7 · p. 8
  • The benchmark distinguishes completed-cycle counting from periodic-rate estimation and isolates motion training from strictly event-only translation compensation.Source: §7 · p. 8
  • The findings support careful metric design, matched ablations and transparent baseline adaptation, but do not establish real-flight performance.The evidence remains limited to a small ideal-sensor simulation.Source: §7 · p. 8

Data, code and assistance statement

The experiment retains extensive local artifacts, but this version has no deposited public raw-data or code repository. The statement also records asset-rights boundaries and the role of AI assistance.

Data, code and assistance statement

  • Acquisition manifests, source hashes, trained weights, validation selections, per-stream predictions and audit outputs are retained with the experiment.The manuscript source contains the numerical tables and figure needed to reproduce this document.Source: Data, code and assistance statement · p. 9
  • No public raw-data or code repository has been deposited for this version, and no public accession is claimed.Source: Data, code and assistance statement · p. 9
  • Upstream geometry sources retain their owners' redistribution rights, and the adapted EEPPR core retains its GPL-3.0 license in the local experiment package.The manuscript does not assign open-source licenses to pinned RLFlapping and BIRD_SIM assets or redistribute their meshes in its source archive.Source: Data, code and assistance statement · p. 9
  • AI assistance supported implementation, analysis checks, figure layout and English drafting; reported numerical results come from executed simulations and frozen evaluation artifacts.The schematic is deterministic vector artwork, not an experimental observation.Source: Data, code and assistance statement · p. 9

A Geometry provenance and adaptations

The benchmark combines an adapted MuJoCo Menagerie observer with pinned target geometries from three named sources. It preserves selected geometry while changing scene components and adding free-root and observation-rig tracking features.

A Geometry provenance and adaptations

B Nonstationary schedules and sampling

The benchmark uses changing-rate chirps and stop–restart schedules to test cycle counting under nonstationary wing motion. Labels come from tracked motion rather than the analytic phase.

B Nonstationary schedules and sampling

  • The chirp rises linearly from 3 to 7 Hz during the first half of each recording, then falls linearly back to 3 Hz.Its phase is the analytic integral of frequency, not f(t)t.Source: B Nonstationary schedules and sampling · p. 10
  • Stop–restart motion has three active intervals at 3, 5, and 7 Hz, separated by rests during which phase freezes.A quintic smoothstep ramps amplitude over 0.4 s at stop/restart boundaries; the final active segment continues beyond the recording endpoint.Source: B Nonstationary schedules and sampling · p. 10
  • Recordings last 12 or 16 seconds, use one of three profiles, and receive paired geometry and excitation across distances and views.Each recording follows a 0.8 s tracking warm-up; labels derive from actual tracked motion, not the analytic phase.Source: B Nonstationary schedules and sampling · p. 10

C Compensation and EEPPR implementation details

The implementation specifies gates for event-based background translation compensation and engineering choices for the adapted EEPPR baseline. It also reports checkpoint selections and preserves artifact provenance.

C Compensation and EEPPR implementation details

  • Compensation retains the prior offset when tracking gates fail, with per-bin shifts capped at 2.5 pixels and cumulative offsets clipped to ±8 pixels per axis.Tracking requires at least 20 active background pixels; at least eight tracks and a 60% inlier fraction must survive.Source: C Compensation and EEPPR… · p. 10
  • The adapted EEPPR baseline collapses duplicate event coordinates to Boolean occupancy and uses a fixed ROI, choices that restrict conclusions about the original method.Its upstream correlation and peak/template-selection functions remain unmodified.Source: C Compensation and EEPPR… · p. 10

D All checkpoint selections

  • Table 6 reports selected epochs for all 72 CNN runs in seed order 0, 1, and 2; epoch zero is an initialization checkpoint, yet every run executed the same 480 training updates.Source: Table 6 · p. 10, D All checkpoint selections · p. 10
  • The review package preserves code, weights, selections, predictions, and audits, while excluding raw acquisitions; tables use frozen results without retraining during writing.Source, input, and model hashes and the test chronology are preserved separately.Source: D All checkpoint selections · p. 10
Loading 2609.37465v1…