Optical-Flow Wingbeat Counting in MuJoCo: A Comparison of Convolutional, Spiking, and Attention-Based Temporal Models

• A correct total or steady-frequency estimate can still miss individual events or place them at the wrong times, motivating explicit event definitions and timing-aware evaluation.Source: §1 · p. 1
• Prior work has used optical flow for bumblebee wingbeat-frequency estimation and neural methods for repetition counting, so this study positions itself as an application-specific evaluation [3].Source: §1 · p. 1
• Three independent observer–target pairs provide target-specific tests with controlled geometry and explicit sensor inputs, without cross-observer fusion or communication.Source: §1 · pp. 1–2
• The study asks whether signed optical flow supports reproducible simulated event counting, how three temporal modules compare under matched conditions, and which errors persist despite correct totals.Its stated contributions are an inspectable workflow, a paired comparison, and analysis of timing and window-boundary failures.Source: §1 · p. 2

• Prior optical-flow and repetition-counting studies motivate the task, but their published benchmarks are not reproduced or directly comparable to this study’s exact-count accuracy and unnormalized count MAE [3].Source: §2 · p. 2
• The experiment uses signed horizontal and vertical optical flow alone as neural input, distinguishing its setup from prior RGB-plus-motion approaches.Source: §2 · p. 2
• Causal convolution, surrogate-gradient spiking, self-attention, and relative-position biases are established components; here they form alternative temporal modules after a shared convolutional spatial encoder [7] [8] [9] [10].The spiking and attention systems are CNN hybrids, not fully spiking image encoders or pure vision Transformers.Source: §2 · p. 2

3.1 Simulation and image acquisition

• Three target assets were observed by forward-facing virtual cameras at 1.5 or 3.0 m, with 128 × 96 images acquired at 60 Hz while ideal inverse dynamics stabilized the simulated bodies.The setup controls visual motion but does not demonstrate aerodynamic forces sustaining flight.Source: §3.1 · pp. 2–3

3.2 Paired scenes and split

• The dataset contains 1,440 clips from 240 paired scene configurations, split 180:60 by complete scene so target and distance variants remain together.Moving frequencies ranged from 2.5–7.5 Hz; appearance, illumination, background, and wing amplitude varied, and stationary controls remained in the primary metrics.Source: §3.2 · p. 3

3.3 Reference phase and one-event-per-cycle definition

• A wingbeat is one positive-direction crossing of the primary wing’s neutral position, equivalently an increasing 2π phase boundary; left and right wings do not count as separate events.Labels use commanded frequency and amplitude only for annotation, not neural inference.Source: §3.3 · p. 3
• Evaluation uses the interval (0.5, 2.9833] s after 30 history frames; a crossing exactly at the start is excluded, and no post-clip frame can confirm a final candidate.Source: §3.3 · p. 3

• Three independent camera–target pairs are evaluated at two separately acquired distances, each using signed optical flow from its own RGB stream.Source: Figure 1 · p. 4
• The temporal convolutional, spiking, and attention modules are alternatives, not an ensemble, and use the same spatial encoder structure with independently trained weights.Source: Figure 1 · p. 4, BIRD_SIM · p. 4
• Simulator phase supplies training and reference-event evaluation information but does not enter inference; the counter records interpolated crossing times after causal confirmation.Source: Figure 1 · p. 4

4.1 Inputs and shared spatial encoder

• Signed horizontal and vertical optical-flow displacements form each frame’s input, and a shared convolutional encoder produces a 64-dimensional representation with motion-magnitude features.The input uses Farnebäck flow; segmentation and label-dependent target crops are not supplied to the network.Source: §4.1 · p. 4

4.2 Alternative temporal modules

• The temporal alternatives are a 31-frame causal TCN, two recurrent 136-neuron LIF layers, and masked causal attention over preceding frames.The LIF readout uses a second-layer spike trace, without a direct analog-input or membrane bypass; the implementation has no measured energy advantage.Source: §4.2 · p. 5

4.3 Learning objective

• The shared objective combines phase-vector, finite-difference, activity-logit, and phase-norm losses, with all components held identical across model families.The study did not establish the individual benefit of any loss component through ablation.Source: §4.3 · p. 5

4.4 Causal event counter

• The same stateful counter tracks increasing phase and confirms a candidate event only after positive hysteresis, with a minimum 0.08-second event separation.Unconfirmed candidates near clip endpoints are not counted, and missing observations are not filled by extrapolating commanded frequency.Source: §4.4 · pp. 5–6

5.1 Training budget and retained baseline

• Each model received 60 epochs and 1,440 optimizer updates, while the six existing CNN–TCN models were retained and twelve new models were frozen before test prediction.There was no validation-based model selection, early stopping, or architecture-specific hyperparameter search.Source: §5.1 · p. 6
• The test scenes had been used during optical-flow development, and each model has one training realization, making this a fixed-split comparison rather than a fresh external holdout.Near- and far-distance models were trained separately, so the comparison does not demonstrate transfer of one trained model between distances.Source: §5.1 · p. 6

5.2 Count and event metrics

• Count evaluation reports exact-count accuracy and unnormalized count MAE across all clips, while event precision, recall, and F1 use pooled timestamp matches within two camera frames.Count MAE is measured in events per clip; unmatched events can occur despite correct clip totals when event timing is displaced.Source: §5.2 · pp. 6–7, Table 2 · p. 7

5.3 Paired uncertainty and implementation checks

• Paired scene-cluster bootstrap intervals are conditional on fixed trained weights and do not quantify retraining variation or support simultaneous claims across contrasts.Synthetic checks verify selected implementation properties, not robustness to untested camera motion or real-world backgrounds.Source: §5.3 · p. 7

6.1 Exact counts and event-level performance

• At 1.5 m, CNN–TCN and CNN–Transformer counted 174/180 clips exactly and CNN–LIF 171/180; at 3.0 m, counts were 170, 171, and 166, respectively.Source: §6.1 · p. 7
• Far-distance CNN–LIF matched 1,947 events, with 11 unmatched predictions and 17 unmatched references; exact-count accuracy was 92.22%, so totals alone missed event-level errors.Table 3 reports reference and predicted event totals and one-to-one matching.Source: §6.1 · p. 7, Table 3 · p. 8

6.2 Target-specific behavior

• For flappy_v2 at 3.0 m, CNN–LIF counted 54/60 clips exactly with event F1 0.98471, while its other two far-distance target conditions exceeded 0.995.The observed difference was not uniform across target types; differing physical wingspans also mean equal distances do not equalize projected image sizes.Source: §6.2 · p. 8, Table 4 · p. 8

6.3 Paired differences

• All 95% intervals for pairwise exact-count accuracy differences included zero; the far-distance CNN–Transformer minus CNN–TCN estimate was +0.56 percentage points.The point estimates do not establish a stable ranking, and an interval spanning zero is not an equivalence test.Source: §6.3 · p. 8, Table 5 · p. 9

6.4 Stationary controls and observation availability

• No architecture produced a false event in 18 stationary test clips at either distance, and all 18 zero-flow controls produced zero counts.Three architecture–clip records had unavailable observations and remained in every primary metric.Source: §6.4 · p. 9, Table 6 · p. 9

6.5 Post-hoc boundary and timing analysis

• Most unmatched events for CNN–TCN, CNN–Transformer, and near-distance CNN–LIF fell within one camera frame of an evaluation endpoint; far-distance CNN–LIF also had seven off-endpoint unmatched events of each type.Seven far-distance CNN–LIF clips had equal predicted and reference totals despite nonzero false positives and false negatives.Source: §6.5 · p. 9

7 Discussion and limitations

• All three modest temporal models supported causal wingbeat counting in the controlled task, while exact totals and correctly timed events remained distinct outcomes.A one-event endpoint loss can make an otherwise accurate clip fail exact-count evaluation.Source: §7 · p. 9
• Endpoint-sensitive errors influenced most observed failures, and changing the boundary protocol could change the reported percentage.The proposed protocol should specify state carry across windows, delayed confirmation after reporting boundaries, and how confirmation delay is scored.Source: §7 · pp. 9–10, Table 7 · p. 10
• The comparison supports a practical development comparison, not a universal architecture ranking or optimally tuned representatives of each model family.Each architecture had one fixed hyperparameter configuration and one training realization per condition; temporal context and augmentation draws also differed.Source: §7 · p. 10
• The simulation used constant within-clip frequencies and nominally stationary cameras, without systematic tests of ego motion, occlusion, blur, dropped frames, or realistic camera noise.Near- and far-distance models were trained separately, and no target type, distance, or background family was withheld as an unseen domain.Source: §7 · p. 10
• The saved-clip evaluation measured no end-to-end camera-to-event latency, sustained onboard throughput, or hardware energy.Dense GPU operations in the spiking path do not support an energy-efficiency claim from binary spikes alone.Source: §7 · p. 10
• The study did not reproduce comparisons against established repetition-counting methods or a classical peak/phase-tracking baseline.The authors recommend a new scene-disjoint holdout, repeated training across seeds, causal baselines, and real video before broader robotic claims.Source: §7 · p. 10

8 Conclusion

• Near-distance exact-count accuracy ranged from 95.00% to 96.67%, and far-distance accuracy ranged from 92.22% to 95.00%, under the same fixed event counter.Paired intervals did not establish an exact-count architecture ranking.Source: §8 · p. 10
• The workflow used convolutional, recurrent spiking, and attention-based temporal modules to count individual events from signed optical flow in controlled MuJoCo scenes..Source: §8 · p. 10
• Event-level analysis found endpoint-sensitive errors and seven far-distance spiking-model clips with correct totals but inaccurate event matching.The conclusion motivates evaluating event timing, confirmation, and cumulative counts together when extending the task to moving cameras and real platforms.Source: §8 · p. 10

Data, code, and model availability

• The ancillary files include the experiment protocol, aggregate statistics, and per-clip count and event-matching records.The statistical records identify paired scenes so aggregate metrics and paired comparisons can be checked.Source: Data, code, and model availability · p. 11
• The complete RGB/optical-flow arrays and trained checkpoints were retained locally and are not deposited in a public repository in this version..Source: Data, code, and model availability · p. 11
• The paper package alone does not enable independent regeneration of the full image-to-prediction pipeline.Upstream model sources are cited separately, and meshes are not redistributed where redistribution permissions are not established.Source: Data, code, and model availability · p. 11

AI assistance

• AI assistance supported implementation, analysis-script development, literature organization, and English drafting..Source: AI assistance · p. 11
• Numerical tables were generated from saved per-clip evaluation records and checked against frozen experiment artifacts..Source: AI assistance · p. 11
• The schematic is an original vector drawing of the implemented information flow..Source: AI assistance · p. 11