PaperlayerView exact revision on arXiv

Source-linked AI summary

Wave Function Backpropagation with Explicit Temporal-Interval Dynamics

Byunggu Yu, Justin Kim

arXiv:2609.00503v1cs.AI

TL;DR

Standard neural networks lack a dedicated mechanism for representing elapsed time when observation intervals vary. This paper introduces WFB, which couples state and interval through a differentiable wave response and evaluates it in a controlled feed-forward trajectory predictor. WFB improves ADE over the original FFN and remains competitive with capacity-matched controls, while the evidence does not establish interval alignment as the source of the gain.

  • Problem

    Standard neural networks often treat elapsed time as an index or ordinary feature, providing no explicit response parameterization for varying observation intervals.

  • Method

    WFB represents neural responses with learnable amplitude, wavenumber, angular frequency, and phase, and applies standard gradients plus a spatial-Laplacian correction in a controlled feed-forward trajectory predictor.

  • Results

    Real-interval WFB reduces ADE by 10.4% relative to the original FFN in position-only evaluation and remains competitive with parameter-matched ReLU controls; shuffled-interval WFB attains the lowest mean ADE.

  • Takeaways & Limitations

    The results support WFB as a feasible structured feed-forward learning formulation, while not attributing the observed gain to correct interval alignment.

  • Takeaways & Limitations

    Sequence-aware propagation is outside the present proof-of-concept evaluation, and future work must study recurrent or attention-based extensions.

Abstract

from arXiv · show

Conventional neural networks learn predominantly through affine transformations followed by nonlinear activations, while elapsed time is often treated as an auxiliary feature or assumed to be uniformly sampled. This paper introduces Wave Function Backpropagation (WFB), a wave-parameterized learning formulation in which neural responses are represented by learnable amplitude, wavenumber, angular frequency, and phase. The formulation associates an observed state with its temporal interval Delta t through the phase of a differentiable spatiotemporal wave. We derive standard WFB gradients and a spatial-curvature correction based on the Laplacian of the wave response. WFB is instantiated in a deliberately feed-forward trajectory predictor to provide a controlled proof of concept; sequence learning is outside the scope of the present evaluation. With motion features, STD-WFB using real intervals reduces average displacement error (ADE) by 20.4% relative to the original FFN baseline. In a new position-only evaluation that removes temporal leakage through precomputed velocity and acceleration, real-interval WFB reduces ADE by 10.4% relative to the original FFN and remains competitive with parameter-matched ReLU controls, obtaining 2.1% lower mean ADE than the matched FFN with explicit Delta t. Shuffled-interval WFB attains the lowest mean ADE, indicating that the present evidence supports the effectiveness of the wave representation but does not attribute the gain to interval alignment. These results establish WFB as a viable structured feed-forward learning formulation and define a clear basis for subsequent architectural studies.

1 Introduction

The paper identifies a gap in how standard neural networks represent irregular elapsed time and introduces WFB, a differentiable wave-based formulation evaluated in a controlled feed-forward trajectory predictor.

  • Motivation: Standard neural networks lack a dedicated parameterization for how responses evolve across varying observation intervals.Elapsed time is commonly represented by indices, appended features, or state differences, approaches that are most adequate under uniform sampling.
  • Motivation: Irregular temporal intervals occur in asynchronous sensing, missed observations, event-based measurements, healthcare records, and physical motion.Existing methods model irregular time through decay, continuous-time latent dynamics, or controlled differential equations.
  • Proposed formulation: WFB represents neural responses with learnable amplitude, wavenumber, angular frequency, and phase, allowing elapsed time to enter through a structured phase displacement.The response remains differentiable and trainable by gradient descent.
  • Evaluation design: The paper uses trajectory prediction as a controlled proof of concept while deliberately retaining a feed-forward architecture without recurrent memory, attention, or task-specific interaction modules.This setting is intended to isolate effects of the wave representation from gains due to stronger sequence architectures.
  • Contributions: The study derives standard and Laplacian-corrected WFB updates and compares real, shuffled, and constant intervals with position-only controls and parameter-matched ReLU baselines.These experiments separate wave-representation effects from model capacity and temporal information embedded in engineered motion features.

2 Related Work

Related work addresses irregular timing, temporal dynamics, and continuous signal representation through several established mechanisms. This paper instead evaluates learnable wave parameters that directly couple spatial state and elapsed interval within a controlled feed-forward predictor.

  • Temporal modeling: Recurrent and continuous-time methods represent irregular timing through hidden-state decay, gating, latent dynamics, or differential-equation formulations.These approaches address missing observations, irregular intervals, or temporal evolution without defining the paper’s wave-parameterized learning rule.
  • Wave and periodic representations: Fourier features and sinusoidal implicit representations motivate periodic signal modeling, but WFB exposes amplitude, spatial frequency, temporal frequency, and phase as jointly learnable parameters.An observed interval directly enters the phase rather than functioning only as a fixed positional encoding or generic sine activation.
  • Trajectory prediction: Trajectory prediction research emphasizes social interaction, multimodality, attention, and motion priors in task-specific architectures.Those architectures are not the focus of this study.
  • Position of this work: The paper uses trajectory prediction as a physically interpretable testbed for assessing whether wave-parameterized activations learn from spatial states and observed temporal intervals.The evaluation deliberately uses a controlled feed-forward architecture rather than interaction-focused sequence modeling.

3 Wave-Parameterized Representation

WFB replaces conventional affine-activation components with parameterized harmonic responses whose phase couples spatial input to elapsed time. The paper’s implementation jointly learns wave-field parameters while treating separable amplitudes as an architectural choice.

  • From static activations to waves: Conventional neural components apply affine transformations followed by activation functions, whereas WFB uses parameterized harmonic responses.This substitution makes the response explicitly dependent on spatial and temporal variables.
  • Spatiotemporal coupling: The practical WFB formulation directly associates state x with elapsed interval ∆t through the wave phase.The interval represents elapsed time for the current observation, not sequence order by itself.
  • Scope boundary: Ordered temporal dependencies require an additional propagation mechanism, which the present formulation leaves to future recurrent or attention-based architectures.The current evaluation therefore addresses interval-conditioned responses rather than sequence learning.
  • Amplitude design: The evaluated implementation uses separable spatial and temporal amplitudes, allowing those contributions to be optimized separately.This factorization is an architectural choice rather than a requirement of WFB.
  • Learnable parameters: WFB learns amplitude, wavenumber, angular frequency, and phase jointly with the remaining network parameters under the task objective.Amplitude controls response magnitude; wavenumber, angular frequency, and phase control spatial sensitivity, temporal phase change, and phase offset.

4 Wave Function Backpropagation

WFB obtains parameter updates by applying the task-loss chain rule to differentiable wave responses, with temporal-frequency updates explicitly scaled by elapsed time. A spatial-Laplacian correction provides an optional curvature signal for selected temporal parameters.

  • Standard gradients: Standard WFB gradients are obtained by differentiating the wave response and applying the chain rule to the task loss.The resulting updates are defined for the wave parameters through their effects on the response.
  • Standard gradients: The ω_n gradient is explicitly scaled by ∆t, so different elapsed intervals produce different temporal-frequency updates.This is the direct gradient-level mechanism by which interval values influence learning.
  • Laplacian correction: The curvature correction adds a penalty based on the spatial Laplacian of the wave response to study whether spatial curvature stabilizes the learned wave field.The regularization strength is controlled by λ, while the correction is derived from spatial rather than temporal curvature.
  • Implementation: The implementation applies the Laplacian correction to Q_t = {A_tn, ω_n, θ_n} while holding k_n fixed in that correction branch.Parameters outside Q_t continue receiving supervised task gradients.
  • Update modes: Standard, Laplacian-only, and combined standard–Laplacian modes differ in whether selected temporal parameters receive supervised gradients, correction gradients, or both.The selected-parameter correction is implemented directly rather than interpreted as optimization of a global objective L + P.

5 Proof-of-Concept Evaluation

The evaluation uses feed-forward trajectory predictors to test WFB under controlled temporal and input conditions. Data retain irregular observation intervals, with normalized trajectory windows predicting future center coordinates.

  • Evaluation scope: The proof-of-concept asks whether WFB improves feed-forward trajectory prediction, remains competitive without temporal leakage, and responds consistently across gradient-update variants.All experiments use feed-forward predictors to isolate the WFB representation; sequence learning is outside the evaluation scope.
  • Data preparation: The dataset contains 425,090 frame-level observations from 14,703 trajectories and 453,338 windows, split by source and agent before normalization and extraction.The split prevents overlapping windows from the same agent appearing in different partitions.
  • Preprocessing: Motion features and elapsed intervals are standardized using statistics estimated only from the training partition.The observation representation includes normalized position and motion-related inputs.
  • Prediction setup: Each input window contains 8 observations, while the target contains the next 12 center coordinates.ADE, FDE, MSE, and RMSE are reported in normalized coordinate units rather than meters.

5.3 Models and Training Protocol

The models compare a conventional FFN with WFB and capacity-controlled position-only variants under a shared training and evaluation protocol. WFB experiments also vary how curvature-derived gradients modify temporal parameters.

  • Models: The original FFN flattens an 8 × 6 motion-feature matrix, whereas WFB projects observations to 128 dimensions before applying a wave response and the feed-forward prediction head.The WFB block exposes learnable amplitude, wavenumber, angular-frequency, and phase parameters.
  • Capacity control: The position-only comparison removes velocity and acceleration, and compares real, shuffled, or constant intervals with original, explicit-interval, and parameter-matched FFNs.WFB has 401,560 trainable parameters, compared with 400,753 for matched FFN and 402,452 for matched FFN with explicit ∆t.
  • Update rules: STD-WFB uses task-loss gradients for all parameters, Laplacian-WFB replaces selected temporal gradients, and STD-Laplacian-WFB adds the curvature correction.The correction weight λ scales temporal gradient corrections rather than adding a term to the reported MSE objective.
  • Training protocol: All models use MSE loss and AdamW with learning rate 10^-3, weight decay 10^-4, batch size 512, gradient clipping at 1.0, and early stopping on validation ADE.Principal comparisons use five seeds, so small differences are interpreted descriptively rather than as definitive statistical superiority.
  • Metrics: ADE, FDE, MSE, and RMSE are reported as trajectory and coordinate prediction metrics, with lower values preferred for every metric.Coordinate-wise MSE and its square root, RMSE, are also reported.
  • Decoder control: The standard comparisons use a shared ReLU-based MLP decoder, while a linear decoder is reserved for an architectural ablation.The ablation tests whether the nonlinear wave representation can reduce the need for an additional nonlinear decoding network.

5.4 Trajectory Prediction and Temporal Ablation

Temporal ablations show that WFB improves motion-feature trajectory prediction, but the shuffled-interval result prevents attributing the gain to correct interval alignment. Figure 1 reports mean ADE with five-seed variability and lower values preferred.

  • Real intervals: 20.4% lower mean ADE is obtained by STD-WFB-FFN with real intervals relative to the original FFN baseline.FDE, MSE, and RMSE improve in the same direction.
  • Constant intervals: 12.2% lower ADE is obtained with constant ∆t, indicating that the gain is not attributable only to temporal variation.The comparison does not by itself isolate the wave representation from increased model capacity.
  • Figure 1 guide: Figure 1 displays mean ADE points with standard-deviation error bars over five seeds, and lower values indicate better performance.The figure summarizes motion-feature trajectory prediction under temporal-input ablations.
  • Shuffled intervals: 21.7% lower ADE is achieved by shuffled-∆t WFB relative to FFN, slightly outperforming real ∆t while preserving the interval distribution.Shuffling breaks observation-level interval alignment, so the evidence supports WFB effectiveness without attributing the gain to correct alignment.

5.5 Position-Only Capacity-Controlled Evaluation

The position-only evaluation removes temporal leakage and controls model capacity while comparing WFB with feed-forward baselines and Laplacian variants. Real-interval WFB remains competitive, while shuffled intervals perform best and weak Laplacian corrections are preferable to larger ones.

  • Position-only comparison: 10.4% lower ADE: real-interval WFB improves on the original position-only FFN using only observed positions.This evaluation removes precomputed velocity and acceleration from the spatial inputs.
  • Capacity control: 2.1% lower mean ADE: WFB outperforms the parameter-matched FFN with explicit ∆t, although the difference is modest relative to five-seed variability.The reported mean ADEs are 0.011787 ± 0.001123 for WFB and 0.012043 ± 0.000518 for the matched explicit-∆t FFN.
  • Interval intervention: Shuffled-interval WFB achieves the lowest mean ADE, 0.011418 ± 0.000274, compared with 0.012374 ± 0.000644 for constant-interval WFB.The results therefore support WFB effectiveness but do not establish interval alignment as the source of improvement.
  • Laplacian weight: 1.9% lower ADE: the best combined standard–Laplacian setting uses λ = 10^-5 and reaches 0.011513 ± 0.000039 versus 0.011739 ± 0.000640 for standard WFB.Increasing λ to 10^-3, 10^-1, and 100 raises ADE to 0.017112, 0.039341, and 0.050845.
  • Laplacian weight: Laplacian-only WFB performs poorly across tested weights, with ADE between 0.192761 and 0.229382.Its curvature-driven temporal update lacks sufficient task-level credit assignment when used without the supervised gradient.

5.7 Interpretability Analysis

The interpretability and decoder analyses examine learned wave-parameter diversity, sensitivity to interval interventions, and the accuracy–complexity trade-off. The results suggest that wave-based nonlinear structure can support accurate prediction with a lightweight linear readout, while interval sensitivity does not establish correct ordering.

  • Learned wave parameters: Learned wave channels retain diverse parameters: amplitude is sparse and right-skewed, while k, ω, and θ span broader signed distributions.The analyzed checkpoint reports A = 0.0010 ± 0.0024, k = −1.5739 ± 0.2138, ω = −1.0196 ± 0.2445, and θ = 0.0203 ± 0.3602.
  • Learned wave parameters: Wave-parameter associations are strongest with mean speed, whereas correlations with temporal-interval summary statistics are mostly weak.Spearman correlations with mean speed are −0.322 for A, −0.283 for k, −0.297 for ω, and 0.300 for θ.
  • Interval sensitivity: 30.9% higher ADE: doubling inference-time intervals causes the largest tested degradation relative to real ∆t.Mean ADE rises from 0.011210 with real intervals to 0.014672 when intervals are scaled by 2.0.
  • Decoder ablation: 25.7% lower ADE: the linear-decoder STD-WFB achieves the lowest ADE in the decoder ablation relative to the FFN baseline.It also reduces ADE by 19.2% relative to STD-WFB with the ReLU-MLP decoder.
  • Decoder ablation: 93.4% fewer parameters: decoder removal reduces the model from 402,072 to 26,520 trainable parameters.Linear operations fall from 405,504 to approximately 30,720 MACs per sample, while the WFB layer evaluates 1,024 cosine responses.
  • Decoder ablation: The findings suggest that STD-WFB’s wave representation supplies much of the required nonlinear expressiveness, allowing a lightweight linear readout in this controlled setting.This does not establish that linear decoders are universally superior.

5.9 Discussion

WFB couples spatial state and elapsed time within a learnable wave response, yielding a structured feed-forward representation whose effectiveness is supported by accuracy, sensitivity, interpretability, and curvature-correction results.

  • Wave representation: WFB couples state and Δt within one phase function, with amplitude, k, ω, and θ controlling response strength, spatial variation, temporal variation, and phase.These jointly optimized parameters let trajectory dynamics interact directly within the wave response.
  • Accuracy and capacity control: 20.4% lower ADE was achieved by STD-WFB-FFN with real Δt than the original FFN using motion features.In position-only inputs, real-interval WFB also improved ADE by 10.4% relative to the original FFN and remained competitive with parameter-matched ReLU controls.
  • Interval sensitivity: 30.9% higher ADE followed interval doubling, while replacing, reversing, or rescaling intervals also changed predictions.This supports Δt as operational within the model rather than merely appended as a feature.
  • Decoder ablation: A lightweight linear decoder could read out the representation, but the preliminary efficiency result requires multi-seed and runtime confirmation.The decoder ablation indicates feasibility without establishing a definitive efficiency advantage.
  • Curvature correction: λ = 10^-5 produced the best STD-Laplacian-WFB-FFN setting, improving ADE over STD-WFB-FFN when curvature complemented the task gradient.The Laplacian-only correction was not sufficiently aligned with the supervised trajectory objective.

6 Conclusion and Future Work

The paper formulates WFB as a differentiable wave-based learning mechanism and evaluates it in a controlled feed-forward trajectory-prediction proof of concept. WFB outperforms the original FFN and remains competitive with matched ReLU predictors, while interval alignment is not established as the gain’s source; sequence-aware propagation remains future work.

  • Conclusion: WFB directly associates neural inputs with elapsed intervals through learnable amplitude, wavenumber, angular frequency, and phase, with standard gradients and a spatial-Laplacian regularizer.The formulation is evaluated in a deliberately controlled feed-forward setting, using trajectory prediction as validation rather than framework scope.
  • Conclusion: STD-WFB outperformed the original FFN baseline in both motion-feature and position-only evaluations.Real-interval WFB was also competitive with parameter-matched ReLU predictors, while shuffled-interval WFB achieved the lowest mean ADE.
  • Conclusion: The results support WFB as a feasible and effective structured feed-forward learning formulation, but do not establish correct interval alignment as the source of the gain.The constant-interval and matched-baseline results indicate that model capacity and the wave representation both contribute to performance.
  • Future work: Sequence-aware propagation is outside the present proof of concept and is reserved for future recurrent or attention-based WFB architectures.Future work also includes optimization stability, parameter identifiability, theoretical approximation properties, computational complexity, and additional data.
Loading 2609.00503v1…