PaperlayerView exact revision on arXiv

Source-linked AI summary

L-FNO: Lorentzian Fourier Neural Operator for Stochastic Event Dynamics

Songhee Kang, Jihoon Kang

arXiv:2608.13562v1cs.LGstat.ML

TL;DR

Sparse, clustered events challenge neural operators built for continuous regression and motivate models that capture both covariates and event history. L-FNO combines a Lorentzian spectral memory with Poisson likelihood training, achieving stronger rare-event ranking and likelihood performance across synthetic and real-world benchmarks.

  • Problem

    Existing methods largely separate functional inputs, stochastic event modeling, and rare-event prediction, leaving sparse event dynamics insufficiently integrated.

  • Method

    L-FNO combines an FNO covariate path, causal Lorentzian excitation kernels, and Poisson negative log-likelihood training to learn event intensities.

  • Results

    L-FNO achieved the best PR-AUC across all eight synthetic scenarios, improving macro-average PR-AUC from 0.347±0.085 to 0.748±0.048 and average NLL from 0.231±0.019 to 0.190±0.019.

  • Takeaways & Limitations

    Structured spectral memory and likelihood-based learning provide effective inductive biases for neural operators modeling stochastic event dynamics.

  • Takeaways & Limitations

    The positive-amplitude Lorentzian parameterization cannot represent inhibitory dynamics such as refractory behavior.

Abstract

from arXiv · show

Modern operational systems face uncertainty even in routine conditions, where rare, bursty, and self-exciting events emerge from both exogenous covariates and endogenous event dynamics. Standard neural operators are typically trained as regression-style function-to-function models rather than conditional-intensity estimators, limiting their suitability for sparse event regimes. We introduce the Lorentzian Fourier Neural Operator (L-FNO), a stochastic neural operator that combines an FNO-style covariate path, Lorentzian spectral kernels for history-dependent excitation, and a likelihood-based training objective. We evaluate L-FNO on eight synthetic point-process benchmarks and three real-world datasets covering disease outbreak prediction and semiconductor fault or defect detection. L-FNO improves event likelihood, calibration diagnostics, and rare-event detection over regression- and likelihood-based neural operator baselines. These results show that structured spectral memory and likelihood-based learning provide effective inductive biases for neural operator models of stochastic event dynamics.

1 Introduction

Rare operational events are temporally structured through clustering, context-dependent bursts, and nonlinear, delayed relationships with exogenous covariates, but conventional neural operators lack explicit stochastic predictability mechanisms. L-FNO addresses this gap by combining an FNO architecture with Lorentzian spectral kernels equivalent to exponential Hawkes excitation and by targeting predictive temporal structure in stochastic event systems.

  • Motivation: Rare events such as semiconductor defects, financial fraud, and epidemics exhibit clustering and context-dependent bursts despite appearing irregular and unpredictable.The introduction frames whether this structure can be learned from inherently sparse data as a central research question.
  • Research gap: Structural patterns can be learned from sparse data by combining potential event dependencies with exogenous variables, whereas existing neural operators require modification for stochastic prediction under uncertainty.The stated limitation is especially important when observations are sparse and outcomes are stochastic.
  • Motivation: Climate covariates and Lumpy Skin Disease outbreaks show a nonlinear, non-instantaneous mapping with long quiescent periods and hot, humid seasonal concentration.The motivating example uses outbreaks in Thailand and covariates measured across 67 provinces.
  • Related work: Temporal point processes model self-excitation, but neural extensions focused on event sequences have limited ability to incorporate rich exogenous covariates.This limitation creates a critical gap for applications such as epidemic forecasting.
  • Contribution: L-FNO extends FNO with a structured Lorentzian spectral kernel whose transfer function α_k/(β_k + iω) is the Fourier transform of the exponential kernel α_ke−β_kt.The paper presents this equivalence as a spectral formulation of Hawkes excitation and connects it to causal-kernel expressivity in Hardy space H2(C+).

2 Related Work

Prior work separately addresses neural operators for high-dimensional functional mappings, neural temporal point processes for stochastic event arrivals, and rare-event prediction. L-FNO is positioned at their intersection by mapping functional covariates and event history to stochastic event intensities.

  • Neural operators: Neural operators learn discretization-invariant mappings between function spaces, with DeepONet and FNO enabling scalable surrogate modeling for parametric PDEs.These methods suit high-dimensional, structured exogenous inputs rather than tabular covariates.
  • Temporal point processes: Temporal point processes model stochastic event arrivals in continuous time, spanning classical Hawkes processes and neural models such as RMTPP, Neural Hawkes Process, SAHP, and THP.Hawkes models use history-dependent intensity functions, whereas neural TPPs learn sequence representations.
  • Limitations of neural TPPs: Most neural TPPs remain history-driven sequence models and do not natively learn resolution-invariant mappings from high-dimensional exogenous fields to stochastic event distributions.This limitation matters when external functional covariates, rather than event history alone, trigger rare events.
  • Research gap: Existing work largely treats functional conditioning, stochastic event modeling, and rare-event prediction in isolation.Neural operators usually output continuous solution fields, neural TPPs emphasize endogenous histories, and rare-event methods often use classification, anomaly detection, or threshold exceedance.
  • Positioning of L-FNO: L-FNO learns stochastic event intensities from functional covariates and event history by combining an FNO-style exogenous covariate path with Lorentzian event-history memory.The design targets high-dimensional external conditions and continuous-time stochastic event generation under rare and extreme regimes.

3 Method

L-FNO models conditional event intensity from exogenous covariates and past events using Lorentzian spectral memory, an FNO operator stack, and Poisson NLL training. Its causal exponential memory captures Hawkes-type excitation, while likelihood-based learning addresses sparse-event objective mismatch.

  • Intensity formulation: The model estimates conditional intensity λ(t) from covariates X(t) and event history H_t, treating binary observations as unit-width Poisson counts rather than Bernoulli probabilities.For a unit-width bin, the event probability is 1 − exp[−λ(t)].
  • Architecture: Three stacked L-FNO Blocks jointly process covariates and strictly lagged event history in the spectral domain.The covariate and history sequences are transformed by discrete Fourier transforms, with a learnable complex operator applied to K low-frequency modes.
  • Causal spectral memory: The Lorentzian factor α_k/(β_k + iω) induces learnable exponential memory, producing a causal Hawkes-type excitation from past-event impulses.The parameters α_k and β_k control per-channel amplitude and decay, while strictly lagged history prevents future-event access.
  • Objective mismatch: In sparse-event regimes with p = E[dN(t)] ≪ 1, MSE gradients are dominated by non-events and can bias predictors toward low, weakly dynamic intensities.The population MSE optimum remains E[dN(t) | X(t), H_t], but sparse observations create a practical mean-collapse failure mode.
  • Likelihood training: Poisson NLL supplies asymmetric calibration pressure, with unbounded penalties for zero intensity at event times and steady downward pressure at non-event times.A Softplus output enforces λ(t) > 0, and the objective directly targets event intensity rather than a regression residual.

4 Experiments

The experiments evaluate L-FNO across eight synthetic point-process benchmarks and three real-world datasets spanning rarity, clustering, excitation, inhibition, and operational anomalies. Comparisons use FNO-based baselines and complementary likelihood, calibration, and rare-event detection metrics.

  • Synthetic benchmarks: Eight synthetic benchmarks vary structural properties including rarity, clustering, multi-scale self-excitation, nonlinear intensity, and inhibitory dynamics.Their characterization uses event rate, Fano factor, Allan factor slope, and memory coefficient M.
  • Real-world datasets: Three real-world datasets cover semiconductor fault detection, wafer-defect prediction, and province-level Lumpy Skin Disease outbreak records.The LSD data combine ERA5 climate covariates with a wind-directed spatial Hawkes history term.
  • Baselines and protocol: L-FNO is compared with FNO-MSE and FNO-NLL to isolate the contributions of likelihood-based learning and architectural design choices.Appendix F additionally compares against Neural Hawkes and Neural Hawkes with covariates (NH-X) under the same evaluation protocol.
  • Evaluation metrics: Performance is reported using NLL, Brier score, PR-AUC, and AUC, with NLL as the primary probabilistic metric and PR-AUC as the primary rare-event detection metric.Brier score provides an auxiliary binary-label calibration diagnostic, while AUC measures complementary discrimination.

5 Experimental Results

L-FNO outperforms operator baselines across synthetic and real-world stochastic-event benchmarks, with gains in likelihood, discrimination, and rare-event sensitivity. Its Lorentzian spectral memory captures nonlinear clustered dynamics, while peak-volatility overestimation remains a documented behavior.

  • Baselines: The study compares L-FNO with FNO-MSE, FNO-NLL, FNO-WMSE, Neural Hawkes, and covariate-augmented Neural Hawkes baselines.Main Section 5 results focus on FNO-MSE and FNO-NLL; full results for FNO-WMSE, NH, and NH-X appear in Appendix F.
  • Synthetic benchmarks: 0.748±0.048 macro-average PR-AUC improves over 0.347±0.085 across eight synthetic scenarios.L-FNO achieves the best PR-AUC in all eight synthetic scenarios.
  • Synthetic benchmarks: 0.190±0.019 average NLL improves over 0.231±0.019 for the best baseline on synthetic benchmarks.The best baseline is selected between FNO-MSE and FNO-NLL separately for each scenario and metric, using five random seeds.
  • Real-world benchmarks: L-FNO achieves the best mean NLL, Brier score, PR-AUC, and AUC on all three real-world datasets.Table 4 reports mean ± standard deviation over random seeds, with the best baseline selected separately for each dataset and metric.
  • Outbreak modeling: L-FNO captures nonlinear outbreak-intensity spikes with higher fidelity and greater sensitivity to clustered events, but tends to overestimate during peak volatility.The overestimation is characterized as a conservative yet robust indicator for epidemic risk signaling.
  • Real-world benchmarks: 0.739 ± 0.032 average PR-AUC improves over 0.271 ± 0.040, with PR-AUC reaching 0.885 ± 0.008 on FDC and 0.772 ± 0.012 on LSD.The strongest discrimination gains appear in FDC and LSD, and the results attribute these gains beyond the likelihood objective to Lorentzian spectral memory.

6 Conclusion and Discussion

L-FNO bridges Fourier operator learning and temporal point processes through structured spectral kernels and Poisson-likelihood training. Its design targets sparse, rare-event dynamics while exposing limitations involving inhibitory behavior and spatial coupling.

  • Core contribution: L-FNO bridges Fourier operator learning and temporal point processes using a structured spectral kernel and Poisson-likelihood training scheme.The paper links regression-style operator learning with stochastic event modeling through likelihood-based learning.
  • Core contribution: L-FNO parameterizes self-exciting impulse responses with Lorentzian spectral kernels to estimate conditional-intensity structure under sparse observations.The model can reveal predictive temporal structure where standard regression-based operators often fail.
  • Limitations and future work: The Lorentzian parameterization enforces α > 0, precluding inhibitory dynamics such as refractory behavior; signed kernels are proposed as a remedy.Past events that suppress future intensity are not represented by the current parameterization.
  • Limitations and future work: The current spatial formulation treats regions as loosely coupled time series, motivating a fully spatiotemporal operator that jointly models space and time.The proposed future direction is a more principled joint treatment of spatial and temporal structure.

A Reproducibility and Asset Documentation · B Broader Impact Details

The paper supports reproducibility through planned release of implementation, data generators, baseline scripts, and preprocessing code, while documenting restrictions on proprietary data. It frames L-FNO as decision support for high-stakes applications, requiring expert oversight, uncertainty-aware interpretation, and safeguards.

  • A Reproducibility and Asset Documentation: The implementation, synthetic data generators, baseline scripts, and public-data preprocessing code will be released in an anonymized repository during review and publicly upon acceptance.
  • A Reproducibility and Asset Documentation: The paper acknowledges public datasets from SECOM/UCI, ERA5/Copernicus, and WOAH-WAHIS.
  • A Reproducibility and Asset Documentation: The proprietary FDC dataset cannot be redistributed because of access restrictions.
  • A Reproducibility and Asset Documentation: The FDC preprocessing pipeline, feature construction, and train–test splitting are documented despite the dataset’s redistribution restrictions.
  • B Broader Impact Details: L-FNO may aid early risk detection in public health, environmental monitoring, and industrial safety settings.
  • B Broader Impact Details: False alarms may create unnecessary operational costs, while missed detections may delay intervention.
  • B Broader Impact Details: Over-reliance on automated forecasts may reduce expert oversight, so L-FNO should be used only as decision support.
  • B Broader Impact Details: High-stakes deployments require domain-expert review, uncertainty-aware interpretation, and appropriate safeguards.

C Proof of Proposition 1 … E Full Multi-Seed Benchmark Results

The appendix proves that stable exponential modes can approximate causal finite-energy kernels and details L-FNO’s dual spectral architecture. The implementation uses strictly lagged histories, positive Lorentzian parameters, stacked blocks, and a Softplus intensity head.

  • C Proof of Proposition 1: For any causal finite-energy kernel g ∈ L2([0, ∞)) and ε > 0, finite stable exponential modes approximate g within ε.The modes use complex amplitudes and positive decay rates.
  • C Proof of Proposition 1: The proof maps causal L2 kernels isometrically to H2(C+) through the one-sided Laplace transform, reducing time-domain approximation to H2 approximation.This uses the Paley–Wiener theorem and L2 boundary values on the imaginary axis.
  • C Proof of Proposition 1: Stable first-order rational responses correspond in time to causal exponential modes α_ke^−β_kt⊮(t ≥ 0), establishing the Lorentzian parameterization’s time-domain interpretation.Each mode has a left-half-plane pole and explicit causal support.
  • C Proof of Proposition 1: The expressivity result permits signed or complex amplitudes, whereas the experiments constrain Lorentzian amplitudes to be nonnegative for excitatory Hawkes modeling.Decay rates are initialized with log-spaced timescales and learned end-to-end.
  • D Implementation Details of the Lorentzian–Fourier Neural Operator: Event-history inputs contain only past event indicators, so the current event dN(t) is excluded when predicting λ(t).The history sequence is strictly lagged relative to the predicted conditional intensity.
  • D Implementation Details of the Lorentzian–Fourier Neural Operator: Each L-FNO block combines a free FNO covariate path with a Lorentzian event-history path, adds them spectrally, and transforms the result back to the time domain.The block then applies a 1 × 1 skip connection and GELU nonlinearity.
  • D Implementation Details of the Lorentzian–Fourier Neural Operator: The Lorentzian amplitudes and decay rates use Softplus parameterizations, and three blocks feed a pointwise Softplus head producing a nonnegative conditional intensity.This enforces positive spectral parameters and output intensity.
  • D.1 Code Snippet: The code snippet implements the dual spectral update with real FFT inversion, Softplus parameters, a skip connection, and GELU activation.It accepts latent covariates x and strictly lagged history h.

E.1 Synthetic Benchmarks … F.2 Shared Training Configuration

L-FNO outperforms neural-operator baselines across synthetic and real-world benchmarks, especially in event ranking and rare-event discrimination, while calibration remains weaker in selected synthetic regimes. The comparisons use likelihood-based and stress-test baselines under shared FNO training settings.

  • E.1 Synthetic Benchmarks: L-FNO achieves the highest PR-AUC in all eight synthetic scenarios, raising macro-average PR-AUC from 0.347 ± 0.085 for FNO-NLL to 0.748 ± 0.048.It also obtains the best macro-average NLL, Brier score, and AUC, with especially large gains in B1-Rare, B4-MultiScale, B6-Inhibitory, and B8-LongMemory.
  • E.1 Synthetic Benchmarks: FNO-NLL has lower NLL in B2-Cascade and B7-ZeroInflated, and lower Brier scores in B2-Cascade, B7-ZeroInflated, and B8-LongMemory.These exceptions indicate that calibration is more sensitive in cascade, structurally silent, and long-memory regimes.
  • E.2 Real-World Benchmarks: L-FNO achieves the best mean NLL, Brier score, PR-AUC, and AUC on all three real-world datasets.Across datasets, average NLL improves from 0.186 ± 0.012 for FNO-NLL to 0.114 ± 0.007, while average PR-AUC improves from 0.271 ± 0.040 to 0.739 ± 0.032.
  • E.2 Real-World Benchmarks: L-FNO reaches PR-AUC values of 0.885±0.008 on FDC and 0.772 ± 0.012 on LSD.These are identified as the strongest discrimination gains across the real-world datasets.
  • F Additional Baselines: Details and Results: FNO-WMSE is an event-upweighted MSE stress test that assigns zero loss weight to non-event timesteps.Its purpose is diagnostic: emphasizing rare positives can improve sensitivity while causing severe likelihood and probability-calibration degradation.
  • F.1 Model Descriptions: FNO-NLL is the main calibrated operator baseline because it shares L-FNO’s Poisson NLL objective but removes structured Lorentzian memory.The supplied passage contrasts this calibrated baseline with FNO-WMSE’s event-only reweighting.
  • F.1 Model Descriptions: Neural Hawkes models conditional intensity from event history, while Neural Hawkes with Covariates concatenates X(t) to the LSTM input at each timestep.NH lacks exogenous covariates, whereas NH-X’s sequential architecture limits covariate integration across arbitrary temporal scales compared with L-FNO’s parallel FNO stack.
  • F.2 Shared Training Configuration: FNO models use AdamW with weight decay 1 × 10−4, initial learning rate 3 × 10−4, cosine annealing over 300 epochs, and a backbone width of 32.Input windows default to length 96, except D3-LSD with L = 64; stride is 8 by default and 4 for LSD, with 12 covariate-path modes and 4 Lorentzian-path modes retained.

F.3 Full Synthetic Results — Appendix Models … J Full Benchmark Visualizations

Appendix results show that FNO-WMSE is near-random and that NH and NH-X trail L-FNO on event detection, while the benchmark suite uses causal, stationary synthetic dynamics and evaluates semiconductor and outbreak datasets under specified preprocessing protocols. Experiments were run in pure PyTorch on a single Apple M-series workstation without cloud or multi-GPU training.

  • F.3 Full Synthetic Results — Appendix Models: FNO-WMSE achieves NLL ≈0.90–0.95 across all benchmarks, indicating near-random performance, while NH and NH-X underperform L-FNO on PR-AUC.The result is attributed to recurrent-processing limitations when rich covariates are available and event detection is prioritized over average log-likelihood.
  • G.1 Scenario Parameters and Stability Conditions: Each synthetic scenario uses a discrete-time closed-loop Hawkes simulation with dN_t ∼ Bernoulli(p_t), where p_t = clip(λ_t, ε, 0.99).The history state updates after current-event sampling, preserving causal intensity generation.
  • G.1 Scenario Parameters and Stability Conditions: For α > 0, stationarity is ensured by the condition |η|_disc = α/(1 −e^−β) < 1; B6-Inhibitory uses α < 0 and B7-ZeroInflated uses Markov switching.The intensity at time t depends only on events through time t −1.
  • H.1 D1-FDC: Semiconductor Fault Detection: D1-FDC contains up to 85 semiconductor process variables per timestep, derives events from excursions above the 90th quantile, and uses a chronological 80/20 split.Missing timestamps are linearly interpolated.
  • H.2 D2-SECOM: Semiconductor Wafer Defect: D2-SECOM provides 590 sensor measurements across 1,567 production runs, with heavily missing sensors discarded, remaining NaNs median-imputed, and binary defect labels supplied directly.Sensors with more than 50% missing values are discarded.
  • H.3 D3-LSD: Thailand LSD Outbreak 2021: D3-LSD comprises province-level outbreak records for all 67 Thai provinces over 365 days in 2021, aligned with ERA5 climate covariates and a wind-directed spatial Hawkes kernel.The spatial history uses only past outbreak events and does not leak the current label.
  • I Computational Resources: All experiments ran on a single Apple M-series workstation without cloud computing or multi-GPU training, using pure PyTorch without custom CUDA kernels.Approximate runtimes per model per scenario at 300 epochs are summarized in Table 10.

J.1 Synthetic Benchmark Visualizations · J.2 Real-world Data Visualizations

The visualizations show that L-FNO captures complex synthetic temporal dependencies and sharp intensity transitions while outperforming standard baselines. On real-world scenarios, it better detects clustered events and maintains calibrated quiescent-period intensities, whereas purely history-dependent modeling struggles when exogenous drivers dominate.

  • J.1 Synthetic Benchmark Visualizations: L-FNO consistently captures complex temporal dependencies across eight synthetic point-process dynamics.Figure 4 visualizes predicted conditional intensities λ(t) alongside observed event occurrences.
  • J.1 Synthetic Benchmark Visualizations: L-FNO captures sharp transitions in synthetic benchmarks including Burst and Cascade.The visualizations compare intensity estimates across the synthetic benchmarks.
  • J.1 Synthetic Benchmark Visualizations: L-FNO outperforms FNO-NLL and Neural Hawkes on the synthetic intensity visualizations.Figure 4 presents comparative intensity estimation across the benchmarks.
  • J.2 Real-world Data Visualizations: Across three real-world scenarios, L-FNO shows higher sensitivity to clustered event occurrences.Figure 5 compares L-FNO, FNO-NLL, and Neural Hawkes for real-world intensity estimation.
  • J.2 Real-world Data Visualizations: L-FNO maintains a calibrated baseline during quiescent periods in the real-world scenarios.The real-world visualizations show intensity estimates during both clustered and inactive periods.
  • J.2 Real-world Data Visualizations: In the LSD case, Neural Hawkes exhibits poor calibration and fails to adapt to climate-driven intensity changes.This illustrates the limitation of purely history-dependent models when exogenous drivers dominate event dynamics.
Loading 2608.13562v1…