PaperlayerView exact revision on arXiv

Source-linked AI summary

Solving Inverse Stochastic Problems from Discrete Particle Observations Using the Fokker-Planck Equation and Physics-informed Neural Networks

Xiaoli Chen, Liu Yang, Jinqiao Duan, George Em Karniadakis

arXiv:2008.10653v1physics.comp-phcs.LGstat.ML

TL;DR

Inferring stochastic-system coefficients and PDF dynamics is challenging when only sparse particle observations are available and the initial PDF is unknown. The paper uses a PINN and a KL-based loss to connect those observations with a parametrized Fokker-Planck equation. For type I, even one snapshot with 100 samples yielded reasonable accuracy, while type II noise intensities were harder to learn.

  • Problem

    The paper addresses how to infer stochastic-system coefficients and PDF dynamics when only a few discrete particle observations are available and the initial PDF is unknown.

    Source: §1 · p. 3

  • Method

    A PINN uses the variational form of KL divergence to connect particle observations with a parametrized FP equation and infer the PDF over time.

    Source: §1 · p. 3, §4 · pp. 18–19

  • Results

    For type I, one snapshot and 100 samples yielded reasonable accuracy; for type II, noise intensities were harder to learn and required more snapshots and samples for accurate results.

    Source: §4 · p. 19

  • Takeaways & Limitations

    Including particle observations directly in the PINN loss was more accurate in most cases, especially with small data, than first estimating the PDF and using it as data.

    Source: §4 · p. 19

    For α-stable Lévy noise, the formulation assumes independence across dimensions.

    Source: §2.1 · p. 4

Abstract

from arXiv · show

The Fokker-Planck (FP) equation governing the evolution of the probability density function (PDF) is applicable to many disciplines but it requires specification of the coefficients for each case, which can be functions of space-time and not just constants, hence requiring the development of a data-driven modeling approach. When the data available is directly on the PDF, then there exist methods for inverse problems that can be employed to infer the coefficients and thus determine the FP equation and subsequently obtain its solution. Herein, we address a more realistic scenario, where only sparse data are given on the particles' positions at a few time instants, which are not sufficient to accurately construct directly the PDF even at those times from existing methods, e.g., kernel estimation algorithms. To this end, we develop a general framework based on physics-informed neural networks (PINNs) that introduces a new loss function using the Kullback-Leibler divergence to connect the stochastic samples with the FP equation, to simultaneously learn the equation and infer the multi-dimensional PDF at all times. In particular, we consider two types of inverse problems, type I where the FP equation is known but the initial PDF is unknown, and type II in which, in addition to unknown initial PDF, the drift and diffusion terms are also unknown. In both cases, we investigate problems with either Brownian or Levy noise or a combination of both. We demonstrate the new PINN framework in detail in the one-dimensional case (1D) but we also provide results for up to 5D demonstrating that we can infer both the FP equation and} dynamics simultaneously at all times with high accuracy using only very few discrete observations of the particles.

1 Introduction

Stochastic systems span many disciplines, but their case-dependent coefficients may vary across space and time, motivating data-driven inference. This paper uses PINNs to combine sparse particle observations with the Fokker-Planck equation to infer its coefficients and PDF dynamics.

  • Stochastic systems occur across disciplines, and their noise types and coefficients vary by case, motivating data-driven inference of system coefficients and dynamics.Source: §1 · p. 2
  • Earlier methods inferred SDEs from observations, while this work follows [17] but uses the FP equation, which governs PDF dynamics.Source: §1 · p. 2
  • Forward FP solvers generally assume a known initial condition, while inverse PDE methods require observations of the solution at space-time points.Source: §1 · pp. 2–3
  • A PINN uses a KL-based loss to combine particle observations with a parametrized FP equation, pairing particle-data and FP-residual losses while avoiding high-dimensional grid generation.Source: §1 · p. 3, Figure 1 · p. 3

2 Methodology

The method treats sparse particle snapshots as evidence for learning an evolving PDF and, when needed, unknown Fokker–Planck coefficients. It combines a variational KL density estimator with a neural network constrained by the FP equation and the observations.

2.1 Problem setup

  • The setup uses sparse particle snapshots at discrete times to infer either the PDF with a known FP operator or unknown operator terms such as drift or diffusion.Source: §2.1 · pp. 4–5

2.2 PDF estimation using neural networks

  • A variational KL formulation estimates density from finite i.i.d. samples by optimizing positive neural functions, with a uniform reference density on a hypercube containing the sample support.Source: §2.2 · pp. 5–6

2.3 Solving the FP equation using discrete particle observations

  • A space-time neural network approximates p(x, t), remains positive, and is trained to satisfy the FP equation while matching observed samples at snapshot times.Source: §2.3 · p. 6
  • Snapshot data loss is inspired by the variational KL estimator and combines with PDE loss using weight τ; minibatches and quadrature or Monte Carlo approximate terms.Source: §2.3 · pp. 7–8
  • When particle observations are available, kernel density estimates can instead supply a density-based data loss.Source: §2.3 · p. 8

3 Computational Results

The 1D Brownian results show that PINNs can recover PDF dynamics from sparse particle snapshots, and direct particle observations outperform using kernel-estimated PDFs as input. The section also examines joint drift, diffusion, and PDF inference under combined noise and extends inverse problems to two dimensions.

3.1.1 Problem I: Brownian noise

  • With one snapshot, physics-informed learning captured PDF dynamics at all times, including both peaks with only 100 samples.The observation data were available at t = 0.3.Source: §3.1.1 · p. 9
  • Direct particle observations produced better PINN results than kernel-estimated PDF inputs and outperformed kernel density estimates at individual times.The comparison uses the L2 error across the tested methods.Source: §3.1.1 · p. 11

3.1.2 Problem I: L´evy noise

  • The paper also reports a systematic study of Problem I with Lévy noise in Appendix A.Source: §3.1.2 · p. 11

3.1.3 Problem II: Brownian and L´evy noise

  • For combined Brownian and Lévy noise, polynomial drift fits represented drift better, while neural-network drift models estimated diffusion coefficients more accurately.Accuracy was higher when Brownian and Lévy noise were considered separately than together.Source: §3.1.3 · p. 11

3.2.1 2D Brownian noise

  • In 2D Brownian cases, learning drift parameters gave better results with as few as five snapshots; neural-network drift approximations had parameter-estimation errors above 10%.Explicitly fitting the drift also gave the best drift-term results.Source: §3.2.1 · p. 13
  • The 2D Brownian experiments used seven snapshots and compared parameter learning with two neural-network drift approximations and a single network for the potential.The PDF-error figures report results at t = 1.Source: §3.2.1 · p. 13, Figure 5 · p. 14

3.2.2 2D L´evy noise

  • For 2D Lévy noise, the study compares parameterized drift, two drift networks, and a network approximating the potential using seven snapshots.The reported figures show PDF errors and drift-term phase diagrams; the supplied passages do not state their comparative outcomes.Source: §3.2.2 · p. 16, Figure 7 · p. 16, Figure 8 · p. 17

3.3 High-dimensional problems

The high-dimensional experiments extend inverse-problem inference to a 5D Brownian case, using seven discrete snapshots and two polynomial drift forms.

3.3 High-dimensional problems

  • The paper places the 5D Brownian results alongside similar 3D and 4D cases in Appendix F.Source: §3.3 · p. 17
  • The 5D Brownian experiments used seven snapshots, 100,000 samples per time, and Monte Carlo integration with 200,000 samples.The FP residual was evaluated at 10,000 points.Source: §3.3 · p. 18
  • The experiments compared cubic drift terms without a constant term against cubic drift terms with a constant and quadratic term, learning drift and diffusion parameters in each case.The tested drift forms were applied across five dimensions.Source: §3.3 · p. 18

4 Summary

The framework uses PINNs and a variational Kullback–Leibler loss to infer PDFs from sparse particle snapshots while learning unknown FP coefficients. Tests cover inverse problems in 1D–5D, with accuracy depending on the problem and drift parameterization.

4 Summary

  • For type II problems, the framework jointly infers the PDF and unknown drift and noise intensity, representing drift with polynomials or neural networks.For a 2D case, two neural networks approximate the drift terms.Source: §4 · p. 19
  • With one snapshot and 100 samples, type I problems yielded reasonable initial distributions and PDF dynamics; type II noise intensities required more snapshots and samples for accurate inference.The paper reports using relatively few snapshots, such as 5 or 7, for type II problems.Source: §4 · p. 19
  • Polynomial drift parameterizations were more accurate for drift inference but required prior knowledge of the drift form; accurate results were demonstrated in 5D, while higher dimensions need further active-learning work.A companion study used GANs and the particle SDE to obtain accurate results in 20D with relatively few samples [17].Source: §4 · p. 19

Appendices

The appendices extend tests to 1D and 2D for both inverse problems under Brownian or Lévy noise, and to 3D and 4D for type II with Brownian noise.

Appendices

  • Appendix tests cover inverse problems I and II with Brownian or Lévy noise in 1D and 2D, plus type II Brownian-noise cases in 3D and 4D.Source: Appendices · p. 19

A Problem I: L´evy noise

For type I with Lévy noise, the appendix tests one-snapshot inference across sample sizes and compares direct particle observations with KDE-derived PDF data. The direct-observation PINN is reported as more accurate.

A Problem I: L´evy noise

  • With one snapshot at t = 0.3, the predictions converged with increasing sample counts of 100, 1,000, and 10,000.The setting used a(x) = x − x^3, α = 1.5, σ = 0, and ε = 1; Figure 10 shows predictions against MC reference solutions and KDE estimates.Source: A Problem I: L´evy noise · p. 20, Figure 10 · p. 20
  • Using particle observations directly in the PINN produced more accurate results than using KDE-estimated PDFs as input data in the multiple-snapshot comparison.The comparison used 200 samples at t = 0.1, 0.3, and 0.5; Figure 11 displays the errors and PDF predictions.Source: A Problem I: L´evy noise · p. 21, Figure 11 · p. 21

B Problem II: Brownian noise

For type II with Brownian noise, the appendix compares drift and PDF inference under two observation schedules and two drift representations. Both schedules capture the PDF’s two peaks, while the later schedule yields smaller drift error.

B Problem II: Brownian noise

C Problem II: L´evy noise

The appendix evaluates inverse problem II with Lévy noise, using sparse particle observations to infer drift and PDF. It compares polynomial and neural-network drift parameterizations across two observation schedules.

C Problem II: Lévy noise

D Problem II: 2D Brownian noise

The 2D Brownian-noise example tests inverse problem II using particle observations to learn the drift and PDF. Four cases vary the observation schedule and drift parameterization.

D Problem II: 2D Brownian noise

E Problem II: 2D L´evy noise

The appendix applies inverse problem II to a 2D Lévy-noise example, comparing drift-learning cases with different observation schedules and parameterizations.

E Problem II: 2D Lévy noise

F Problem II: 3D and 4D Brownian noise

The appendix extends inverse problem II to 3D and 4D Brownian-noise cases, using cubic-polynomial drift parameterizations and multiple particle-observation schedules.

F Problem II: 3D and 4D Brownian noise

Loading 2008.10653v1…