PaperlayerView exact revision on arXiv

Source-linked AI summary

A Physics-Informed Deep Learning Paradigm for Traffic State and Fundamental Diagram Estimation

Rongye Shi, Zhaobin Mo, Kuang Huang, Xuan Di, Qiang Du

arXiv:2106.03142v4cs.LG

TL;DR

Traffic state estimation must recover traffic variables from partial observations, yet model-driven approaches can have limited dynamics and data-driven approaches require extensive representative data. This paper introduces PIDL+FDL, which learns an unknown fundamental diagram within a physics-informed framework while estimating states and model parameters. On real highway data, the authors report improved estimation accuracy and data efficiency over baseline methods, alongside FD estimation capability.

  • Problem

    Model-driven TSE may inadequately capture real-world dynamics, while data-driven TSE requires massive representative historical data for generalizable estimates.

    Source: §I · p. 1

  • Method

    PIDL+FDL integrates an ML surrogate for an unknown fundamental diagram into PIDL to jointly estimate traffic states, FD relations, and model parameters.

    Source: §I · p. 1, A. PIDL for TSE · p. 3

  • Results

    PIDL+FDL methods generally achieved the best estimation accuracy and data efficiency over the TSE baselines, and the reshaped variant jointly conducted high-quality TSE, parameter identification, and FD estimation with relatively small amounts of observed data.

    Source: C. Results and Discussion · p. 8, D. Discussions on Fundamental Diagram… · p. 9

  • Takeaways & Limitations

    The experiments show that PIDL+FDL can estimate traffic states, model parameters, and the underlying FD simultaneously on highway data.

    Source: §VII · p. 9

    Hyperparameter tuning is complex and the model basically must be tuned for each scenario, limiting real-world applicability; the effect of high noise and corruption also needs further investigation.

    Source: §VII · p. 9

Abstract

from arXiv · show

Traffic state estimation (TSE) bifurcates into two categories, model-driven and data-driven (e.g., machine learning, ML), while each suffers from either deficient physics or small data. To mitigate these limitations, recent studies introduced a hybrid paradigm, physics-informed deep learning (PIDL), which contains both model-driven and data-driven components. This paper contributes an improved version, called physics-informed deep learning with a fundamental diagram learner (PIDL+FDL), which integrates ML terms into the model-driven component to learn a functional form of a fundamental diagram (FD), i.e., a mapping from traffic density to flow or velocity. The proposed PIDL+FDL has the advantages of performing the TSE learning, model parameter identification, and FD estimation simultaneously. We demonstrate the use of PIDL+FDL to solve popular first-order and second-order traffic flow models and reconstruct the FD relation as well as model parameters that are outside the FD terms. We then evaluate the PIDL+FDL-based TSE using the Next Generation SIMulation (NGSIM) dataset. The experimental results show the superiority of the PIDL+FDL in terms of improved estimation accuracy and data efficiency over advanced baseline TSE methods, and additionally, the capacity to properly learn the unknown underlying FD relation.

I. INTRODUCTION

Traffic state estimation must reconstruct flow, density, and speed from partial sensor observations, but model-driven and data-driven approaches face different constraints. PIDL+FDL embeds a learned fundamental diagram in the physics-informed component, enabling joint state estimation, parameter identification, and FD estimation.

I. INTRODUCTION

  • Model-driven methods can miss real-world dynamics and require substantial data for calibration, while data-driven methods rely on massive representative histories for generalization.Data-driven methods also lack explicit traffic models and their interpretable, deductive insights.Source: §I · p. 1
  • PIDL combines a physics-informed network for regularization with a physics-uninformed network for state estimation to integrate model-driven and data-driven components.The paper proposes an ML surrogate for the FD rather than hard-coding a potentially complicated FD equation.Source: §I · p. 1
  • PIDL+FDL jointly estimates traffic states, learns the FD without FD output measurements, and identifies model parameters outside the FD terms.The FD surrogate is trained under physical regularization alongside TSE.Source: §I · pp. 1–2
  • The authors validate PIDL+FDL on Greenshields-based LWR and ARZ models and NGSIM data, reporting improved estimation accuracy and data efficiency over baselines.They also report that the method can estimate FD relations and model parameters.Source: §I · p. 2

II. RELATED WORK OF TRAFFIC STATE ESTIMATION

  • Unlike approaches that fit velocity functions using probe-vehicle measurements, this paper learns an unobserved FD relation end-to-end under PIDL regularization.The framework assumes conservation, optionally a momentum equation, but does not specify a mathematical relation between traffic quantities.Source: §II · p. 2

III. MATHEMATICAL SETTING FOR PIDL+FDL

PIDL estimates traffic states over a continuous space-time domain from partial observations while enforcing the governing traffic model. Its training objective balances data fit and physical consistency, and can identify unknown model parameters.

A. PIDL for TSE

  • PIDL represents traffic states with a neural network taking time and location as inputs, then uses a physics-informed network to compute model residuals from its estimates.The residual approaches zero as the estimated states approach the true states.Source: A. PIDL for TSE · pp. 2–3
  • Observation points carry measured states, whereas auxiliary points need no measurements and can be deployed freely for regularization.Both sets are drawn from the grid representing the continuous time-space domain.Source: A. PIDL for TSE · p. 3
  • Training minimizes a weighted sum of observation error and physical residual error, with residuals evaluated at auxiliary points.The observation error is MSE against observed target states; the physical error is MSE between auxiliary-point residuals and zero.Source: A. PIDL for TSE · p. 3
  • Unknown traffic-model parameters are treated as learnable variables, and training jointly optimizes them with the state-estimation network.The resulting parameterized model represents the physics inferred from observed data, while the network estimates states across the grid.Source: A. PIDL for TSE · p. 3

B. PIDL + FDL for TSE

PIDL+FDL replaces a hard-coded hidden traffic relation with an ML surrogate, retaining physics-based regularization while jointly learning states, the relation, and model parameters. It can also add localized regularization to reshape abnormal FD curves.

B. PIDL + FDL for TSE

  • Complicated model terms can make differentiation-based PIDL training sensitive to network design, so PIDL+FDL represents the hidden relation with a learnable ML surrogate.The surrogate is intended to be more differentiation-friendly than hard-encoding complicated terms.Source: B. PIDL + FDL for TSE · p. 3
  • The FD learner takes estimated traffic variables as input and represents the unknown fundamental-diagram relation with a parameterized ML model.Its output replaces the hidden relation in the traffic-model residual.Source: B. PIDL + FDL for TSE · p. 3
  • Training jointly optimizes state estimates, FD-learner parameters, and model parameters, even though the hidden FD output is not observed by sensors.The learned FD is obtained under PIDL physical regularization rather than directly from FD output data.Source: B. PIDL + FDL for TSE · p. 4
  • An additional regularizer can impose FD concavity over a selected density interval to reshape abnormal edge-region curves.The interval is controlled by hyperparameters a and b so other density regions are not regularized.Source: B. PIDL + FDL for TSE · p. 4

IV. PIDL+FDL FOR GREENSHIELDS-BASED LWR

The first numerical example tests PIDL+FDL on a Greenshields-based LWR model, using a simulated ring-road density field as ground truth. The experiment estimates the field from observations while also recovering the FD and model parameters.

IV. PIDL+FDL FOR GREENSHIELDS-BASED LWR

  • The first numerical example tests whether PIDL+FDL can estimate traffic dynamics governed by a Greenshields-flux LWR model.Source: §IV · p. 4
  • The example uses a diffusively corrected LWR equation, with diffusion coefficient ϵ, alongside the conservation-law LWR formulation.The diffusion term is included to represent phenomena such as delayed driving behavior due to driver reaction time.Source: §IV · p. 4
  • A simulated ring-road density field, generated with a Godunov scheme, serves as ground truth for estimating the traffic field, FD relation, and model parameters.The simulation describes a propagating shockwave that continues across the ring-road boundary.Source: §IV · p. 4

A. PIDL+FDL Architecture Design

PIDL+FDL replaces a hard-coded fundamental diagram (FD) with a learned flow function, enabling traffic-state estimation, parameter identification, and FD estimation when the FD is unknown.

A. PIDL+FDL Architecture Design

  • A neural network estimates flow from density to represent the otherwise unknown FD, without assuming a specific FD relation.The setup has conservation-law and boundary-condition knowledge, but no direct flow observations.Source: A. PIDL+FDL Architecture Design · p. 4
  • The PUNN estimates traffic density, while the PINN+FD Learner calculates the residual using the learned flow function and parameter ϵ.The PINN's connecting weights are fixed, and ϵ is held in a variable node.Source: A. PIDL+FDL Architecture Design · p. 4, Figure 1 · p. 5
  • The training loss combines observation and auxiliary-point errors with two boundary-condition penalties.B1 penalizes differences between estimates at x=0 and x=1; B2 penalizes differences in their first derivatives.Source: A. PIDL+FDL Architecture Design · p. 5

B. TSE+FDL using Observation from Loop Detectors

Loop-detector experiments show that PIDL+FDL can estimate density, identify a model parameter, and recover an unknown FD; density estimation and parameter identification improve with more detectors.

B. TSE+FDL using Observation from Loop Detectors

  • Training uses detector observations, 100,000 randomly selected auxiliary grid points, and boundary points; the simulated density solution is treated as ground truth.Boundary auxiliary points are deployed at x=0 and x=1 for selected time points.Source: B. TSE+FDL using Observation from Loop… · p. 5, Figure 2 · p. 5
  • With 4 loops, estimated density closely matches the true dynamics; errors below 6 × 10−2 are empirically visually indistinguishable.The reported L2 estimation error is 1.287 × 10−2.Source: B. TSE+FDL using Observation from Loop… · p. 5
  • With 4 loops, density error was 1.287 × 10−2; with 3 loops, error was 3.327×10−2 and ϵ*=0.00495 versus true 0.005; 5 loops recovered the exact Greenshields FD.Density error improves as detector count increases; model parameters converge to their true values when more than two detectors are used.Source: B. TSE+FDL using Observation from Loop… · pp. 5–6, Table I · p. 6

V. PIDL+FDL FOR GREENSHIELDS-BASED ARZ

The paper tests PIDL+FDL on Greenshields-based ARZ dynamics, a second-order model with density and velocity as state variables, using numerical solutions as ground truth.

V. PIDL+FDL FOR GREENSHIELDS-BASED ARZ

  • The ARZ experiment targets a second-order traffic model whose state variables are density ρ and velocity u.The model includes a vehicle-conservation law and a momentum equation on velocity.Source: §V · p. 6
  • The ARZ setting uses ρmax = 1.13, umax = 1.02, and relaxation time τ = 0.02.The equilibrium velocity is Ueq, and h(ρ) defines traffic pressure.Source: §V · p. 6
  • From bell-shaped initial density and velocity profiles, the Lax–Friedrichs scheme produces numerical solutions treated as ground truth.The solutions cover a 960-by-240 grid over the [0, 3] × [0, 1] domain.Source: §V · p. 6, Figure 4 · p. 6

A. PIDL+FDL Architecture Design for ARZ

The ARZ PIDL+FDL uses an equilibrium-velocity surrogate to form model residuals and jointly estimate traffic states and the relaxation parameter. In the numerical experiment, state estimates become acceptable with more than three loops, and four or more loops allow correct recovery of the Greenshields FD.

A. PIDL+FDL Architecture Design for ARZ

  • The architecture uses equilibrium velocity Ueq as the target FD relation and defines two residuals for the ARZ dynamics, with relaxation time τ as the only model parameter to learn.The architecture is shown in Fig. 5, where τ is held by a variable node.Source: A. PIDL+FDL Architecture Design for ARZ · p. 6, Figure 5 · p. 7
  • More than three loop detectors yield acceptable traffic-state estimates, using error thresholds of 6.00 × 10^-2 for density and 2.90 × 10^-2 for velocity.Table II reports PIDL+FDL performance on ARZ dynamics.Source: A. PIDL+FDL Architecture Design for ARZ · p. 7, Table II · p. 7
  • With four or more loops, PIDL+FDL correctly learns the Greenshields FD relation, and FDL performance improves as loop-detector count increases.The corresponding visualization is referenced as Figure 7 in the supplementary material in [22].Source: A. PIDL+FDL Architecture Design for ARZ · p. 7

VI. PIDL+FDL-BASED TSE ON NGSIM DATA

This section evaluates PIDL+FDL-based traffic-state estimation on real-world NGSIM data and compares its performance with baselines.

VI. PIDL+FDL-BASED TSE ON NGSIM DATA

  • The real-data evaluation uses the NGSIM dataset to assess PIDL+FDL-based traffic-state estimation against baseline methods.The section first introduces the methods and then the baselines.Source: §VI · p. 7

A. NGSIM Dataset

The study uses vehicle trajectories from a US 101 segment in the NGSIM dataset, sampled into a spatial-temporal grid and observed through loop detectors.

A. NGSIM Dataset

  • The NGSIM data cover about 680 meters of US 101 for 2,770 seconds, based on vehicle trajectories converted from camera video.The monitored segment was recorded on June 15, 2005.Source: A. NGSIM Dataset · p. 7
  • After preprocessing, the traffic-state grid contains 21 spatial cells and 1,770 temporal cells, with density calculated about every 30 meters over 1.5-second periods.Grid points are placed at the center of each cell.Source: A. NGSIM Dataset · p. 7
  • Loop detectors provide observations every 1.5 seconds and are assumed to record density and average velocity at selected locations.By default, the detectors are evenly installed along the highway segment.Source: A. NGSIM Dataset · p. 7

B. TSE Methods for Real Data

On NGSIM, PIDL+FDL-based methods generally achieve the best estimation accuracy and data efficiency among the evaluated baselines. The section compares LWR- and ARZ-based implementations and explains how observation density and model complexity relate to performance.

B. TSE Methods for Real Data

  • LWR-PIDL+FDL uses the first-order LWR equation, estimates velocity as Q(ρ)/ρ, and trains with 80% of grid points as auxiliary points.A reshaping regularization term is applied because noisy real data can produce abnormal learned FD curves.Source: B. TSE Methods for Real Data · pp. 7–8, Figure 6 · p. 7
  • ARZ-PIDL+FDL applies the second-order ARZ model with boundary conditions removed from the loss, using the other LWR-PIDL+FDL experimental setup.Its architecture follows Fig. 5.Source: B. TSE Methods for Real Data · p. 8
  • The comparison includes linear two-dimensional interpolation, adaptive smoothing, spatial LSTM, pure neural network, extended Kalman filter, and PIDL baselines.The LSTM estimates the next cell's traffic density and velocity using information from previous cells along the spatial dimension.Source: B. TSE Methods for Real Data · p. 8

C. Results and Discussion

  • With few loop detectors, EKF, Interp2, and AS outperform NN/LSTM, whose performance catches up as detector count increases.The paper notes that EKF performance flattens, while interpolation and smoothing may miss subtle unobserved state changes.Source: C. Results and Discussion · p. 8
  • PIDL+FDL-based methods generally achieve the best estimation accuracy and data efficiency over the TSE baselines on NGSIM.Fig. 7 presents the comparison across different numbers of loop detectors.Source: C. Results and Discussion · p. 8, Figure 7 · p. 8
  • LWR-PIDL+FDL can outperform ARZ-PIDL+FDL in this NGSIM setting, consistent with the reported training-flexibility trade-off for more sophisticated models.The paper attributes the trade-off to complicated terms that can make performance sensitive to PINN structural design.Source: C. Results and Discussion · p. 8

D. Discussions on Fundamental Diagram Estimation

PIDL+FDL estimates traffic states, model parameters, and fundamental diagrams together, and its real-highway experiments outperform baselines in estimation accuracy, data efficiency, and FD estimation.

D. Discussions on Fundamental Diagram Estimation

  • PIDL-based FD curves have predefined forms: the 3-parameter LWR flux fits density-flow points unsatisfactorily, while the quadratic Greenshields ARZ curve captures them only to a limited extent.Fig. 8 compares estimated curves; learned equilibrium velocity is converted to flow for visualization.Source: D. Discussions on Fundamental Diagram… · pp. 8–9, Figure 8 · p. 9
  • With reshaping regularization, PIDL+FDL fits NGSIM density-flow points and captures their characteristics; its learned curves fit observed data around ρ ∈ [0, 0.5], while sparse high-density regions can curl upward.The reshaping prior imposes concavity over ρ ∈ [0.6, 0.7].Source: D. Discussions on Fundamental Diagram… · p. 9
  • For ARZ-PIDL+FDL, τ estimates from 3, 6, and 8 loops are 23.36, 25.99, and 27.66, respectively, reasonably close to data-fitted values 27.6, 28.8, and 30.5.The learned equilibrium velocity function and τ reconstruct the ARZ model.Source: D. Discussions on Fundamental Diagram… · p. 9
  • The framework jointly performs high-quality traffic-state estimation, model-parameter identification, and FD estimation using relatively small amounts of observed data.The LWR-PIDL+FDL reconstruction has no additional model parameters; ARZ-PIDL+FDL learns relaxation time τ.Source: D. Discussions on Fundamental Diagram… · p. 9

VII. CONCLUSION

  • On real highway data, PIDL+FDL-based approaches outperform baselines in estimation accuracy, data efficiency, and fundamental-diagram estimation.The framework is evaluated using loop detector data.Source: §VII · p. 9
  • Hyperparameter tuning across scenarios such as spatial and temporal resolution, grid size, and observation error complicates real-world applicability.The paper also identifies noisy data as an issue requiring further investigation.Source: §VII · p. 9
Loading 2106.03142v4…