PaperlayerView exact revision on arXiv

Source-linked AI summary

Physics-Informed Neural Network Surrogate for Oxygen Vacancy Dynamics in epitaxial $\mathrm{SrTiO_3}$ on Si memristors via Dynamic Spectral Optimization

Rodion Podorozhny, Nikoleta Theodoropoulou, Jelena Tešić

arXiv:2609.02966v1cond-mat.mtrl-scics.LG

TL;DR

PINN reliability remains difficult to assess in stiff, multiscale physical regimes, motivating evaluation against reference solvers across parameter spaces. This work applies a staged, physics-informed surrogate with log-space residual training to coupled transport in SrTiO3/Si memristors. It reports experimental current-voltage agreement with R2 > 0.96 and a differentiable forward pass for parameter optimization.

  • Problem

    PINN residual minimization can fail in stiff or nonlinear regimes, while practical understanding of errors in complex parameterized settings and localized discrepancies from reference solvers remains incomplete.

    Source: §I · p. 2, §V · p. 5

  • Method

    The framework evaluates PINN generalization against reference solvers and trains sequential subnetworks with log-space PDE residual losses for coupled ionic-electronic transport.

    Source: §I · p. 2, §III · p. 4, B. Composite PDE Training Loss · p. 7

  • Results

    R2 > 0.96 validates inverse parameter extraction against experimental conductive-AFM current-voltage curves.

    Source: C. Physical Validation Diagnostics and… · p. 10

  • Takeaways & Limitations

    The fully differentiable forward pass supports end-to-end optimization of physical transport and interface parameters alongside network weights.

    Source: D. Discussion: PINN vs. COMSOL… · p. 10

    Newton–Raphson mesh solvers can be ill-conditioned, fail to converge, or require impractically long runtimes for dynamic sweeps and multi-parameter studies.

    Source: §III · p. 3

Abstract

from arXiv · show

Physics-informed neural networks (PINNs) offer a promising framework for modeling semiconductor devices, yet standard architectures struggle with severe numerical stiffness and multiscale spatial discrepancies inherent to oxide heterostructures. Here, we demonstrate a cascaded PINN architecture coupled with a custom second-order Chebyshev second kind polynomial spectral optimizer (DSO V2 Hybrid) to model ion-electronic drift-diffusion transport in Pt/SrTiO$_3$/Si memristive heterostructures across a 20 nm STO film on a 380 $\mu$m Si substrate. By isolating potential, carrier density, and vacancy transport into four sequentially trained sub-neural-networks, our model circumvents condition numbers exceeding $10^{16}$ without operator splitting. The trained surrogate reproduces experimental conductive-AFM current-voltage hysteresis ($R^2 > 0.96$) while ensuring strict Poisson consistency across continuous space. Compared to conventional finite-element solvers (e.g., COMSOL), the PINN surrogate enables differentiable inverse parameter estimation and linear time inference.

I. INTRODUCTION AND RELATED WORK

The introduction identifies gaps in PINN reliability assessment and frames SrTiO3/Si memristors as a stiff, multiscale test case. The physical system couples electronic transport and mobile oxygen vacancies across the Pt/STO/Si structure.

I. INTRODUCTION AND RELATED WORK

  • PINN accuracy remains incompletely understood in complex parameterized settings, particularly for localized discrepancies from high-fidelity solvers that global metrics may mask [6,8].Existing theoretical bounds rely mainly on simplified linear assumptions, while standard residual minimization can fail in stiff or strongly nonlinear regimes [1,2,7,8].Source: §I · p. 2
  • The work probes PINN generalization across physical parameter spaces by mapping field and observable discrepancies against reference solvers, using oxygen-vacancy concentrations as a focus.SrTiO3/Si heterostructures provide a proof of concept because ionic-electronic coupling, space-charge accumulation, and field-dependent drift create severe stiffness.Source: §I · p. 2

II. PHYSICAL SYSTEM

  • The modeled Pt/STO/Si device has a 20 nm STO film on a 380 µm Si substrate, with coupled electronic drift-diffusion and mobile-vacancy dynamics.The normalized domain runs from the Pt/STO interface through the STO/Si junction to the back contact, and vacancy transport uses drift and diffusion.Source: §II · pp. 2–3
  • Bias-driven redistribution of oxygen vacancies across the 20 nm STO film modulates interfacial space charge, band bending, and the Pt/STO Schottky barrier [10].Modeling therefore requires resolving nonlinear coupling between ionic space charge and exponential electron injection.Source: §II · p. 2

III. COMSOL MODELING LIMITATIONS

The section describes why conventional finite-element models struggle with Pt/STO/Si transport: scale disparity, static vacancy approximations, stiffness, simplified statistics, and costly inverse fitting. It then introduces a PINN framework using staged subnetworks, log-space losses, and continuous PDE residual minimization.

III. COMSOL MODELING LIMITATIONS

  • A five-order-of-magnitude length-scale disparity forces mesh refinement and leads standard models to truncate the 380 µm Si substrate to 10 µm and simplify contact geometry to 1D [10,11].The disparity spans the Debye screening length, 20 nm STO film, and bulk Si substrate.Source: §III · p. 3
  • COMSOL workflows use static 5 nm charged vacancy blocks rather than solving time-dependent vacancy drift-diffusion during voltage sweeps, preventing continuous self-consistent electromigration modeling [10,11].The blocks are fixed near either the Pt/STO interface or film surface.Source: §III · p. 3
  • With λ2 ≈ 1.07 × 10^-1, ill-conditioned Newton–Raphson meshes can fail to converge or run impractically long, impeding dynamic sweeps and multi-parameter studies [10,11].The small Debye parameter relative to the normalized charge-density term produces sharp gradients across thin depletion regions.Source: §III · p. 3
  • Standard finite-element models use parabolic bands, complete dopant ionization, and Maxwell-Boltzmann rather than Fermi-Dirac statistics to reduce nonlinear complexity and maintain stability [10].Source: §III · p. 3
  • Forward-only finite-element fitting requires thousands of separate PDE evaluations for parameters such as series resistance, ideality factors, and barrier modulation, making systematic inverse optimization computationally intractable [11].Source: §III · p. 3
  • The proposed PINN framework combines log-space loss transformations, sequential multi-stage subnetworks, and continuous PDE residual minimization to address mesh stiffness, truncation, and convergence failures.Source: §III · p. 4

IV. CASCADED PHYSICS INFORMED NEURAL SUB-NETWORKS DESIGN

The PINN uses four sequential subnetworks to map vacancy concentration to electrostatic potential, carrier concentrations, and terminal current. Each stage embeds structure suited to its physical role.

Overall cascade

  • The cascade maps vacancy concentration to potential, carrier concentrations, and terminal current in sequence, isolating the coupled transport stages.The stages are cvnet, φnet, carriernet, and currentnet; their PDE-coupled fields feed a multi-objective loss.Source: §IV · p. 4, Figure 1 · p. 4

Stage 1: Vacancy Concentration Network

  • The vacancy network maps position, time, voltage, and sweep direction to local vacancy density, with a time-scaled hysteresis component and bounded output.The stated output range is Cv ∈ [0.3, 5.0].Source: §IV · pp. 4–5

Stage 2: Electrostatic Potential Network

  • The potential network blends region-specific STO and Si subnetworks across the heterojunction and anchors the STO potential to the Schottky contact.The STO boundary condition includes a vacancy-modulated correction for barrier lowering.Source: §IV · p. 5

Stage 3: Carrier Concentration Network

  • The carrier network represents electron and hole concentrations in log-space relative to analytical drift-diffusion equilibrium profiles.It predicts learned non-equilibrium departures from those profiles.Source: §IV · p. 5

Stage 4: Current Network

  • The current network maps interfacial state variables to total current while accounting for vacancy-driven Schottky barrier modulation and parasitic series resistance.Its predicted base current is scaled by a barrier factor and forward-bias factor, then corrected for series resistance.Source: §IV · p. 5

V. METHODOLOGY

The methodology addresses highly ill-conditioned, non-convex PINN loss landscapes, where stiff coupling and saddle-rich geometry challenge optimization. It motivates second-order approaches while noting their computational and curvature limitations.

Optimization challenges

  • PINN losses can have condition numbers around 10^16, dense multi-saddle geometry, and severe spectral anisotropy.The paper notes that Adam’s diagonal preconditioning struggles with cross-coupled ill-conditioning and rich saddles.Source: §V · p. 5

Loss landscape

  • The drift-diffusion Poisson system’s loss landscape has severe spectral anisotropy and stiff gradient coupling; Figure 2 depicts the carriernet loss landscape.The paper identifies use of second-order optimizer variants as one mitigation approach.Source: Figure 2 · p. 6, §V · p. 6

Optimization challenges

  • Positive-semidefinite pseudo-second-order preconditioners cannot capture negative curvature or resolve extreme non-separable ill-conditioning without numerical instability.They approximate curvature from gradient outer products rather than explicit Hessian inversion.Source: §V · pp. 5–6

Second-order optimization

  • Exact curvature extraction requires matrix solves with O(n^3) cost, motivating blockwise, Krylov, and Chebyshev approaches.The passage describes these methods as ways to reduce computational time while retaining second-order fidelity.Source: §V · p. 6

A. Hybrid Dynamic Spectral Optimizer

The hybrid Dynamic Spectral Optimizer uses Chebyshev recurrence and adapts its updates to tensor size and curvature properties. The methodology also describes subnet-specific relaxation, log-space carrier representations, and curvature smoothing.

Dynamic Spectral Optimizer

  • DSO uses a second-kind Chebyshev polynomial recurrence to approximate Hessian inversion through matrix-vector products, with polynomial degree adapted to spectral properties.DSO V2 Hybrid uses two update rules based on tensor size.Source: A. Hybrid Dynamic Spectral Optimizer · p. 6

Adaptive relaxation

Carrier concentration log-space formulation

  • Carrier concentrations are represented in exponential log-space because small potential perturbations trigger exponential changes in n and p.The formulation combines equilibrium concentrations with learned departures and scaling parameters.Source: A. Hybrid Dynamic Spectral Optimizer · p. 7

Dynamic block partitioning and curvature smoothing

  • Parameter blocks, tensor dimensions, and EMA curvature-smoothing parameters are dynamically updated to retain second-order sensitivity while limiting gradient noise and ill-conditioning.Source: A. Hybrid Dynamic Spectral Optimizer · p. 7

Sub-network-specific optimizer performance

B. Composite PDE Training Loss

The composite PDE training loss uses log-space residuals to address scale imbalance in the normalized electrostatic equation and combines physical residuals with boundary, regularization, initial-condition, and hysteresis terms.

Electrostatic residual

  • The normalized electrostatic residual is formulated in log-space because λ² ≈ 0.107 is much smaller than the normalized reaction term, creating imbalanced gradients and ill-conditioning.The passage states that ε_num = 10^-10 ensures numerical stability.Source: B. Composite PDE Training Loss · p. 7

Vacancy drift residual

Composite loss

  • The composite PDE training loss balances mathematical residual satisfaction for PDE physical residuals, boundary conditions, physical regularizations, and hysteresis constraints as distinct loss components.The displayed coefficients are 5 for each of BC0 and BCi, 0.1 for regularization, 10 for initial-condition, and 3 for hysteresis terms.Source: B. Composite PDE Training Loss · p. 7

C. Network Formulation, Activation, and Training

Training progresses from analytical pretraining to PDE residual minimization and then joint fitting to experimental I–V curves. Activation selection and automatic differentiation support accurate residual evaluation.

Training Pipeline

  • Analytical pretraining runs for 7,000 epochs per sub-network, followed by 10,000 epochs of PDE residual minimization over 500 Sobol points while φnet and hysteresis parameters are frozen.The final fitting stage unfreezes all weights and physical parameters for joint optimization against experimental I–V curves using PCGrad.Source: C. Network Formulation, Activation,… · p. 7

Differentiable Experimental Fitting

  • Experimental fitting jointly optimizes all network weights and physical parameters against I–V curves, using PCGrad to resolve gradient interference between trace and retrace branches.The parameters include Rseries, ntrace/ret, β, and δCv.Source: C. Network Formulation, Activation,… · p. 8

Activation Function Selection Strategy

  • GELU is selected for its smoothness and non-zero second-order gradients; ReLU has vanishing second derivatives, while SIREN produces about 29× higher loss than GELU.The stated rationale includes GELU's superior Hessian conditioning and non-periodic behavior for monotonic electrostatic potential profiles.Source: C. Network Formulation, Activation,… · p. 8

Collocation and Automatic Differentiation

  • Second-order spatial derivatives of φ and Cv are evaluated at 500 Sobol points using vector-mapped automatic differentiation rather than manual finite differences.The text states this preserves exact gradient evaluation while optimizing computational performance.Source: C. Network Formulation, Activation,… · p. 8

Physical and Learnable Parameters

A. Pretraining and Loss Landscape Analysis

Analytical pretraining losses vary across the cascaded sub-networks, alongside distinct loss-landscape and kernel diagnostics. The reported optimizer gains are largest for currentnet and more modest for φnet.

Loss Landscape Analysis

  • Carrier exponentiation drives Hessian condition numbers up to approximately 10^16, while sub-networks show different condition-number trajectories, negative spectral mass, and off-diagonal curvature coupling.The analysis also reports distinct loss landscapes across the sub-networks.Source: A. Pretraining and Loss Landscape… · p. 8

NTK Diagnostics

  • NTK diagnostics reveal differences in positive-semidefinite Gauss–Newton kernel spectra, kernel evolution, and mode-wise convergence, supporting specialized hybrid optimizers across sub-network loss landscapes.The passage describes mode-wise first-order training dynamics associated with the kernel spectrum.Source: A. Pretraining and Loss Landscape… · p. 9

Pretraining Results

  • The reported improvement is 6.6× over Adam (3.09×10−2) and 23× over DSO-EMA (1.10×10−1), more modest than other sub-networks, where currentnet improvements reach up to 545×.Table III compares analytical pretraining losses across sub-neural-networks.Source: A. Pretraining and Loss Landscape… · p. 9, Table III · p. 9

B. Empirical Activation Benchmark

The physical validation reports close agreement with experimental I–V curves and consistency with physical criteria. The activation benchmark identifies GELU as superior for Hessian stability and residual minimization.

B. Empirical Activation Benchmark

  • GELU provides superior Hessian stability and residual minimization among the compared activation functions, with its non-zero second derivative contrasting with ReLU's.Table IV is identified as the quantitative comparison of activation functions.Source: B. Empirical Activation Benchmark · p. 9, Table IV · p. 9

C. Physical Validation Diagnostics and Experimental I–V Fitting

  • R2 coefficients exceed 0.96; Δinterface = 0.265, Wratio = 1.52, and ρratio = 46.5 are reported as confirming vacancy-driven barrier modulation, Poisson consistency, and smooth depletion dynamics.Table VI is captioned as reporting that all validation tests passed using the S04N dataset.Source: C. Physical Validation Diagnostics and… · p. 10, Table VI · p. 10
  • Oxygen partial pressure during growth modulates oxygen-vacancy concentration, and the PINN maps growth parameters to continuous internal fields and extracts Vup and Vdn.The experimental I–V curves were acquired using conductive atomic force microscopy, with trace and retrace denoting increasing and decreasing voltage ramps.Source: C. Physical Validation Diagnostics and… · pp. 9–10

D. Discussion: PINN vs. COMSOL Solution Quality

The PINN surrogate is reported to match experimental I–V behavior while providing continuous, self-consistent fields across the full Pt/STO/Si domain. The discussion contrasts its differentiable parameter optimization and inference with COMSOL’s computational and convergence burdens, then outlines future extensions.

D. Discussion: PINN vs. COMSOL Solution Quality

  • The model represents ionic redistribution through dynamic latent variables and unifies ionic, carrier, and potential fields in a pointwise differentiable computational graph.The potential is represented as a continuous function of position, time, voltage, and vacancy concentration.Source: D. Discussion: PINN vs. COMSOL… · p. 10
  • R2 > 0.96 for agreement with experimental data, alongside continuous internal fields and faster inference than finite-element solvers.The comparison concerns Pt/STO/Si heterostructures and reports PINN inference scaling as O(Nθ) versus O(n^3) per COMSOL I–V curve.Source: Figure 3 · p. 10, §VII · p. 11
  • Table VII is presented as a comprehensive comparison of PINN and COMSOL solution quality.The supplied table passage gives its scope but no metric values.Source: Table VII · p. 10
  • The forward pass optimizes transport and interface parameters with network weights by backpropagation, removing outer-loop parameter sweeps.The discussion also describes replacing mesh-generation overhead, spatial-domain truncation, and Newton–Raphson non-convergence risks with GPU-based inference.Source: D. Discussion: PINN vs. COMSOL… · p. 10

VII. CONCLUSION AND FUTURE WORK

  • Future work will extend the cascaded PINN to 2D and 3D crossbar arrays, add thermal dynamics, and integrate it into hardware design and parameter-extraction pipelines.The conclusion also proposes studying optimizer spectra to inform online switching between optimization modes.Source: §VII · p. 11
Loading 2609.02966v1…