PaperlayerView exact revision on arXiv

Source-linked AI summary

Development and Validation of a Physics-Guided Machine Learning Extrapolation Framework Using a Classical Transient Diffusion Benchmark

Ashutosh Yadav, Alok Dubey, Prodyut Ranjan Chakraborty, Harshal Akolekar

arXiv:2609.09912v1cs.LG

TL;DR

Engineering machine-learning models often lack reliable validation beyond limited training ranges, making extrapolation the central challenge. The paper evaluates physics-guided BiLSTM and PINN frameworks on an analytically solvable transient diffusion benchmark using sequential exact validation, and reports accurate, physically consistent predictions beyond the training domain.

  • Problem

    Engineering machine-learning models are commonly trained over limited operating ranges, while reliable predictions and validation may be required beyond those ranges.

  • Method

    The study evaluates physics-guided BiLSTM and PINN frameworks on a transient diffusion benchmark with exact analytical ground truth and a train–predict–validate–extend procedure.

  • Results

    The proposed framework produces accurate and physically consistent extrapolated predictions beyond the training domain.

  • Takeaways & Limitations

    Exact validation at every recursive step enables quantitative certification of extrapolation reliability in a controlled engineering benchmark.

  • Takeaways & Limitations

    Backward extrapolation remains especially challenging near Fo→0 because the transient solution is strongly nonlinear and singular, while standard architectures do not consistently capture the required dynamics.

Abstract

from arXiv · show

Machine learning models used in engineering are typically trained within limited operating ranges, yet reliable predictions are often required beyond these domains. Consequently, the primary challenge is extrapolation rather than interpolation. Rigorous validation is hindered by the scarcity of data outside the training range. To address this limitation, a novel extrapolation framework is integrated with established machine learning architectures to enable accurate and physically consistent predictions beyond the training domain. The framework is established by systematically evaluating two physics-guided architectures: a Bidirectional Long Short-Term Memory (BiLSTM) network and a Physics-Informed Neural Network (PINN). A classical one-dimensional transient diffusion problem is adopted as a benchmark because its exact analytical solution provides unlimited, reliable data across the spatio-temporal domain, enabling rigorous quantitative validation. The problem is particularly challenging because the solution evolves from an initial singularity through a strongly nonlinear transient regime before approaching a steady-state linear profile. When training data are confined to an intermediate portion of this evolution, backward extrapolation toward the singularity becomes especially demanding. To improve reliability, physics-guided coordinate transformations, boundary-aware learning strategies, and stability-enhancing temporal marching are incorporated. Extrapolation is evaluated using a train-predict-validate-extend strategy, in which validated predictions are recursively added to the training set to progressively extend the prediction horizon. The results demonstrate accurate and physically consistent predictions beyond the training domain, highlighting the framework's potential for engineering applications where data availability is limited.

1. Introduction

The paper addresses reliable engineering-machine-learning extrapolation beyond limited training ranges using analytically verifiable transient diffusion data. It compares physics-guided BiLSTM and PINN approaches while targeting the nonlinear, singular early-time regime where backward extrapolation is especially difficult.

  • Reliable extrapolation, rather than interpolation, is the central engineering machine-learning challenge because data outside operating ranges are costly or infeasible to obtain.
  • The one-dimensional transient heat-conduction benchmark provides an analytical solution across the full spatio-temporal domain for quantitative validation beyond training data.Its dimensionless formulation generalizes across systems sharing the same Fourier number.
  • The benchmark transitions from boundary singularities and strongly nonlinear early-time behavior toward a linear steady state, creating distinct extrapolation regimes.
  • Backward extrapolation toward Fo→0 is especially difficult because models must recover sharper fields from later, diffused states, and errors increase substantially in the strongly nonlinear regime.
  • The reported limitations indicate that reliable extrapolation requires representations and training treatments aligned with diffusion physics, including the Fourier-number scaling, boundary behavior, and early-time gradients.
  • The study proposes complementary physics-guided BiLSTM and PINN frameworks intended to support extrapolation under different levels of available physical knowledge.The BiLSTM emphasizes data-driven temporal evolution with minimal physics-guided modifications, while the PINN incorporates stronger physical structure.

2. Methodology

The methodology uses an exact one-dimensional transient diffusion solution to generate dimensionless data, train models over a narrow intermediate Fourier-number window, and test extrapolation in both directions. A sequential train–predict–validate–extend procedure verifies each prediction against the analytical solution before extending the horizon.

  • 2.1. Governing Equations: The benchmark is a one-dimensional transient diffusion problem with Dirichlet boundaries, expressed using dimensionless position, temperature, and Fourier number.The dimensionless formulation removes dependence on particular diffusivity, length, and boundary-temperature values.
  • 2.2. Analytical Solution: The analytical series solution supplies exact reference values throughout the domain, with higher-order modes decaying faster as Fourier number increases.At small Fourier numbers, more modes remain significant and the solution varies rapidly; at large Fourier numbers, it approaches the linear steady-state profile.
  • 2.5. Extrapolation Strategy: Extrapolation proceeds incrementally in both directions, with backward prediction toward Fo→0 providing the more demanding test because it requires recovering increasingly fine structure.
  • 2.5. Extrapolation Strategy: Each predicted step is compared with the exact analytical solution, and only validated predictions are added to the working history before the horizon is extended.This train–predict–validate–extend cycle emulates deployment while retaining pointwise verification at every stage.
  • 2.4. Analytical Dataset & Training: Training is restricted to Fo∈[0.001, 0.009], a strongly transient window containing 81 Fourier-number values and 32,481 spatio-temporal data points.The complete dataset spans a wider range and uses finer sampling where the solution changes most rapidly.
  • 2.4. Analytical Dataset & Training: Model performance is assessed using MAE, RMSE, and R^2, with interpolation checks inside the training window and extrapolation tests outside its boundaries.

3. Demonstration I: Physics-Guided Sequential Extrapolation Framework Using BiLSTM

The Physics-guided BiLSTM addresses three extrapolation barriers by aligning time with diffusion physics, emphasizing boundary errors, and controlling sequential error growth during backward marching.

  • 3.3. Physics-guided BiLSTM Architecture: The framework applies three physics-derived corrections: Root-Fourier input, boundary-weighted loss, and relaxation marching for sequential extrapolation.These corrections target distorted time scaling, near-boundary gradients, and error accumulation outside the training window.
  • 3.3. Physics-guided BiLSTM Architecture: The BiLSTM uses two independent LSTM chains that process sequences in opposite temporal directions and concatenate their hidden states.The implementation also uses three bidirectional LSTM layers, a sliding window of K = 8, and a four-layer decoder for the spatial temperature profile.
  • 3.3.1. Root-Fourier Transformation: Root-Fourier coordinates align the network input with the diffusion timescale by replacing the plain Fourier-number representation.The transformation reflects the square-root dependence of thermal penetration depth on Fourier number and softens the early-time singularity.
  • 3.3.2. Boundary-Weighted Loss: Boundary weighting assigns w(0) = w(1) = 6.0 and w(0.5) ≈ 1.0, penalizing boundary errors six times more than interior errors.The weighting targets high-curvature regions where sequential extrapolation is most vulnerable.
  • 3.3.3. Relaxation Marching: Relaxation marching blends each new prediction with the previous state to limit error accumulation during backward extrapolation.Backward marching reverses diffusion-mode decay, amplifying higher-order modes; under-relaxation constrains this growth.

4. Demonstration II: Modified PINN Extrapolation Framework

The Modified PINN addresses near-singular extrapolation by transforming time, enforcing boundary conditions exactly, and training across progressively expanded Fourier-number ranges. It provides an independent validation vehicle for the same train-then-validate methodology used with the BiLSTM.

  • 4.1. PINN and Limitations: The Modified PINN targets two physics-driven barriers: residual-gradient singularity near 𝐹𝑜→0 and boundary-error leakage into the interior.These arise because the residual gradient scales as 𝐹𝑜−3∕2 and soft boundary penalties leave errors that diffuse inward.
  • 4.2. Network Architecture and Training Configuration: The modified architecture and training strategy are presented as a second, independent demonstration of extrapolation-validation methodology.The figure describes curriculum Fourier-number expansion toward demanding extrapolation targets.
  • 4.3.1. Log-Time Transformation: A log-time coordinate reduces the residual-gradient scaling from approximately 𝐹𝑜−3∕2 to 𝐹𝑜−1∕2, improving conditioning near the initial condition.The untransformed gradient grows by six orders of magnitude from 𝐹𝑜=0.01 to 𝐹𝑜=0.0001, compared with two orders for the transformed gradient.
  • 4.3.2. Hard Boundary Condition Enforcement: Hard boundary-condition enforcement makes the boundary values exact for any network output, allowing optimization to focus on PDE and initial-condition terms.The ansatz removes the boundary-error component that can otherwise leak inward through PDE coupling.
  • 4.4. Role of the PINN Framework in Extrapolation Validation: The framework combines PDE-residual training with a train-then-validate procedure that is unchanged from the BiLSTM case.The PINN uses unlabeled points through the governing equation, while the shared methodology validates predictions against the analytical solution.

5. Results and Discussions

The results first establish in-domain fidelity for both surrogate frameworks before evaluating extrapolation against the analytical solution. The interpolation checkpoint confirms accurate reproduction within the training window, with the Modified PINN achieving tighter fits than the Physics-guided BiLSTM at both targets.

  • 5.1. Interpolation: Interpolation is evaluated at 𝐹𝑜= 0.002 and 𝐹𝑜= 0.007 within the training range 𝐹𝑜∈[0.001, 0.009] before extrapolation testing.The checkpoint is intended to establish baseline fidelity rather than serve as the main result.
  • 5.1.1. BiLSTM Interpolation: The Physics-guided BiLSTM achieves 𝑅2 = 0.99003 with MAE = 0.00607 at 𝐹𝑜= 0.002 and 𝑅2 = 0.99953 with MAE = 0.00239 at 𝐹𝑜= 0.007.The higher-Fourier-number target is easier because high-frequency content has largely decayed.
  • 5.1.2. Modified PINN Interpolation: The Modified PINN achieves 𝑅2 = 0.99978 with MAE = 0.00122 at 𝐹𝑜= 0.002 and 𝑅2 = 0.99998 with MAE = 0.00067 at 𝐹𝑜= 0.007.Both fits are tighter than the corresponding Physics-guided BiLSTM results.

5.2. Extrapolation: BiLSTM

Extrapolation is tested on both sides of the training range and validated against the analytical ground truth. Physics-guided modifications substantially improve the difficult backward case, while progressively extended sweeps show reliable predictions across lower and upper Fourier-number regimes.

  • 5.2.1. Bi-LSTM Extrapolation: The unmodified Bi-LSTM performs worse backward than forward, with 𝑅2 = 0.81832 and MAE = 0.02389 at 𝐹𝑜= 0.0009 versus 𝑅2 = 0.98824 and MAE = 0.01050 at 𝐹𝑜= 0.0091.Both predictions are checked against the analytical ground truth.
  • 5.2.1. Bi-LSTM Extrapolation: 𝑅2 rises from 0.81832 to 0.98310 at 𝐹𝑜= 0.0009, while forward extrapolation reaches 𝑅2 = 0.99890 at 𝐹𝑜= 0.0091 after physics-guided modifications.The corresponding MAE values are 0.02389 for the standard backward case, 0.00596 for the modified backward case, and 0.00375 for the modified forward case.
  • 5.2. Extrapolation: The progressively extending validation procedure evaluates targets across both sides of the training window rather than relying on a single extrapolation point.Validated predictions are recursively used to extend the prediction horizon.
  • 5.2.2. Lower-Regime Robustness: The lower-regime sweep remains accurate: 𝑅2 = 0.98029 with MAE = 0.00626 at 𝐹𝑜= 0.00075 and 𝑅2 = 0.98775 with MAE = 0.00510 at 𝐹𝑜= 0.00085.The nearer target performs slightly better than the more distant target, consistent with gradual error softening.
  • 5.2.3. Upper-Regime Robustness: The upper-regime sweep remains high at 𝑅2 = 0.99607 with MAE = 0.00676 for 𝐹𝑜= 0.0095 and 𝑅2 = 0.98484 with MAE = 0.01585 for 𝐹𝑜= 0.012.The furthest forward target remains comparable to the lower-regime results.

5.3. Extrapolation: PINN

The Modified PINN substantially improves extrapolation at both sides of the training range, with strong performance across lower and upper Fourier-number targets validated against the analytical solution.

  • 5.3.2. PINN Extrapolation at Target Fourier Numbers: R^2 = 0.99973 with MAE = 0.00099 at Fo = 0.0009, improving from the unmodified PINN's R^2 = 0.60400 with MAE = 0.05880.At Fo = 0.0091, the Modified PINN also achieves R^2 = 0.99995 with MAE = 0.00106.
  • 5.3.3. PINN Extrapolation Across Fourier Numbers: R^2 = 0.95917 with MAE = 0.00717 at Fo = 0.00075, while the nearer target Fo = 0.00085 reaches R^2 = 0.99903 with MAE = 0.00159.Accuracy softens gradually for targets deeper in the early-time regime.
  • 5.3.3. PINN Extrapolation Across Fourier Numbers: Unlike the BiLSTM, the PINN predicts directly at target Fourier numbers, so lower-regime degradation reflects target difficulty rather than accumulated stepwise prediction error.This provides a distinct error-growth mechanism for comparison with recurrent marching.
  • 5.3.3. PINN Extrapolation Across Fourier Numbers: The Modified PINN is validated across eight upper-regime targets spanning Fo = 0.00910–0.01200, in addition to lower-regime and target-level tests.These tests cover both sides of the training window against ground truth.

5.4. Extrapolation Error Growth

Physics-guided BiLSTM extrapolation remains accurate farther forward than backward, reflecting diffusion's smoothing in forward time and the difficulty of reconstructing sharper earlier profiles.

  • 5.4. Extrapolation Error Growth: Progressively extending predictions one validated step at a time provides a more demanding test of error growth than evaluating only isolated targets.Figures 15 and 16 implement this forward and backward extension from the training interval.
  • 5.4. Extrapolation Error Growth: R^2 decreases only from 0.9984 to 0.9905 as forward targets extend to Fo = 0.01110, roughly 23% beyond the training boundary.Predicted curves remain almost indistinguishable from the analytical solution across the forward sweep.
  • 5.4. Extrapolation Error Growth: R^2 falls from 0.9929 to 0.9870 by Fo = 0.00080, then reaches 0.9639 at Fo = 0.00060, about 40% outside the training domain.The gap near the heated boundary increases as the backward target approaches the initial singularity.
  • 5.4. Extrapolation Error Growth: Backward extrapolation is harder because it must reverse diffusion's smoothing and infer information erased from later, smoother temperature fields.The relaxation-marching scheme limits error accumulation but does not remove the underlying reconstruction difficulty.

5.5. Direct Framework Comparison

Both frameworks pass the extrapolation-validation test, but they differ in prediction mechanism, data requirements, and computational cost, leading to different deployment advantages.

  • 5.5. Direct Framework Comparison: Both PG-BiLSTM and Modified PINN achieve accurate, physically consistent extrapolation with predictions validated directly against analytical ground truth.The two methods use sequential data-driven and residual-based, largely data-free mechanisms, respectively.
  • 5.5. Direct Framework Comparison: Figures 15 and 16 evaluate Physics-guided BiLSTM predictions at four increasingly distant forward and backward targets.These sweeps complement the direct framework comparison with progressive extrapolation tests.
  • 5.5. Direct Framework Comparison: PG-BiLSTM marches one Fourier number at a time using prior predictions, whereas Modified PINN predicts directly by enforcing the heat equation at collocation points.The BiLSTM is suited to targets near available labelled data, while the PINN does not require labelled data at the target.
  • 5.5. Direct Framework Comparison: Despite 422,801 versus 18,541 parameters, the BiLSTM trains roughly 2.7× faster: 45 min versus 120 min on the same GPU.The difference is attributed to PINN PDE-residual gradient computation during training.

5.6. Engineering Applications

The frameworks support extrapolation in data-limited thermal systems, with PG-BiLSTM suited to nearby labelled histories and Modified PINN suited to sparse or unavailable instrumentation.

  • 5.6. Engineering Applications: The common engineering need is trustworthy prediction outside the range of available data, addressed here through direct validation against ground truth.The examples remain within data-limited thermal-system applications.
  • 5.6. Engineering Applications: PG-BiLSTM suits inverse heat conduction and short-horizon thermal control because it reconstructs or forecasts fields through validated step-by-step marching.Examples include recovering earlier temperature histories and predicting electronics-cooling hotspots before new sensor data arrive.
  • 5.6. Engineering Applications: Modified PINN suits settings with sparse or no instrumentation, including melt-pool regions in additive manufacturing and transient aerodynamic or fire loading.It can estimate fields without labelled data at the target by satisfying the governing equation.
  • 5.6. Engineering Applications: Both architectures achieve accurate, physically consistent extrapolation beyond the training domain with every prediction checked against analytical ground truth.This validation supports their potential use when operating conditions exceed available data.

6. Conclusions

The study demonstrates that extrapolation reliability can be engineered and quantitatively checked through physics-guided corrections and a validated benchmark methodology. Its broader contribution is a transferable train-predict-validate-extend framework for settings where data are limited and extrapolation is unavoidable.

  • The framework demonstrates that extrapolation reliability can be engineered and checked rather than assumed, using a controlled benchmark with exact ground truth.The heat-conduction problem serves as a testbed for validating extrapolation methodology, not as the study’s sole object.
  • Physics-guided corrections address distinct extrapolation weaknesses in the BiLSTM and PINN, including coordinate treatment, singularity handling, boundary enforcement, and backward-marching stability.The corrections are tied to specific features of the governing equation and weaknesses exposed during validation.
  • The corrected PINN passes ground-truth validation throughout the tested range, particularly at later times where labelled data are scarce.
  • The transferable methodology recommends proving extrapolation behaviour on an analytically or numerically verifiable benchmark before trusting models where verification is unavailable.The demonstrated procedure sequentially trains, predicts, validates, and extends the prediction horizon.
  • Future work could extend the methodology to multidimensional, nonlinear, and other transient systems, including settings without exact ground truth.These extensions remain prospective and are identified as future work.

CRediT Authorship Contribution Statement

The contribution statement assigns methodological, analytical, software, writing, supervision, and project-management roles across the listed authors.

  • Ashutosh Yadav contributed methodology, investigation, validation, data curation, formal analysis, software, writing, and visualization.
  • Alok Dubey contributed methodology and manuscript review and editing.
  • Ranjan Chakraborty contributed conceptualization, resources, manuscript review and editing, supervision, and project administration.
  • Harshal Deepak Akolekar contributed conceptualization, software, formal analysis, resources, manuscript review and editing, and supervision.

Declaration of Generative AI and AI-Assisted Technologies

The authors disclose the use of generative AI and AI-assisted technologies to improve manuscript grammar and readability, while retaining responsibility for the scientific content.

  • Generative AI and AI-assisted technologies were used to improve the manuscript’s grammar and readability.
  • The authors retain responsibility for all scientific content.
Loading 2609.09912v1…