PaperlayerView exact revision on arXiv

Source-linked AI summary

On architectural choices for interpretability and thermodynamic consistency in Physically Recurrent Neural Networks in the low-data regime

M. A. Maia, K. A. Meyer, A. M. C. M. van Gils, I. B. C. M. Rocha, F. P. van der Meer

arXiv:2610.04067v1cs.LGcs.NAmath.NA

TL;DR

History-dependent surrogate models must balance generalization and interpretability, and generating training data can be costly. The paper studies PRNN decoder constraints and related architectural choices to improve data efficiency and interpretability. It reports alignment between fictitious stresses and microscopic states, multi-task use of microscopic information, and an architecture satisfying energetic consistency and non-negative dissipation.

  • Problem

    Purely data-driven surrogates can sacrifice generalization and interpretability, while history-dependent materials require memory beyond current strains; generating sufficiently large training datasets can also be costly.

    Source: §1 · pp. 1, 3

  • Method

    The paper studies linear PRNN decoder architectures, emphasizing weight normalization, and examines an adjoint encoder-decoder design with positive scalar contributions.

    Source: §1 · p. 3

  • Results

    Architectural choices aligned fictitious stresses with microscopic states without explicit training on them, while an adjoint encoder-decoder with positive scalar contributions satisfied Hill-Mandel consistency and non-negative dissipation.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The authors conclude that architectural choices can improve data efficiency, provide insight into the latent space, and enable recovery of microscopic information.

    Source: §1 · p. 3

    The fictitious-stress correspondence has only been investigated for the material settings studied, and may not hold for more intricate micromodels.

    Source: §5.1 · p. 16, §7 · p. 25

Abstract

from arXiv · show

In this paper, we unravel the effect of different decoder architectures on the interpretability of the latent space of the Physically Recurrent Neural Network. Particular emphasis is given to a new weight normalization constraint, which acts as a regularization technique and enables robust training in the low-data regime. A brief visual exploration illustrates how these changes impact the latent space and how the fictitious stress can align with the true state of the RVE without explicit training. Reaping the benefits of a meaningful latent space, a case study illustrates how information from the microscopic level can be retrieved and incorporated into a multi-task approach that does not require extra parameters or larger training sets. Another key contribution shows that a specific architectural choice can naturally lead to a thermodynamically consistent formulation. By enforcing an adjoint encoder-decoder structure with positive scalar contributions, this modification ensures energy consistency across scales and non-negative dissipation, leading to even lower training requirements. This alternative completes the study on interpretability, inductive bias, and thermodynamic consistency, and demonstrates that data efficiency can be improved with careful architectural choices rooted in the underlying physics.

1. Introduction

History-dependent multiscale surrogates face a tension between computational efficiency, generalization, and interpretability, while recurrent models can have latent states that are difficult to interpret. This study examines how PRNN architecture can improve data efficiency, connect latent quantities to micromechanics, and encode thermodynamic consistency.

1. Introduction

  • Full-order FE2 simulations become intractable for large, time-dependent analyses because they solve numerous local finite-element problems at each time increment.FE2 exchanges information between macro and micro scales, using each micromodel’s homogenized response in place of an explicit macroscale constitutive model.Source: §1 · p. 1
  • Purely data-driven surrogates can sacrifice generalization and interpretability, while history-dependent materials require memory beyond current strains.Recurrent networks provide evolving states, but their hidden states are not as straightforward to interpret as micromodel states; prior methods also have reported limitations in extrapolation and material properties.Source: §1 · pp. 1–2
  • The PRNN combines an encoder-decoder with embedded constitutive models, giving latent quantities a prescribed physical interpretation and updating internal variables through the material models.The encoder’s strain-like values feed fictitious material points; their stresses pass to the decoder, while internal variables provide memory.Source: §1 · p. 3
  • The study restricts the decoder to linear forms and emphasizes weight normalization, which acts as regularization and enables robust training in the low-data regime.The material layer contains elastoplastic models to simplify model selection and visualization of the architectural effects.Source: §1 · p. 3
  • Visual analysis finds that architectural choices can align fictitious stresses with the micromodel’s true state without training on those quantities, enabling microscopic information in a multi-task formulation without extra parameters or larger datasets.The low-data setting matters because producing training data can be costly in computational homogenization and experimental campaigns.Source: §1 · p. 3
  • An adjoint encoder-decoder with positive scalar contributions satisfies Hill-Mandel energetic consistency across scales and the non-negative dissipation constraint.The paper presents these architectural choices as a route to improved data efficiency, latent-space insight, and recovery of microscopic information.Source: §1 · p. 3

2. Computational homogenization

The section introduces FE2 homogenization and its PRNN surrogate, which retains constitutive models while replacing the microscopic boundary-value problem with learned components. It describes the material models and architectural choices used to pursue interpretability and thermodynamic consistency.

2.1. Concurrent multiscale model

  • FE2 downscales macroscopic strain to RVE boundary conditions, computes microscopic response, and recovers homogenized stress, but nested microscale boundary-value problems make it computationally costly.Periodic boundary conditions are used for the RVE.Source: §2.1 · pp. 4–5

2.2. Constitutive models

  • The composite RVE pairs an elastoplastic matrix with elastic fibers; the matrix model uses associative plastic flow and nonlinear saturating isotropic hardening.The constitutive models have symmetric positive definite elastic stiffness and tangent matrices under the stated model assumptions.Source: §2.2 · p. 5

2.3. Replacing the micromodel with an approximate model

  • PRNNs place intact constitutive models between an encoder and decoder, using fictitious material-point strains to compute stresses and internal variables before decoding homogenized stress.This preserves complex time- and path-dependent constitutive behavior in the material layer.Source: §2.3 · p. 6
  • The decoder combines fictitious-point stresses into homogenized stress with positive weights, while this study restricts encoder and decoder to linear relations for low-data architectural variations.Positive decoder weights are imposed by taking the absolute values of unconstrained weights during optimization and evaluation.Source: §2.3 · p. 6

3. Architectural choices for interpretability and thermodynamical consistency

The section compares decoder connectivities for interpretability and flexibility, then derives an adjoint encoder-decoder architecture intended to enforce thermodynamic consistency. A sum-to-one weight normalization makes decoder weights interpretable as relative contributions without adding a hyperparameter.

3.1. Towards sparsity

  • Dense decoding lets cross-component stresses contribute to macroscopic stress, unlike full-order homogenization, motivating Component and Point designs with progressively greater sparsity.Component removes cross-component connections; Point also shares one weight across a material point’s stress components, making it least flexible and most interpretable.Source: §3.1 · p. 7, Figure 3 · p. 7
  • The sum-to-one normalization makes decoder weights relative contributions analogous to volume-averaging weights, without requiring an additional hyperparameter.The constraint applies to weights linking each macroscopic stress component to contributing fictitious-point stresses.Source: §3.1 · p. 7

3.2. Thermodynamic consistency

  • The TC design derives macroscopic stress from local energies using an adjoint encoder-decoder structure, and has the same trainable-parameter count as Point.Positive weights and the adjoint structure support energy consistency; the encoder may take any form.Source: §3.2 · pp. 8–9
  • When each local tangent is SPD and weights are positive, the adjoint arrangement ensures the network’s macroscopic tangent is also SPD.This property follows from the chosen constitutive models in the examples discussed.Source: §3.2 · p. 10
  • Only the thermodynamically consistent (TC) architecture satisfies both energetic consistency and non-negative dissipation; Dense and Component do not guarantee either property.Point and TC architectures admit a consistent weighted sum for macroscopic dissipation, but only TC guarantees both properties.Source: §3.2 · pp. 9–10

4. Accuracy under severe data scarcity

Decoder sparsity and weight normalization improve PRNN performance under data scarcity, while the thermodynamically consistent (TC) architecture performs especially well with one training curve. Normalization can help generalization but may overconstrain the point decoder.

4. Accuracy under severe data scarcity

  • The study tests 2–30 fictitious material points across one to five short training paths, and keeps normalized decoder weights summing to one throughout training.Unnormalized decoder weights are initialized to sum to one but remain free to change during training.Source: §4 · pp. 10–11

4.1. Unnormalized decoders

  • Sparse decoders generally outperform Dense in the low-data regime, reaching the 10% error mark with four training curves for smaller networks.With five curves, the four decoder types converge toward similar accuracy; training on one or two curves remains challenging overall.Source: Figure 6 · p. 11, §4.1 · p. 12
  • TC networks trained on one curve predict accurately, improve training and validation together, and produce smooth homogenized stresses across the loading space.Unlike Dense networks, TC networks show no signs of overfitting under severe data scarcity.Source: §4.1 · p. 12, Figure 7 · p. 12, Figure 8 · p. 13

4.2. Normalized decoders

  • Normalized decoders generally outperform their unnormalized counterparts, with errors below 10% using as few as two curves and reduced error variance.The sum-of-weights constraint limits any single material point's influence, stabilizing outputs across differently sampled training sets.Source: §4.2 · pp. 12–13
  • Normalized point networks reach 13% error with one curve but plateau around 12% after three to four curves, indicating underfitting from overconstraint.Other normalized alternatives achieve errors below 5% as training data increase.Source: §4.2 · p. 13
  • Both TC and Normalized TC achieve relative errors below 10% after training on a single curve.The adjoint encoder-decoder structure with material-point scaling provides the best overall performance across nearly all tested training-set and material-layer combinations.Source: §4.2 · pp. 13–14

5. Comparison of latent space variables with microscopic fields

The study compares fictitious PRNN stresses and internal variables with microscopic matrix fields. Normalized component decoders provide the closest stress-distribution match, while homogenization accuracy and latent-space interpretability trade off.

5.1. Fictitious vs microscopic stresses

  • Normalized component decoders best match microscopic matrix stress distributions, with mean-stress errors as low as 10% and maximum-stress errors below 20% in the best case.The maximum-stress improvement occurs for larger networks; other normalized architectures do not show the same marked improvement for that measure.Source: §5.1 · pp. 15–16
  • TC and Normalized TC produce accurate homogenized stresses but do not closely match microscopic stress distributions, revealing a trade-off between accuracy and interpretability.Sparse architectures offer an alternative when both accurate homogenization and an interpretable latent space are sought.Source: §5.1 · p. 16
  • Fictitious stresses are closer to the matrix phase than to the full micromodel in this setup, where matrix behavior dominates.For micromodels with richer constitutive behavior or multiple phases, that correspondence may not hold and requires examination.Source: §5.1 · p. 16

5.2. Fictitious vs microscopic internal variables

  • Fictitious internal-variable agreement with microscopic values is poorer than stress agreement, even with normalized decoders and tailored sparsity.The PRNN is trained on homogenized stress snapshots and embeds only the matrix constitutive model, omitting richer RVE details.Source: §5.2 · pp. 16–17

6. Exploiting latent space interpretability for multitasking

The paper uses interpretable fictitious stresses to estimate maximum microscopic hydrostatic stress and evaluates joint and sequential multitask learning. Joint learning balances the homogenized-stress and maximum-stress tasks, with performance depending on network size.

6.1. Incorporating maximum hydrostatic stress data

  • The added Maximum task uses hydrostatic stresses computed at fictitious material points and selects their maximum at each time step.The approach adds supervision for microscopic maximum hydrostatic stress alongside the homogenized-stress task.Source: §6 · p. 20, §6.1 · p. 20

6. Exploiting latent space interpretability for multitasking

  • Without explicit maximum-stress training, normalized component PRNNs show correlation with microscopic maxima, but test errors remain around 30% with two curves and about 24% on average with four to five.The best performance is approximately 16%, which the authors judge insufficient for practical use.Source: §6 · p. 20, Figure 16 · p. 21

6.1. Incorporating maximum hydrostatic stress data

  • Joint learning is preferred over sequential learning across both tasks: it slightly improves Maximum-task errors and significantly lowers Average-task errors.In the sequential approach, only the decoder is trainable in the second stage, limiting network expressivity.Source: §6.1 · pp. 21–22, Figure 17 · p. 21

6.2. Double-task vs single-task PRNNs

  • Double-task learning trades some Average-task accuracy for better Maximum-task predictions, especially in smaller networks; the trade-off diminishes once networks are large enough for both tasks.For 10 material points and five curves, total error can be as low as 6 MPa, corresponding to 7–10% relative error on each task.Source: §6.2 · p. 22, Figure 19 · p. 23
  • The fictitious material point supplying the maximum changes along a loading path, consistent with the evolving location of the maximum in the micromodel.Larger material layers provide more candidates to represent the maximum hydrostatic stress.Source: §6.2 · pp. 22–23

7. Final remarks

The paper examines how decoder architecture shapes PRNN interpretability and accuracy in the low-data regime. Weight normalization can improve robustness, while an adjoint encoder-decoder architecture adds thermodynamic guarantees and can achieve low stress-prediction errors with one training curve.

7. Final remarks

  • Weight normalization can improve decoder interpretability, robustness, and accuracy, though its effect depends on architecture and it may restrict flexibility.It acts as regularization without additional hyperparameter tuning; the observed stress correspondence is limited to the material settings studied and should be verified for each material and architecture.Source: §7 · pp. 23, 25
  • Decoder weights can be interpreted as relative volume contributions from fictitious material points, and fictitious stresses can reflect RVE counterparts without explicit training for that purpose.Sparse decoders make individual point contributions easier to disentangle and visualize.Source: §7 · pp. 23, 25
  • With one training curve, the adjoint encoder-decoder architecture achieved homogenized stress prediction errors consistently below 10%.Positive scalar contributions enforce energetic consistency between micro- and macroscopic scales and non-negative dissipation.Source: §7 · p. 25
  • The thermodynamically consistent formulation inherits embedded material-model properties and satisfies the Hill-Mandel principle, but fictitious and true microscopic distributions did not agree.This separates thermodynamic consistency from correspondence with the microscopic state.Source: §7 · p. 25

Appendix A. Matrix vs full micromodel

For networks trained on five curves, fictitious-stress relative errors increased with model complexity and plateaued around 10%; the match was best for matrix-phase quantities.

Appendix A. Matrix vs full micromodel

  • The match with matrix-phase quantities is best overall compared with quantities from the full micromodel.The figure compares mean and maximum stresses over the matrix phase and full micromodel.Source: Appendix A · p. 27, Figure A.21 · p. 28
  • The relative-error comparison uses fictitious material-point stresses against true microscopic stresses for networks trained on five curves.It reports both mean- and maximum-stress errors across the two domains.Source: Appendix A · p. 27, Figure A.21 · p. 28
  • For both mean- and maximum-stress metrics, fictitious-to-true stress relative error grows with network complexity and plateaus around 10%.The comparison covers PRNNs with different decoders trained on five curves.Source: Appendix A · p. 27, Figure A.21 · p. 28
Loading 2610.04067v1…