PaperlayerView exact revision on arXiv

Source-linked AI summary

Meta-modeling game for deriving theoretical-consistent, micro-structural-based traction-separation laws via deep reinforcement learning

Kun Wang, WaiChing Sun

arXiv:1810.10535v1cs.LGphysics.comp-phphysics.geo-phstat.ML

TL;DR

The paper addresses the difficulty of selecting and validating surrogate-model information and configurations for traction–separation predictions. It proposes a deep-reinforcement-learning framework that constructs constitutive laws through self-play, and concludes that the approach can automate model generation and support discovery of subtle physical couplings.

  • Problem

    Deriving, verifying, and validating surrogate constitutive laws that capture essential information with sufficient accuracy and robustness is difficult and time-consuming.

    Source: §1 · p. 2

  • Method

    The framework uses deep reinforcement learning to explore constitutive-law modeling choices under a defined objective and universal mechanical principles.

    Source: §1 · p. 2

  • Results

    The paper presents an agenda-based modeling game in which reinforcement learning searches for ways to generate constitutive laws under stated physical rules.

    Source: §7 · p. 19

  • Takeaways & Limitations

    The approach may let modelers focus less on curve-fitting and more on finding actions that lead to predictive models.

    Source: §7 · p. 20

    The model uses a validation metric for scores and rewards but does not yet handle noise or uncertainty.

    Source: §7 · p. 22

Abstract

from arXiv · show

This paper presents a new meta-modeling framework to employ deep reinforcement learning (DRL) to generate mechanical constitutive models for interfaces. The constitutive models are conceptualized as information flow in directed graphs. The process of writing constitutive models are simplified as a sequence of forming graph edges with the goal of maximizing the model score (a function of accuracy, robustness and forward prediction quality). Thus meta-modeling can be formulated as a Markov decision process with well-defined states, actions, rules, objective functions, and rewards. By using neural networks to estimate policies and state values, the computer agent is able to efficiently self-improve the constitutive model it generated through self-playing, in the same way AlphaGo Zero (the algorithm that outplayed the world champion in the game of Go)improves its gameplay. Our numerical examples show that this automated meta-modeling framework not only produces models which outperform existing cohesive models on benchmark traction-separation data but is also capable of detecting hidden mechanisms among micro-structural features and incorporating them in constitutive models to improve the forward prediction accuracy, which are difficult tasks to do manually.

1 Introduction

The introduction motivates automating traction–separation model construction because current surrogate and data-driven approaches still require difficult, time-consuming choices about information and model configuration. It proposes deep reinforcement learning to explore constitutive-modeling choices under defined objectives and mechanical rules.

1 Introduction

  • Mixed-mode traction–separation predictions matter across applications, while many earlier cohesive-zone models focused on mode I.The cited applications include impact damage and seismic-event prediction.Source: §1 · p. 1
  • Computational homogenization can capture heterogeneous microstructures, but its many required representative-element simulations increase computational demand.Surrogate models are often derived to replicate homogenized responses, yet deriving, verifying, and validating accurate, robust surrogates remains difficult and time-consuming.Source: §1 · p. 2
  • Existing data-driven approaches still leave time-consuming trial and error in choosing inputs and information-flow configurations that improve traction prediction.This includes selecting sub-scale information beyond displacement-jump history for recurrent neural-network models.Source: §1 · p. 2
  • The proposed approach frames constitutive-law generation as deep-reinforcement-learning self-play, using neural networks to estimate rewards and update action probabilities and value functions.The objective can be defined for a task, while mechanical principles constrain model generation and validation supplies the environment in which scores are sampled.Source: §1 · pp. 2–3
  • Deep reinforcement learning can explore modeling choices whose rewards are difficult to evaluate without comparing predictions against benchmark data.The authors identify reward approximation as crucial because evaluating every possible configuration is infeasible for complex traction–separation models.Source: §1 · p. 3
  • Once established, the framework can generate models without human intervention, evaluate them during model writing, and incorporate expert models into its action space.The authors also propose it as a tool for comparing existing models against an objective and for discovering subtle physical coupling mechanisms.Source: §1 · pp. 3–4

2 Representing traction-separation law in directed graph

The section represents traction-separation models as directed graphs of information flow, with displacement jump as the root and traction as the leaf. It distinguishes fixed mechanics-based edges from phenomenological relations and constrains which graphs are admissible.

2 Representing traction-separation law in directed graph

  • A constitutive model is represented as a directed graph linking physical quantities, with displacement jump δ as the root and traction t as the leaf.Edges encode information flow used to generate traction-separation predictions.Source: §2 · pp. 4, 6
  • Admissible graphs have δ as their only root, t as their only leaf, no cycles, and every connected internal vertex lies on a path from δ to t.Isolated internal vertices are permitted and need not contribute to the final model.Source: §2 · p. 4
  • Earlier prescribed linkages required prior knowledge of relationships among physical quantities, whereas this graph formulation can search connections among a given set of vertices.Finding hidden vertices and appropriate edges is outside this paper’s modeled actions and is left for future work.Source: §2 · pp. 4–5
  • For the demonstrated law, traction depends on displacement-jump history over a cohesive or cohesive-frictional surface and on a collection of state variables.The graph treatment is presented for traction-separation laws, though the authors note possible extensions to other constitutive relations.Source: §2 · p. 5
  • The model uses manually selected internal variables, including porosity, coordination number, fabric measures, and contact-network connectivity measures, to predict traction.The candidate set includes degree assortativity, average shortest path length, and graph density.Source: §2 · p. 5
  • Definition edges encode universal mechanics principles and remain fixed, while phenomenological edges express material relations that can be chosen to fit data.The latter may use linear, quadratic, exponential, or artificial-neural-network forms.Source: §2 · p. 6

3 Score system for model evaluation and objective function

The score system evaluates model quality through performance measures and critical admissibility features. It separately assesses calibration and blind-prediction accuracy, and checks whether their error distributions are consistent.

3 Score system for model evaluation and objective function

  • The model score is a positive real-valued function in [0, 1] of performance measures and critical features, including accuracy, computation speed, frame indifference, and thermodynamic consistency.Performance measures have associated weights; critical features enforce model constraints.Source: §3 · p. 7

3.1 Accuracy of calibrations and forward predictions

  • Calibration and blind-prediction errors are evaluated against target data using MSE, with feature scaling applied to target and model outputs.Training and test datasets are mutually exclusive; scaled feature values are compared sample by sample.Source: §3.1 · p. 7
  • Accuracy is summarized from the MSE distribution across the entire, training, or test dataset, using a selected percentile and user-chosen critical MSE threshold.A model is considered satisfactorily accurate when the selected percentile MSE is no greater than εcrit.Source: §3.1 · pp. 7–8

3.2 Consistency of accuracy between calibrations and forward predictions

  • Prediction consistency is assessed with a K-sample Anderson-Darling goodness-of-fit test of whether training and test MSEs come from the same distribution.The resulting binary consistency measure is based on whether the test p-value meets the chosen significance level.Source: §3.2 · p. 8

4 Game of the traction-separation law

The traction-separation modeling process is represented as a game in which players build admissible directed graphs by selecting edges. Completed graphs are scored, and their information-flow paths are instantiated as neural-network mappings for model calibration and prediction.

4 Game of the traction-separation law

  • Each game begins with a node graph and proceeds through single-edge activations until a complete admissible digraph is formed, after which its model score is evaluated.The player may be a human or an AI agent.Source: §4 · p. 8
  • The final reward compares a game's model score with the average score of previously played games, while earlier action rewards and move probabilities are estimated.A game above the average wins; one that does not is assigned a loss.Source: §4 · p. 9
  • The example game represents a state as binary edge statuses and an action as the next edge to activate; legal moves exclude activated or oppositely directed edges and must preserve graph admissibility.The example has 13 candidate edges, with the game states and legal actions illustrated in the gameplay sequence.Source: §4 · pp. 9–10, Figure 4 · p. 11
  • A completed digraph is evaluated by identifying paths from δ to t and constructing mappings between predecessor-successor nodes, which are calibrated on training data and executed along the paths for predictions.The numerical examples use GRU networks with two 32-neuron hidden layers and a linear dense output layer.Source: §4 · pp. 10–11

5 Deep reinforcement learning for generating constitutive laws

The agent generates constitutive laws by playing a game in which graph actions build models, guided by deep reinforcement learning. Neural-network policy and value estimates, combined with Monte Carlo Tree Search, guide self-play and are updated from model rewards without human guidance.

5 Deep reinforcement learning for generating constitutive laws

  • The agent uses self-play to improve constitutive models from game rewards, without human guidance or a human-provided starter database.The learning can begin even when initial digraphs make little sense for a traction-separation model.Source: §5 · pp. 11–12
  • A neural network maps the current constitutive-law graph to action probabilities and an estimated expected score, guiding gameplay.Source: §5 · p. 12
  • Monte Carlo Tree Search selects actions using an upper confidence bound, expands a new leaf, and evaluates it with the policy/value network.Source: §5 · p. 12
  • MCTS returns root-state search probabilities to guide the next self-play action, with temperature controlling exploration.Source: §5 · pp. 12–13, Figure 6 · p. 13
  • Training uses self-play states, search policies, and final game rewards, combining reward mean-squared error with policy cross-entropy.Self-play and network training repeat until the generated graph’s score stops improving.Source: §5 · p. 13

6 Numerical Experiments and Applications

The experiments test whether deep reinforcement learning can discover useful graph relationships among microstructural quantities and whether its generated traction-separation model can serve in multiscale finite-element simulations. The reported results show learning across increasingly complex model spaces, improved predictions from selected microstructural information, and agreement between the finite-element model and DEM-based simulations.

6.1 Numerical Experiment 1: Determining optimal physical relationships for traction-separation laws

  • Experiment 1 lets the agent discover graph edges among porosity, coordination number, and fabric tensor without manually specifying their relations.Its score combines accuracy, prediction, and consistency terms; calibration and forward prediction use 50 and 150 loading cases, respectively.Source: §6.1 · p. 14
  • After five DRL iterations, model scores concentrate around the maximum, and the final low-exploration gameplay shows strong performance.The score distribution shifts upward and narrows over iterations; the agent continues some exploration to avoid local-maximum convergence.Source: §6.1 · p. 15, Figure 7 · p. 15

6.2 Numerical Experiment 2: Data-driven discovery for enhancement of traction-separation laws

  • Experiment 2 tests whether the agent can incorporate observed microstructural insights into predictions without manually rewriting a constitutive law.The gameplay statistics are reported across five separate DRL runs, with score distributions and means shown by iteration.Source: §6.2 · p. 17, Figure 11 · p. 19
  • Experiment 2 expands the graph action space from 13 to 71 and admissible configurations from 591 to over 20,000, using 50 training and 150 test loading cases.The added fabric and grain-connectivity measures are learned from training data and evaluated on the test database.Source: §6.2 · p. 17
  • 0.191 and 0.915 are the model scores for the first and fourth digraphs, respectively, despite incorporating the same types of microstructural information.The agent selected the strong fabric tensor over the fabric tensor, and the augmented model gave more accurate representations of granular materials than Numerical Experiment 1.Source: Figure 8 · p. 16, §6.2 · p. 18

6.3 Application: Multiscale bridging using DRL-generated traction-separation model in finite element modeling

  • The finite-element application uses a mixed-mode tension-shear test with loading, unloading, and reloading on a granular specimen containing an embedded interface.The optimal model from Numerical Experiment 2 is used at the interface to replace computationally expensive DEM RVEs.Source: §6.3 · p. 18
  • The FEM-DRL and FEM-DEM global traction-displacement curves show great agreement in normal and shear directions.The DRL-generated model is applied as a surrogate for microscale DEM RVEs in multiscale FEM simulations.Source: §6.3 · pp. 18–19

7 Conclusion

The paper frames traction-separation model creation as a meta-modeling game in which deep reinforcement learning explores modeling options through self-practice. It highlights automated model improvement and validation, reuse of expert models, and higher data demands and unaddressed noise and uncertainty.

7 Conclusion

  • The meta-modeling approach lets a computer explore modeling options through self-practice rather than requiring a particular traction-separation law to be explicitly written.The framework uses reinforcement learning to make modeling decisions under stated rules.Source: §7 · p. 19
  • Automated self-play can generate and improve models for different purposes or demands, including speed, accuracy, and robustness.The paper presents this flexibility as a benefit of its automated model-creation game.Source: §7 · p. 20
  • Using validation as the reward mechanism means resultant models are validated at the end of each game.The paper also describes validation as part of the model-writing process.Source: §7 · pp. 20–21
  • Directed-graph representations allow expert-established models to be embedded in the action space, while the framework is described as reusable across different data, objectives, and rules.The paper says this can expand action spaces efficiently and shorten training time.Source: §7 · p. 21
  • The approach may require more data than conventional modeling, especially with more sophisticated validation metrics, and it does not yet handle noise or uncertainty.The paper places noise and uncertainty handling beyond the scope of this study.Source: §7 · pp. 21–22

Appendix A: Generation of synthetic data from discrete element modeling (DEM)

The appendix describes how discrete-element simulations of a granular representative volume element generated traction-separation data and microstructural information. Of 200 simulations, 50 were used for calibration and 150 for blind prediction evaluation.

Appendix A: Generation of synthetic data from discrete element modeling (DEM)

  • The DEM used particles with radii between 1 ± 0.3 mm in an RVE 20 mm high, initially consolidated to an isotropic pressure of 10 MPa.YADE was used for the simulations, with a Cundall elastic-frictional contact model for inter-particle behavior.Source: Appendix A · p. 23
  • 200 numerical experiments simulated a granular representative volume element on a frictional surface under different normal and tangential loading rates and loading-unloading-reloading paths.The RVE was displacement-controlled in the normal and tangential directions.Source: Appendix A · p. 23, Figure 17 · p. 24
  • The simulations recorded traction-separation curves and microstructural quantities including porosity, coordination number, and fabric tensor.Examples of normal and tangential traction-displacement curves are shown for three loading paths.Source: Appendix A · p. 23, Figure 18 · p. 24
  • The 200 simulations were shuffled, with the first 50 used as training data for calibrating graph-edge parameters and the remaining 150 reserved for blind prediction testing.The test simulations were used only to evaluate blind prediction accuracy.Source: Appendix A · pp. 23–24
Loading 1810.10535v1…