PaperlayerView exact revision on arXiv

Source-linked AI summary

Solving differential equations with neural networks: Applications to the calculation of cosmological phase transitions

Maria Laura Piscopo, Michael Spannowsky, Philip Waite

arXiv:1902.05563v2hep-phhep-th

TL;DR

Many differential equations lack analytic solutions, motivating numerical approaches; this paper uses neural networks whose outputs directly represent equation solutions. The approach achieves competitive accuracy on tunnelling profiles, including examples where dedicated solvers fail.

  • Problem

    Many differential equations have no analytic solution, making numerical or approximate methods necessary.

    Source: INTRODUCTION · p. 1

  • Method

    The neural network output represents the solution, with boundary conditions included as terms in the loss rather than imposed through a fixed trial solution.

    Source: INTRODUCTION · p. 2

  • Results

    The neural-network method achieved accuracy competitive with CosmoTransitions and BubbleProfiler and found correct solutions in examples where either dedicated solver failed.

    Source: CONCLUSIONS · p. 10

  • Takeaways & Limitations

    The approach was applied to ordinary, coupled, and partial differential equations, and the authors present it as a way to obtain fully differentiable solutions.

    Source: CONCLUSIONS · p. 10

    The phase-transition setup assumes that the field configuration is invariant under three-dimensional Euclidean rotations.

    Source: CALCULATION OF PHASE TRANSITIONS… · p. 5

Abstract

from arXiv · show

Starting from the observation that artificial neural networks are uniquely suited to solving optimisation problems, and most physics problems can be cast as an optimisation task, we introduce a novel way of finding a numerical solution to wide classes of differential equations. We find our approach to be very flexible and stable without relying on trial solutions, and applicable to ordinary, partial and coupled differential equations. We apply our method to the calculation of tunnelling profiles for cosmological phase transitions, which is a problem of relevance for baryogenesis and stochastic gravitational wave spectra. Comparing our solutions with publicly available codes which use numerical methods optimised for the calculation of tunnelling profiles, we find our approach to provide at least as accurate results as these dedicated differential equation solvers, and for some parameter choices even more accurate and reliable solutions. In particular, we compare the neural network approach with two publicly available profile solvers, \texttt{CosmoTransitions} and \texttt{BubbleProfiler}, and give explicit examples where the neural network approach finds the correct solution while dedicated solvers do not. We point out that this approach of using artificial neural networks to solve equations is viable for any problem that can be cast into the form $\mathcal{F}(\vec{x})=0$, and is thus applicable to various other problems in perturbative and non-perturbative quantum field theory.

INTRODUCTION

Because analytic solutions are unavailable for many differential equations, the paper proposes using neural networks to solve them directly. It applies this approach to tunnelling in electroweak phase transitions, relevant to baryogenesis and potential gravitational-wave signals.

INTRODUCTION

  • Many differential equations lack analytic solutions, so numerical or approximate methods are needed; the paper proposes neural networks as a different approach.The authors frame direct equation solving as a departure from neural-network applications that infer model parameters from data.Source: INTRODUCTION · p. 1
  • The paper applies direct neural-network equation solving to tunnelling in electroweak phase transitions, whose rates matter for baryogenesis and possible stochastic gravitational-wave signals.The cited motivations include baryogenesis and potential signals at future gravitational-wave experiments [26].Source: INTRODUCTION · pp. 1–2
  • The network output itself solves the differential equation, while boundary conditions enter as loss terms rather than being built into a fixed trial solution.The authors note that this makes the approach independent of choosing a boundary-condition-specific trial form.Source: INTRODUCTION · p. 2

Design of the network and optimisation

The method represents the unknown solution with a differentiable neural network and trains it by minimizing equation and boundary-condition violations. It also uses training and stability techniques to assess solutions and handle challenging domains.

Design of the network and optimisation

  • A single network with m outputs predicts solutions to m coupled equations, and its architecture and training domain are selected for the problem.The paper identifies layer and unit counts, training points, activation, optimizer, learning rate, and epochs as setup choices.Source: Design of the network and optimisation · pp. 2–3
  • The network output is identified with the approximate solution, avoiding the boundary-condition-specific trial-solution form used in earlier approaches [33].The paper notes that constructing such trial solutions can be complex for some boundary conditions, including phase transitions.Source: Design of the network and optimisation · p. 2
  • Training minimizes a loss formed from squared differential-equation residuals and boundary-condition terms, using derivatives of the network output and backpropagation.The boundary terms can encode Dirichlet, Neumann, or initial conditions.Source: Design of the network and optimisation · p. 2
  • The examples use Adam [38], a learning rate initially set to 0.01, 5×10^4 epochs, and full-batch training; a small single hidden layer is generally sufficient.The authors report that O(10) hidden units generally yield very accurate solutions.Source: Design of the network and optimisation · p. 3
  • The differential contribution to loss assesses equation satisfaction across the domain; iterative domain expansion can improve stability, and two-step training prevents the trivial all-false-vacuum solution.For rapidly varying solutions, the authors also report that self-adaptively adding anchors in those regions can be numerically beneficial.Source: Design of the network and optimisation · p. 3

Ordinary differential equation examples

The authors test the method on first- and second-order ordinary differential equations with known analytic solutions. The network predictions can be compared with those solutions, while the differential loss contribution offers a check across the domain.

Ordinary differential equation examples

  • The method is tested on first- and second-order ODEs with known analytic solutions and specified boundary conditions.Both examples are defined over x∈[0, 2].Source: Ordinary differential equation examples · p. 3
  • A 10-unit single-hidden-layer network trained on 100 points reproduces the analytic solutions, and accuracy can be improved by increasing epochs, anchors, or hidden units.Figure 1 compares the analytic and network solutions and plots their numerical difference.Source: Ordinary differential equation examples · pp. 3–4, Figure 1 · p. 3
  • The differential contribution to loss shows how much each training example contributes and can help assess equation satisfaction when no analytic solution is known.The authors say it can help distinguish a correct solution from a numerical instability that led to a local minimum.Source: Ordinary differential equation examples · p. 4

Coupled differential equation example

For coupled nonlinear equations, the paper uses iterative training on progressively larger domains to address numerical instability. In its analytic test case, this yields an accurate solution over the full domain, with the differential contribution serving as an equation-satisfaction indicator.

Coupled differential equation example

  • Small backpropagation instabilities can lead to local minima, so the method incrementally extends the training domain while retaining previously learned weights.The authors report that this can achieve numerical stability over arbitrarily large domains.Source: Coupled differential equation example · p. 4
  • In the test of a coupled nonlinear equation with a known analytic solution, the third training step produces an accurate solution over the entire domain.The domain is partitioned into three regions, and the figure shows the successive training steps.Source: Coupled differential equation example · p. 4, Figure 2 · p. 4
  • The differential contribution ˆF indicates whether the calculated solution satisfies the equation across the domain.Figure 2 displays this contribution for the equation for φ1.Source: Coupled differential equation example · p. 4, Figure 2 · p. 4

Partial differential equation example

The neural-network method is applied to a partial differential equation with a known analytic solution, demonstrating its flexibility beyond the ordinary differential equations used in the later physics examples. Its numerical solution is accurate across the domain, with a deeper network improving accuracy further.

Partial differential equation example

  • The method extends to partial differential equations using the same architecture as for ordinary differential equations, with an extra input unit for each additional variable.The PDE example is presented to showcase flexibility, not used in the later phase-transition calculations.Source: Partial differential equation example · p. 4
  • The example states boundary conditions at y = 0 and y = 1: φ(x, 0) = xe−x and φ(x, 1) = e−x(x + 1), without specifying an x-domain.Source: Partial differential equation example · p. 5
  • The NN solution has very good absolute and relative accuracy across the domain, and a deeper network improves accuracy by an order of magnitude.The comparison uses 100 training points on an evenly spaced 10 × 10 grid; the displayed difference is against the analytic solution.Source: Partial differential equation example · p. 5, Figure 3 · p. 5

CALCULATION OF PHASE TRANSITIONS DURING THE EARLY UNIVERSE

The section motivates finite-temperature phase transitions through their connection to electroweak baryogenesis and describes tunnelling from a metastable false vacuum to a true vacuum. It formulates the bubble profile as a solution to coupled field equations and relates the decay rate to the action.

CALCULATION OF PHASE TRANSITIONS DURING THE EARLY UNIVERSE

  • A strong first-order phase transition can provide departure from thermal equilibrium, a condition relevant to electroweak baryogenesis and the baryon asymmetry problem.Within the Standard Model, the other stated conditions—C- and CP-violation—require an extension.Source: CALCULATION OF PHASE TRANSITIONS… · p. 5
  • Tunnelling converts a homogeneous false-vacuum region into an approximate true-vacuum bubble, while the false vacuum persists far away.The false vacuum is a local minimum and decays through barrier penetration when the vacua are non-degenerate.Source: CALCULATION OF PHASE TRANSITIONS… · p. 5
  • At finite temperature, the four-dimensional Euclidean action is reduced to a three-dimensional action by replacing the Euclidean-time integration with a factor of T^-1.The relation is S4(φ) = T^-1 S3(φ).Source: CALCULATION OF PHASE TRANSITIONS… · p. 5
  • Assuming rotational invariance, the bubble profile is obtained from the Euler-Lagrange equation with boundary conditions, and the solution minimizes the action.The decay probability per unit time and volume depends on the action difference; the prefactor A is estimated as approximately T^4 by dimensional analysis.Source: CALCULATION OF PHASE TRANSITIONS… · pp. 5–6
  • Public profile solvers include CosmoTransitions [51] and BubbleProfiler [52], which use different numerical methods for single- and multiple-field cases.For single fields, both use variants of overshooting and undershooting; for multiple fields, BubbleProfiler uses Newton-Kantorovich, while CosmoTransitions varies a test path through field space.Source: CALCULATION OF PHASE TRANSITIONS… · p. 6

Phase transition with a single scalar field

The single-field application solves the bubble-profile equation for thick-wall and thin-wall potentials using a neural network trained on the Euclidean-action problem. The thin-wall comparison finds agreement with the analytic bubble radius for the NN and BubbleProfiler, while CosmoTransitions gives a smaller radius.

Phase transition with a single scalar field

  • The tested potential has two distinct, non-degenerate minima for 0 < ϵ ≲ 0.3; ϵ = 0.01 and ϵ = 0.3 represent the thin-wall and thick-wall cases, respectively.Smaller ϵ values correspond to progressively thinner cases.Source: Phase transition with a single scalar… · pp. 6–7, Figure 4 · p. 6
  • The finite training domain is chosen to contain the transition well inside its boundaries, and the result remains stable when larger domains are used.The implementation uses ρmin = 0.01, with ρmax = 5R for thick-wall and 2R for thin-wall cases.Source: Phase transition with a single scalar… · p. 7
  • The method minimizes the Euclidean action by adjusting the network's test-field configuration through backpropagation to solve the bubble-profile equation.The calculation uses 500 training points and a single hidden layer; it uses 10 sigmoid units for thick-wall cases and one tanh unit for thin-wall cases.Source: Phase transition with a single scalar… · p. 7
  • In the thin-wall case, the NN and BubbleProfiler match the analytic bubble radius, whereas CosmoTransitions finds a smaller radius, yielding a smaller action and larger tunnelling rate.For the thick-wall case, all three methods agree very well.Source: Figure 6 · p. 7, Phase transition with a single scalar… · p. 8
  • Thin-wall solutions can be numerically unstable because a translated profile may approximately solve the differential equation despite transitioning too early.The authors monitor the differential loss contribution ˆF: it is initially concentrated near the incorrect transition and flattens as training converges.Source: Phase transition with a single scalar… · p. 8
  • The NN achieves very good thin-wall stability with a single tanh unit, and adaptive training-point placement improves the results.The adaptive procedure places more examples near the transition and modifies that distribution during training; the authors suggest further investigation.Source: Phase transition with a single scalar… · pp. 7–8

Phase transition with two scalar fields

The neural-network method handles thick- and thin-wall tunnelling profiles for two scalar fields, including cases where dedicated solvers fail. For thick walls, it agrees closely with BubbleProfiler; for thin walls, it finds a path when BubbleProfiler does not.

Phase transition with two scalar fields

  • The two-field potential has a local minimum at φ1 = φ2 = 0 and a global minimum near φ1 ≈ φ2 ≈ 1; δ is set to 0.4 for thick-wall and 0.02 for thin-wall cases.Source: Phase transition with two scalar fields · p. 8
  • For the thick-wall case, NN and BubbleProfiler agree closely on both field profiles and the path, while CosmoTransitions differs more for φ2.The small differential loss contribution supports convergence to a correct solution; the network also finds initial field positions matching BubbleProfiler without using them as training inputs.Source: Phase transition with two scalar fields · p. 8, Figure 7 · p. 8, Figure 8 · p. 9
  • For the thin-wall case, a deeper network improves performance, and the NN path agrees well with CosmoTransitions when BubbleProfiler finds no solution.No plot is shown because all three codes do not produce a solution.Source: Phase transition with two scalar fields · pp. 8–9
  • The neural-network approach copes with both thick- and thin-wall cases where CosmoTransitions or BubbleProfiler fail.For the thin-wall case, BubbleProfiler finds no solution.Source: Phase transition with two scalar fields · p. 9

Singlet-scalar extended Standard Model with finite

The section applies the coupled-equation method to a Z2-symmetric singlet-scalar extension of the Standard Model, including thermal corrections. At the stated parameter point, the NN bubble profiles agree very well with both dedicated solvers.

temperature contributions

  • The model adds a single Z2-symmetric scalar to the Standard Model Higgs sector, motivated in part by dark matter and electroweak baryogenesis.A strong electroweak phase transition is a crucial requirement for the latter motivation.Source: temperature contributions · p. 9
  • The calculation uses a tree-level barrier and includes the high-temperature one-loop thermal corrections as temperature-dependent mass corrections.The potential used is the sum of the tree-level and thermal contributions.Source: temperature contributions · pp. 9–10
  • The displayed setup assumes Z2 symmetry, though relaxing it would add three parameters to the most general renormalisable potential.Source: temperature contributions · p. 9
  • As temperature falls, the preferred vacuum changes from the Z2-broken state (0, w) to the electroweak state (v, 0), with nucleation set by S3(TN)/TN ≃ 140.At the critical temperature, the electroweak and restored-Z2 minima are degenerate.Source: temperature contributions · p. 10
  • At T = 85 GeV, NN, BubbleProfiler, and CosmoTransitions show very good agreement on the singlet-scalar bubble profiles, with small differential loss contributions indicating convergence.The example uses TC = 110 GeV, λm = 1.5, λs = 0.65, and the nucleation temperature reported by BubbleProfiler.Source: Figure 9 · p. 9, temperature contributions · p. 10

CONCLUSIONS

The paper presents neural networks as a flexible approach to solving differential equations and applies them to electroweak tunnelling profiles. The method matches dedicated solvers competitively and succeeds in examples where they fail, though robust domain-size selection remains needed for full automation.

CONCLUSIONS

  • The method avoids trial functions by putting boundary conditions into the loss, uses the differential loss shape to assess solutions, and proposes iterative domain growth and adaptive anchor placement.Source: CONCLUSIONS · p. 10
  • The approach produced fully differentiable solutions for ordinary, coupled, and partial differential equations with known analytic solutions, with good performance from relatively small networks.Source: CONCLUSIONS · p. 10
  • Across the tested tunnelling examples, the NN converges with competitive accuracy and finds correct solutions where CosmoTransitions or BubbleProfiler fails.A more robust way to choose domain sizes is still needed to develop a fully automated tool.Source: CONCLUSIONS · p. 10
  • The authors envision extending the approach beyond differential equations to problems in perturbative and non-perturbative quantum field theory.Source: CONCLUSIONS · p. 10

Related papers

Loading 1902.05563v2…