Committee neural network potentials control generalization errors and enable active learning

Christoph Schran; Krystof Brezina; Ondrej Marsalek

Source-linked AI summary

Committee neural network potentials control generalization errors and enable active learning

Christoph Schran, Krystof Brezina, Ondrej Marsalek

arXiv:2006.01541v2Jun 2, 2020physics.chem-phcs.LGphysics.comp-phstat.ML

TL;DR

The paper addresses the challenge of building robust interatomic potentials with representative training sets while controlling generalization error. It develops committee neural network potentials that average independently trained models, use disagreement for uncertainty-aware active learning, and bias simulations. Applied to water, the approach produces a compact 814-structure model with excellent agreement across diverse classical and quantum conditions.

Problem
Robust machine-learning potentials require representative training sets, whose construction is especially challenging for condensed-phase systems.
Method
The method combines independently trained committee NNPs with disagreement-based query-by-committee training-set expansion and simulation biasing.
Results
The final water model contains 814 structures and shows excellent agreement with DFT reference energies, forces, and properties across classical and quantum simulations.
Takeaways & Limitations
Committee disagreement enables largely automated, data-driven training-set generation while minimizing required ab initio calculations.

Abstract

from arXiv · show

It is well known in the field of machine learning that committee models improve accuracy, provide generalization error estimates, and enable active learning strategies. In this work, we adapt these concepts to interatomic potentials based on artificial neural networks. Instead of a single model, multiple models that share the same atomic environment descriptors yield an average that outperforms its individual members as well as a measure of the generalization error in the form of the committee disagreement. We not only use this disagreement to identify the most relevant configurations to build up the model's training set in an active learning procedure, but also monitor and bias it during simulations to control the generalization error. This facilitates the adaptive development of committee neural network potentials and their training sets, while keeping the number of ab initio calculations to a minimum. To illustrate the benefits of this methodology, we apply it to the development of a committee model for water in the condensed phase. Starting from a single reference ab initio simulation, we use active learning to expand into new state points and to describe the quantum nature of the nuclei. The final model, trained on 814 reference calculations, yields excellent results under a range of conditions, from liquid water at ambient and elevated temperatures and pressures to different phases of ice, and the air-water interface - all including nuclear quantum effects. This approach to committee models will enable the systematic development of robust machine learning models for a broad range of systems.

I. INTRODUCTION

Machine-learning potentials offer powerful chemical-system modeling, but their reliability depends on representative training sets. Committee models address this challenge by improving predictions, estimating generalization error, and guiding automated training-set expansion.

Robust and representative training sets are a crucial, challenging component of machine-learning potentials, especially for condensed-phase systems.
Averaging independently trained committee members usually improves prediction accuracy relative to individual models.
Committee disagreement, measured from member-prediction variation, estimates generalization error and supports model validation.
Query by committee adds previously unlabeled data with maximal disagreement to systematically improve the model.
The proposed committee NNPs share atomic-environment descriptors, improving accuracy while enabling disagreement monitoring, biasing, and active learning.
The unified framework combines committee models with adaptive active learning and disagreement biasing to develop robust MLPs.

II. COMMITTEE NEURAL NETWORK POTENTIALS

Committee neural network potentials combine independently optimized neural networks into an averaged model while using their disagreement to estimate uncertainty, guide training-set selection, and control simulations.

Committee construction: Committee NNPs use multiple neural networks optimized independently with random subsampling of a shared training set, creating variation among committee members.Each member is trained on a slightly different subset, with a small fraction of the full training set intentionally omitted.
Prediction and uncertainty: The committee prediction averages individual-member predictions, while the standard deviation of those predictions defines the committee disagreement.Energy disagreement provides global information, whereas force disagreement resolves uncertainty for individual atomic environments.
Active learning: Committee disagreement provides an estimate of prediction uncertainty and can identify relevant configurations for expanding a representative, reliable training set.The training set must be representative of planned simulations and dense enough for reliable interpolation between training points.
Simulation control: A thresholded biasing potential acts on energy disagreement to keep simulations within configurations where the committee is well described.The bias remains inactive below a threshold and can erect a barrier against rare excursions associated with high generalization errors.
Implementation: Sharing symmetry functions across committee members avoids repeated descriptor evaluation, while separate neural-network evaluations add only a small computational overhead.The methodology includes on-the-fly energy and force disagreement evaluation and energy-disagreement biasing in CP2K.

III. ACTIVE LEARNING PROCEDURE FOR COMMITTEE NEURAL NETWORK POTENTIALS

The workflow adaptively expands committee neural network potentials across generations by selecting informative configurations, adding reference calculations, and using each model to generate candidates for subsequent conditions. Committee disagreement is monitored throughout to guide training-set growth and control simulation reliability.

Active learning: Active learning selects configurations with the highest committee disagreement to iteratively improve the model and training set.The workflow applies query by committee separately at each selected state point.
Generational workflow: Each generation adds new state points and uses its resulting C-NNP to generate candidate structures for the next generation.The workflow is organized into multiple QbC cycles and generation-level model updates.
Disagreement metric: Force disagreement is selected because it responds to local environments while remaining insensitive to a global potential-energy offset.QbC-cycle convergence is detected by monitoring this disagreement.
Reference calculations: Only actively selected configurations require explicit electronic-structure reference calculations in later generations, reducing the number of expensive calculations.Candidate structures are generated by molecular dynamics using the previous generation’s C-NNP.
Model fitting: Training sets from the selected conditions are combined after convergence, followed by a final tight optimization of the generation’s committee potential.This produces the model used in subsequent production or candidate-generation simulations.
Accuracy control: The final model is accepted when committee disagreement remains within the range observed for conditions represented during training, supporting controlled accuracy in production simulations.The workflow uses disagreement monitoring and biasing to support stable simulations and valid new configurations.

A. Development of the Committee Model

The committee neural network potential is developed through query-by-committee active learning, using disagreement to select configurations, monitor convergence, and control simulations across increasingly diverse water conditions.

First-generation QbC: The first generation begins from liquid water at 300 K and uses query-by-committee iterations to select configurations according to committee force disagreement.The process evaluates training, candidate, and selected structures, then adds the highest-disagreement candidates to the training set.
First-generation QbC: The eight-member C-NNP force RMSE decreases from roughly 60 meV/Å to about 40 meV/Å after roughly 100 added training points.The full committee remains more accurate than smaller committees and individual NNPs, whose initial RMSE is twice as large as the full C-NNP.
First-generation QbC: 111 structures are retained for the first-generation C-NNP after 10 QbC iterations, with convergence reached after roughly 100 added structures.The disagreement decreases as high-disagreement points enter the training set, and the selected-point disagreement approaches the training and candidate distributions.
Disagreement-guided simulation: Committee disagreement is monitored during simulations and can be biased to prevent configurations from entering poorly determined regions of configuration space.For the air-water slab, energy-disagreement biasing suppresses the high-disagreement tail while only subtly affecting force disagreement and local behavior.
Expansion across conditions: The active-learning workflow expands from the initial liquid-water condition to ice, elevated temperatures and pressures, the air-water interface, and quantum nuclei.Generations 3 and 4 add 240 and 205 reference configurations, producing a final training set of 814 structures without ab initio path-integral molecular dynamics.
Expansion across conditions: Force RMSE improves substantially across generations for quantum structures, while classical structures show only minor improvement because generation 1 already describes them well.Figure 5 evaluates force RMSE against the revPBE0-D3 reference on independent test sets, averaged across state points and separated by nuclear treatment.

B. Validation of the Committee Model

The final generation 4 C-NNP accurately reproduces reference structural, dynamical, energy, and force properties across liquid water, ice, and interface conditions, including classical and quantum nuclei.

RMSE validation: 40 and 52 meV/Å are the force RMSEs for classical and quantum structures, respectively, averaged across the independent test conditions.The test set spans liquid water, ice Ih, ice VIII, and the air-water interface.
Static properties: The final model accurately reproduces liquid-water structural properties, including O-H distributions, proton sharing, hydrogen-bond angles, and O-O radial distribution functions.Agreement holds for both classical and quantum descriptions of the nuclei.
Vibrational dynamics: The vibrational spectra show an essentially perfect classical match and a very good quantum match, with only small broadening and red shifts.Low-intensity spectral features are also reproduced accurately, including those visible only on a logarithmic scale.
Liquid-water validation: The model matches reference ab initio structure and vibrational dynamics without requiring explicit AIPIMD simulations during parametrization.This result is reported specifically for liquid water at 300 K.
Cross-condition validation: The water slab has the largest RMSEs, whereas ice Ih and ice VIII are reproduced best; quantum structures generally have slightly larger RMSEs.Despite the interface being the most challenging condition, its RMSEs remain similar to or lower than previous water models.

V. CONCLUSIONS

The work presents committee neural network potentials as an automated framework for robust, uncertainty-controlled machine-learning potentials. Applied to water, the approach produces a compact training set and accurate predictions across diverse classical and quantum conditions.

V. CONCLUSIONS: Committee averaging outperforms individual member NNPs, while committee disagreement monitors and controls accuracy relative to the parent ab initio method.The shared descriptors keep the additional computational overhead low.
V. CONCLUSIONS: Repeated query-by-committee processes generate training sets in a largely automated, data-driven fashion while minimizing required reference ab initio calculations.The workflow was demonstrated for water across varied conditions with classical and quantum descriptions of the nuclei.
V. CONCLUSIONS: The methodology is presented as applicable across broad phase-diagram regions, varied conditions, and systems of increasing complexity.The authors also expect transfer to other neural-network and suitably stochastic kernel-based potentials.
V. CONCLUSIONS: Committee-based models are proposed as a routine component of machine-learning-potential development because of their low additional complexity and effort.

VI. COMPUTATIONAL DETAILS

The computational workflow combines Python orchestration, NNP optimization, DFT reference calculations, and classical or path-integral simulations. Independent validation uses fully converged quantum simulations and reference energies and forces across targeted conditions.

Workflow implementation: The active-learning workflow was implemented in Python to coordinate data manipulation, NNP optimization, DFT evaluations, and candidate-structure sampling across four generations.
Model optimization: NNP optimizations used the open-source n2p2 code with symmetry functions established for reproducing water across a broad range of conditions.
Candidate sampling: Classical candidate-generation simulations ran for 0.5 ns with 0.5 fs timesteps, while path-integral simulations ran for 0.25 ns with 0.25 fs timesteps and four replicas.
Independent validation: The independent validation set contained 8000 reference calculations from eight classical and eight quantum ensembles spanning the targeted conditions.Production simulations used fully converged path integrals with 128 or 256 replicas, depending on temperature and phase.
Liquid-water benchmark: Liquid-water property benchmarks matched previously published AIMD and AIPIMD setups using 64 molecules in a 12.42 Å cubic box at 300 K.

Appendix A: Biasing of Force Disagreement

The appendix considers biasing local force disagreement to detect structural changes, but this approach is computationally expensive and was not implemented.

Biasing each atomic force vector separately provides sensitivity to local structural changes when its disagreement exceeds σ0.The resulting contribution uses a shifted harmonic form.
The corresponding biasing force on other atoms requires differentiating the force-disagreement definition.This introduces a second derivative matrix operator projected along the force direction.
The method was not implemented because evaluating second derivatives for every atom would incur substantial computational cost.The study therefore used total energy disagreement biasing, which worked sufficiently well in practical simulations.

Supplementary material for: Committee neural network potentials control generalization errors and enable active learning

The supplementary material identifies the authors and their affiliation with Charles University in Prague.

The supplementary material lists Christoph Schran, Krystof Brezina, and Ondrej Marsalek as authors.
The authors are affiliated with Charles University, Faculty of Mathematics and Physics, in Prague, Czech Republic.

S1. ADDITIONAL VALIDATION OF THE COMMITTEE MODEL

Additional validation shows that the generation 4 C-NNP model closely reproduces reference structural and dynamical properties of liquid water at 300 K for classical and quantum nuclei.

The generation 4 C-NNP model’s O-H and H-H RDFs overlap perfectly with reference data for classical and quantum nuclei.These comparisons concern liquid water at 300 K.
Low-intensity features of the hydrogen VDOS are accurately reproduced when plotted on a logarithmic scale.
The proton-sharing coordinate δ distribution is accurately captured, including tails near zero associated with strong proton sharing.The match is essentially perfect for classical nuclei and remains accurate for quantum nuclei.

S2. VALIDATION OF GENERATION 1 COMMITTEE MODEL

Generation 1 already reproduces classical liquid-water properties accurately, while quantum properties and independent-test-set errors show visible degradation despite its small classical training set.

Generation 1 reaches the desired accuracy for classical static and dynamic properties of liquid water at 300 K.
Quantum predictions show minor but visible deviations in the O-O RDF and VDOS, including broadened and red-shifted O-H stretching and bending peaks.The overall match remains remarkably accurate despite using no quantum structures in training.
Quantum configurations are reproduced less accurately than classical state points by generation 1, whose training set contains just 111 classical configurations from one AIMD trajectory.
The supplementary analysis repeats the generation 1 validation across structural, dynamical, RDF, VDOS, and proton-sharing properties.
The independent test set contains 8000 uncorrelated configurations spanning liquid water, ice Ih, ice VIII, and the air-water interface under classical and quantum nuclear descriptions.