Source-linked AI summary
Geometry-enhanced Pre-training on Interatomic Potentials
Taoyong Cui, Chenyu Tang, Mao Su, Shufei Zhang, Yuqiang Li, Lei Bai, Yuhan Dong, Xingao Gong, Wanli Ouyang
TL;DR
MLIPs need more labeled data, but ab initio labeling is expensive, especially for complex molecular systems. The paper introduces GPIP, which pretrains on low-cost CMD-generated 3D structures using masking, denoising, and contrastive learning before AIMD-based fine-tuning. Across molecular benchmarks, GPIP consistently improves baseline MLIPs with relatively accessible computational requirements.
Problem
MLIP performance and transferability are limited by scarce labeled data because ab initio calculations are expensive, while existing pretraining datasets are costly and may not cover new molecular systems.
Method
GPIP generates unlabeled 3D structures with CMD simulations and applies masking, denoising, and contrastive self-supervised learning before fine-tuning on AIMD-labeled data.
Results
GPIP-based models consistently and significantly outperform corresponding baseline models across datasets spanning simple molecules to complex periodic systems.
Takeaways & Limitations
CMD-generated structures can enhance MLIP performance with relatively accessible computing power requirements across different invariant and equivariant GNN architectures.
Abstract
from arXiv · showhide
Machine learning interatomic potentials (MLIPs) enables molecular dynamics (MD) simulations with ab initio accuracy and has been applied to various fields of physical science. However, the performance and transferability of MLIPs are limited by insufficient labeled training data, which require expensive ab initio calculations to obtain the labels, especially for complex molecular systems. To address this challenge, we design a novel geometric structure learning paradigm that consists of two stages. We first generate a large quantity of 3D configurations of target molecular system with classical molecular dynamics simulations. Then, we propose geometry-enhanced self-supervised learning consisting of masking, denoising, and contrastive learning to better capture the topology and 3D geometric information from the unlabeled 3D configurations. We evaluate our method on various benchmarks ranging from small molecule datasets to complex periodic molecular systems with more types of elements. The experimental results show that the proposed pre-training method can greatly enhance the accuracy of MLIPs with few extra computational costs and works well with different invariant or equivariant graph neural network architectures. Our method improves the generalization capability of MLIPs and helps to realize accurate MD simulations for complex molecular systems.
1 Introduction
MLIP performance and transferability are constrained by scarce, expensive ab initio labels, while existing pretraining data and methods have limited scalability or generalization. GPIP addresses this by pretraining on low-cost CMD structures with geometry-enhanced self-supervised tasks before fine-tuning on AIMD-labeled data.
- MLIP performance and transferability remain limited because obtaining sufficient labeled training data requires expensive ab initio calculations.
- Existing interatomic-potential pretraining methods rely on costly datasets such as OC20 and ANI-1, limiting scalability and coverage of new molecular systems.
- GPIP first pretrains on inexpensive CMD structures, then fine-tunes on AIMD-labeled data to learn task-related information.
- Its geometry-enhanced self-supervision combines masked-atom restoration with noisy coordinates, noise prediction, and 3D contrastive learning.
- Across varied MLIPs and datasets, GPIP is reported to provide consistent and robust performance with unlabeled CMD structures benefiting interatomic-potential prediction.
2 Results
The paper describes message-passing MLIP architectures and the GPIP framework, which combines low-cost CMD structural data with complementary self-supervised tasks. GPIP is implemented across invariant and equivariant GNNs and evaluated on molecular-potential benchmarks with force accuracy and simulation analyses.
- 2.1 Preliminaries: A molecular conformation is represented by atomic numbers and positions, with atom features initialized through learned embeddings.
- 2.1 Preliminaries: Graph neural networks iteratively aggregate neighbor messages using atom interactions and update atom features for downstream energy and force prediction.
- 2.2 GPIP framework: GPIP is model-agnostic and is implemented with both invariant and equivariant GNN baselines across interatomic-potential benchmarks.
- 2.2 GPIP framework: GPIP generates target-system structures with CMD and pretrains on them to learn topology and spatial structure information.
- 2.2 GPIP framework: Its three complementary tasks restore masked atoms with noisy coordinates, predict noise with masked atoms, and use contrastive learning with a 3D network.
- 2.3 Experiments: MD17 evaluation compares GPIP-based SchNet, DimeNet, SphereNet, and GemNet-T against corresponding baselines, with additional force, stability, and equilibrium-statistics analyses.
3 Discussion
GPIP combines CMD-generated geometric structures with masking, denoising, and contrastive pretraining to improve MLIPs across molecular systems. The approach extracts structural information efficiently, reduces data-generation costs, and consistently improves baseline models.
- CMD data provide physically reasonable structures and broad configuration coverage that can support downstream fine-tuning.Long simulations across temperatures and pressures produce data spanning the downstream task distributions.
- CMD-generated pretraining data substantially reduce computational cost relative to ab initio calculations, with millions of atoms generated within hours using LAMMPS.CPU parallelization and GPU acceleration further reduce the cost of producing pretraining data.
- GPIP offers a way to enhance MLIP performance with relatively accessible computing requirements.
- GPIP generates abundant geometric structures with CMD simulations and pretrains MLIPs using masking, denoising, and contrastive learning.The framework targets both topology and spatial structure information from unlabeled CMD data.
4 Methods
The method builds target-system pretraining datasets from CMD trajectories, then applies three self-supervised tasks to learn spatial and topological information before downstream fine-tuning. Experiments span simple molecules, periodic systems, and electrolyte solutions, using energy- and force-based training objectives.
- Pretraining Dataset: GPIP pretraining datasets are generated from CMD trajectories for simple molecules and complex periodic systems, including electrolyte solutions.The benchmarks include MD17, ISO17, liquid water, and electrolyte solutions.
- Pretraining Dataset: Electrolyte structures cover 16 ion–solvent combinations at concentrations of 1 M or 4 M and temperatures from 100 K to 1000 K.
- Fine-tuning Settings: Fine-tuning uses DFT-derived energies and forces for electrolyte solutions, while dataset-specific training and validation settings vary across benchmarks.
- Pretraining Settings: Three self-supervised tasks—masking, denoising, and contrastive learning—learn spatial and topological information from pretraining data.The final objective combines their losses as Lfinal = Lmask + αLdenoise + βLcontrast.
- Pretraining Settings: Denoising predicts added coordinate noise under randomly masked atoms, while contrastive learning maximizes mutual information between GNN and 3D-network latent features.
- Fine-tuning Settings: Fine-tuning fits total energy and atomic forces using a combined energy-and-force loss.The predicted energy is differentiated with respect to atomic coordinates to obtain force-related supervision.
Tables
The tables report evaluation conventions for MD17, Water, and Electrolyte datasets, including metric units, averaging procedures, and uncertainty estimates. They also identify baseline-source and stability caveats for the Water results.
- MD17: Table 1 reports MD17 force MAEs in kcal/mol/Å, with force weighted 100 relative to energy for fair baseline comparisons.Standard deviations use 5 independent experiments; baseline results come directly from or are adapted from original papers.
- Water: Table 2 reports Water force MAEs in meV/Å, stability in ps, unitless RDF MAEs, and diffusivity MAEs in 10^-9 m2/s.Diffusivity averages 5 runs from 5 random initial configurations, with standard deviations from 5 independent experiments.
- Water: Water baseline force and diffusivity MAEs are taken from [37] without reported standard deviations, and short DimeNet and GemNet-T trajectories prevent diffusivity calculation.These caveats affect direct comparison and metric availability across models.
- Electrolyte: Table 3 reports Electrolyte energy and force MAEs in eV and eV/Å, respectively, with standard deviations from 5 independent experiments.The table’s metrics are reported for the Electrolyte dataset.
Figure Legends/Captions
The figures describe GPIP’s data-generation, pretraining, fine-tuning, evaluation, and embedding-visualization workflow. They specify how ISO17 results are organized and how datasets are projected for comparison.
- Figure 1: Figure 1 shows CMD trajectories generating pretraining structures, followed by three complementary self-supervised tasks and GNN fine-tuning on labeled data.The tasks are masked atom restoration with noisy coordinates, noise prediction with masked atoms, and contrastive learning with 3D Nets.
- Figure 2: Figure 2 organizes ISO17 energy and force MAEs by SchNet training size, SchNet and EGNN force results, and known-versus-unknown test settings.The panels compare 20,000 and 400,000 training examples or sweep training data from 20,000 to 400,000.
- Figure 2: Figure 2 reports energy in kcal/mol and force in kcal/mol /Å, with error bars denoting standard deviation across 5 independent experiments.The caption notes that 400,000-example error bars in panels (c) and (d) are smaller than the data points.
- Figure 3: Figure 3 uses UMAP to project pretraining and fine-tuning data into the SchNet-GPIP embedding for MD17, ISO17, Water, and Electrolyte.The four datasets occupy panels (a) through (d), respectively.