Source-linked AI summary
Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals
Chi Chen, Weike Ye, Yunxing Zuo, Chen Zheng, Shyue Ping Ong
TL;DR
Existing ML models for molecular and crystal property prediction remain limited, motivating universal MEGNet graph networks and strategies for data scarcity. MEGNet achieves higher accuracy than prior models across most reported properties while incorporating global state variables and transferable elemental embeddings.
Problem
Current ML models for property prediction still suffer from limitations, while data availability remains a critical bottleneck for graph-network models.
Method
The paper develops MEGNet graph networks that model interactions among atoms, bonds, and global state, incorporating temperature, pressure, and entropy as global inputs and transferring elemental embeddings.
Results
MEGNet models achieve higher accuracy than prior models for most reported molecular and crystal properties, while global-state inputs unify free-energy-related predictions across conditions.
Takeaways & Limitations
Global state variables and transferable elemental embeddings provide strategies for extending property prediction across thermodynamic conditions and data-limited properties.
Takeaways & Limitations
The unified free-energy MEGNet model does not accurately predict H and G at 100 K and 200 K, largely because of insufficient training data.
Abstract
from arXiv · showhide
Graph networks are a new machine learning (ML) paradigm that supports both relational reasoning and combinatorial generalization. Here, we develop universal MatErials Graph Network (MEGNet) models for accurate property prediction in both molecules and crystals. We demonstrate that the MEGNet models outperform prior ML models such as the SchNet in 11 out of 13 properties of the QM9 molecule data set. Similarly, we show that MEGNet models trained on $\sim 60,000$ crystals in the Materials Project substantially outperform prior ML models in the prediction of the formation energies, band gaps and elastic moduli of crystals, achieving better than DFT accuracy over a much larger data set. We present two new strategies to address data limitations common in materials science and chemistry. First, we demonstrate a physically-intuitive approach to unify four separate molecular MEGNet models for the internal energy at 0 K and room temperature, enthalpy and Gibbs free energy into a single free energy MEGNet model by incorporating the temperature, pressure and entropy as global state inputs. Second, we show that the learned element embeddings in MEGNet models encode periodic chemical trends and can be transfer-learned from a property model trained on a larger data set (formation energies) to improve property models with smaller amounts of data (band gaps and elastic moduli).
Introduction
The paper introduces MEGNet graph networks as a unified framework for predicting properties of molecules and crystals while addressing global-state inputs and limited training data. It evaluates MEGNet across molecular and crystal datasets and develops free-energy unification and transfer-learning strategies.
- Related work: Graph-based models generally outperform classical machine-learning models on QM9 and are less sensitive to atomic-descriptor choices.Prior graph models achieved state-of-the-art molecular performance using only atomic numbers and coordinates.
- Related work: Crystal property models must account for lattice periodicity and additional space-group symmetries.Crystal graph approaches use representations designed to preserve permutation and unit-cell invariance.
- Limitations: Most machine-learning models have been developed on either molecular or crystal datasets, while crystal evaluations of models tested on both are limited to formation energies.The paper identifies separate molecular/crystal development and narrow crystal evaluation as limitations of existing work.
- This work: MEGNet models target multiple properties across approximately 131,000 QM9 molecules and 69,000 Materials Project crystals.The framework is proposed as a general, composable model for quantitative structure-state-property prediction in molecules and crystals.
- Results: MEGNet models outperform prior machine-learning models on multiple molecular and crystal property predictions.The introduction states that the models are evaluated on both QM9 molecules and Materials Project crystals.
- Data strategies: A unified free-energy MEGNet incorporates temperature, pressure, and entropy as global state inputs, increasing training data with minimal parameter growth.The paper also uses elemental embeddings learned from larger datasets for transfer learning on properties with less data.
MEGNet Formalism
MEGNet represents molecules and crystals as graphs with atomic, bond, and global-state attributes, then updates these attributes through relational message passing. Stacked blocks and order-invariant readout produce material-level predictions while supporting molecule- and crystal-specific inputs and datasets.
- MEGNet Formalism: MEGNet maps atomic, bond, and global-state attributes from an input graph to an output graph through sequential update operations.Bond attributes are updated first, followed by atomic and global-state attributes.
- MEGNet Formalism: Bond updates combine each bond with its connecting atoms and the global state, while atom updates aggregate information from connected bonds.The aggregation averages bonds connected to each atom, providing a local pooling operation.
- MEGNet Formalism: Stacking modules expands the effective interaction range, allowing longer-range interactions to be incorporated even with a small initial distance cutoff.The first two update steps are localized, while additional modules let atoms and bonds access longer distances.
- MEGNet Formalism: Global-state attributes receive information from all atoms and bonds and can encode system-level variables such as temperature.The global state also serves as a placeholder for larger-scale information exchange.
- MEGNet Formalism: MEGNet blocks combine preprocessing dense layers, a MEGNet module, and skip connections, while multiple blocks increase expressiveness and support deeper training.Set2set readout reduces atomic and bond vector sets to one vector before final multilayer perceptrons generate the output.
- Atomic, Bond and State Attributes: For crystals, graphs use atomic number and spatial distance as atomic and bond attributes, whereas molecular inputs use attributes defined for molecular structures.In this work, molecules and crystals are distinguished by the absence or presence of lattice periodicity.
- Model Construction and Training: Models divide data into training, validation, and test sets and use graph batching to train molecules or crystals with different numbers of atoms.Molecule models use 90% for training, while crystal formation-energy models use 60,000 training crystals.
Results
MEGNet models achieve strong performance across molecular and crystal property-prediction tasks, while unified and transfer-learning strategies address related targets and limited data. The reported results include competitive accuracy, broad benchmark improvements, and explicit performance boundaries.
- QM9 molecules: MEGNet-Full outperforms state-of-the-art SchNet and MPNN enn-s2s models on 11 of 13 QM9 properties.The two exceptions are dipole moment µ and electronic spatial extent R2.
- QM9 molecules: 1.40 meV for zero-point energy and 0.060 eV for band gap are the reported MEGNet errors that exceed chemical-accuracy thresholds.The passage identifies these as the only errors exceeding the listed thresholds.
- QM9 molecules: Simple MEGNet models using atomic number and spatial distance achieve similar performance, outperforming prior models on 8 of 13 properties.Their errors are only slightly higher and remain within chemical accuracy, although convergence is slower for some properties.
- Unified free-energy model: A combined free-energy MEGNet model unifies U0, U, H, and G using temperature, pressure, and entropy as global inputs.The model achieves an overall MAE of 0.010 eV across the four targets, comparable to separate target-specific models.
- Unified free-energy model: The unified model predicts U accurately at 100 K and 200 K outside training data, but H and G show large deviations from DFT at those temperatures.The authors hypothesize that adding H and G data at multiple temperatures would improve performance.
- Materials Project crystals: MEGNet crystal models outperform SchNet and CGCNN across the reported formation-energy, band-gap, elastic-modulus, and classification comparisons.The reported test accuracy for metal/non-metal classification is 86.9%, with ROC area under curve 0.926; errors for Ef, Eg, KV RH, and GV RH are within DFT errors.
- Materials Project crystals: Elemental embeddings transferred from the formation-energy model accelerate training and improve band-gap and elastic-modulus models.An independently trained band-gap model has an MAE of 0.38 eV, whereas transfer learning improves the corresponding model.
Discussion
The discussion presents MEGNet as a broadly applicable graph-network framework for molecules and crystals, with unified state-dependent models and transferable chemical representations addressing limited materials data. It also relates model architecture and interpretability to interaction range and periodic chemical trends.
- Universal framework: Graph networks provide universal building blocks for accurate property prediction in both molecules and crystals.MEGNet models show significantly improved accuracies over prior models for most properties in both domains.
- Unified free-energy modeling: Global state variables enable unified MEGNet models for related thermodynamic properties instead of separate single-purpose models.The approach incorporates temperature, pressure and entropy as global inputs to predict internal energy, enthalpy and Gibbs free energy.
- Unified free-energy modeling: Unifying related models can increase training data several-fold with minimal growth in model complexity, addressing the small datasets common in materials science.The paper contrasts this strategy with building separate models for thermodynamically related quantities.
- Interpretability and transfer learning: MEGNet elemental embeddings recover periodic chemical trends, including within-group similarities and known exceptions such as Eu and Yb among the lanthanoids.These trends arise from formation-energy models trained on a diverse Materials Project dataset spanning most known crystal prototypes and 89 elements.
- Interpretability and transfer learning: The learned embeddings can support materials-property models with limited data through transfer learning from formation-energy models.Transferred elemental embeddings improved band-gap and elastic-moduli models relative to prior ML models.
- Architecture and interaction range: Stacking MEGNet blocks expands the interaction range, so the required number of blocks depends on the property’s relevant spatial interactions.Most QM9 properties need two blocks for chemical-accuracy MAEs, while zero-point vibrational energy needs five and longer-ranged properties require more.
Conclusion
The work develops MEGNet models that perform across molecules and crystals, while extending graph-network models to state-dependent and data-limited properties. These advances support general property models for accelerating materials discovery.
- MEGNet models perform across a broad variety of target properties for both molecules and crystals.
- Graphs represent atoms and bonds naturally, while graph-network updates enable information flow among atoms, bonds, and global state.
- Global state inputs and transfer learning of elemental embeddings extend MEGNet models to state-dependent and data-limited properties.
- These generalizations address limitations in applying machine learning to chemistry and materials science.
- The models provide a foundation for developing general property models for accelerating materials discovery.
Supporting Information Available
The supporting information includes error distributions, temperature-dependent energy predictions, and visualizations of elemental embeddings across MEGNet-block configurations.
- The supporting information provides MEGNet error distributions on the QM9 dataset.
- It includes energy predictions at different temperatures.
- It provides elemental-embedding visualizations for one- and five-MEGNet-block models.
- A t-SNE visualization covers elemental embeddings for one-, three-, and five-MEGNet-block models.