PaperlayerView exact revision on arXiv

Source-linked AI summary

Crystal Structure Representations for Machine Learning Models of Formation Energies

Felix Faber, Alexander Lindmaa, O. Anatole von Lilienfeld, Rickard Armiento

arXiv:1503.07406v1physics.chem-ph

TL;DR

The study addresses how to represent periodic crystal structures for machine-learning models of solid formation energies, extending approaches successful for molecules. It compares three representations and finds similar accuracy overall, with the sine matrix offering the smallest generalization error and lowest computational expense.

  • Problem

    Primitive-cell Coulomb representations can depend on the chosen unit cell because they omit interatomic distances involving neighboring cells.

  • Method

    The study compares Ewald-sum, extended Coulomb-like, and sine-matrix representations for machine-learning models of crystal formation energies.

  • Results

    0.4−0.6 eV/atom generalization error was obtained for all representations at training sets of 3000 crystals.

  • Takeaways & Limitations

    The sine matrix is the most efficient representation and gives the smallest generalization-error value among the compared methods.

Abstract

from arXiv · show

We introduce and evaluate a set of feature vector representations of crystal structures for machine learning (ML) models of formation energies of solids. ML models of atomization energies of organic molecules have been successful using a Coulomb matrix representation of the molecule. We consider three ways to generalize such representations to periodic systems: (i) a matrix where each element is related to the Ewald sum of the electrostatic interaction between two different atoms in the unit cell repeated over the lattice; (ii) an extended Coulomb-like matrix that takes into account a number of neighboring unit cells; and (iii) an Ansatz that mimics the periodicity and the basic features of the elements in the Ewald sum matrix by using a sine function of the crystal coordinates of the atoms. The representations are compared for a Laplacian kernel with Manhattan norm, trained to reproduce formation energies using a data set of 3938 crystal structures obtained from the Materials Project. For training sets consisting of 3000 crystals, the generalization error in predicting formation energies of new structures corresponds to (i) 0.49, (ii) 0.64, and (iii) 0.37 eV/atom for the respective representations.

I. INTRODUCTION

The study adapts Coulomb-matrix descriptors to periodic crystals and compares three representations for machine-learning formation energies. At 3000 training crystals, the methods achieve broadly similar accuracy, with the sine-based representation performing best among those tested.

  • Study aim: The study targets periodic crystal representations for machine-learning models of solid formation energies.It extends an approach previously used for molecular atomization energies.
  • Representations: Three generalizations are evaluated: an Ewald sum matrix, an extended Coulomb-like matrix, and a sine-function matrix based on crystal coordinates.The extended matrix includes neighboring unit cells, while the sine matrix mimics periodicity and Ewald-matrix features.
  • Results: 0.4−0.6 eV/atom: the three representations have roughly similar accuracy at 3000 training crystals.This is less accurate than the reported 0.02 eV/atom for molecular atomization energies with similarly sized training sets.
  • Scope: The study uses crystal structures and formation energies to test whether molecular machine-learning approaches can be transferred to periodic solids.The paper also discusses the disparity between finite molecular and periodic-system performance.

II. KERNEL RIDGE REGRESSION

The paper uses kernel ridge regression to map feature vectors to target properties through kernel evaluations and regularized fitting. Generalization is assessed with held-out test data, mean absolute error, and repeated k-fold cross-validation.

  • Regularization: Ridge regularization adds λI to the fitting equations, with λ controlling regularization and I denoting the identity matrix.Regularization is introduced to address overfitting from highly flexible ansätze.
  • Kernel ridge regression: Kernel ridge regression extends ridge regression by mapping data into a feature space and using kernel inner products instead of an explicit mapping.The kernel trick makes the feature-space calculation possible through k(x_i, x_j).
  • Kernel choice: The Laplacian kernel with Manhattan norm is used throughout this study because it performed well for molecular atomization-energy prediction.The authors note that Gaussian or other kernels could also have been used.
  • Evaluation: Generalization error is estimated from the mean absolute error on data excluded from the training solution.The excluded data form the test set, while the remaining data form the training set.
  • Evaluation: Repeated k-fold cross-validation with random sampling averages test errors across independent machine-learning runs.The test-set size must be large enough for MAE_test to estimate generalization error accurately, creating a training/test-size trade-off.

III. REPRESENTATIONS

The representation maps atomic identities and positions into a feature vector that should be complete, compact, descriptive, and simple. PCA is used to inspect or reduce representation dimensionality, but it must be constructed from training data only.

  • Representation design: The feature vector maps the positions and identities of all atoms into a vector x for machine-learning input.The representation is described as central to the performance of the machine-learning scheme.
  • Representation properties: A complete, non-degenerate representation includes all input features relevant to the prediction problem and distinguishes different molecules.Relevance depends on the application; chirality may matter for optical activity but not molecular atomization energies.
  • Representation properties: A compact, unique representation minimizes redundant features and gives identical vectors for equivalent inputs such as rotated copies.The desired invariance is illustrated using differently oriented instances of the same molecule.
  • Representation properties: A descriptive representation places inputs with similar target values near one another in feature space.Small structural changes associated with similar properties should generally produce nearby vectors.
  • Representation properties: A simple representation requires little computational effort to generate.The paper contrasts this with using eigenvalues obtained by solving the many-body Schrödinger equation.
  • PCA: PCA reduces dimensionality while conserving as much information as possible, and its construction must use only the training set.The test set must not be used when constructing the PCA representation.

A. Molecular Coulomb matrix

The molecular Coulomb matrix represents atomic systems through a symmetric atom-by-atom matrix, but its ordering and periodic-cell extensions introduce representation concerns.

  • A. Molecular Coulomb matrix: The Coulomb matrix is a symmetric atom-by-atom feature matrix defined in Hartree atomic units.Its diagonal and off-diagonal entries encode different atomic and interatomic quantities.
  • A. Molecular Coulomb matrix: Off-diagonal elements represent pairwise Coulomb repulsion between positive atomic cores, while diagonal elements fit free-atom potential energies.
  • A. Molecular Coulomb matrix: One molecule can produce several Coulomb matrices because atom ordering is not unique.Row- or column-norm sorting can impose atom-index invariance, but sorting introduces differentiability issues.
  • A. Molecular Coulomb matrix: The molecular model benchmark uses 7165 QM7 molecules containing up to seven atoms from C, O, N, and S, with hydrogen completing valencies.Coordinates were relaxed with force fields and atomization energies calculated using PBE0 density functional theory.
  • A. Molecular Coulomb matrix: Applying a primitive-cell Coulomb matrix directly to crystals is non-unique because neighboring-cell distances are omitted and depend on the chosen primitive cell.

B. Ewald Sum Matrix

The Ewald sum matrix extends the Coulomb matrix by encoding full electrostatic interactions between unit-cell atoms and their infinite lattice repetitions.

  • B. Ewald Sum Matrix: Each Ewald sum matrix element represents the full Coulomb interaction between two unit-cell sites, including their infinite lattice repetitions.The matrix remains atom-by-atom while incorporating periodic interactions.
  • B. Ewald Sum Matrix: The Ewald approach replaces a problematic infinite electrostatic double sum with two rapidly converging sums and one constant.The short-range term uses lattice vectors, while the long-range term is evaluated in reciprocal space.
  • B. Ewald Sum Matrix: The short-range sum runs over lattice vectors within a cutoff sphere, and the reciprocal-space sum runs over nonzero reciprocal vectors within a converged cutoff sphere.
  • B. Ewald Sum Matrix: Diagonal entries use the Ewald interaction energy of the lattice formed by equivalent atoms.
  • B. Ewald Sum Matrix: The screening length parameter a changes convergence rates but not the final value of x_ij.The implementation specifies a choice involving M, the number of atoms in the unit cell.

C. Extended Coulomb-like Matrix

The extended Coulomb-like matrix represents interactions between atoms in a primitive unit cell and atoms in neighboring unit cells.

  • C. Extended Coulomb-like Matrix: The representation forms an M by N·M matrix from electrostatic interactions between each of M unit-cell atoms and atoms in the N closest unit cells.It follows the regular Coulomb matrix form while enlarging the set of represented atoms.

D. Sine Matrix

The sine matrix replaces periodic electrostatic sums with a simpler coordinate-based potential that preserves lattice periodicity and divergence at coincident repetitions.

  • D. Sine Matrix: The sine matrix substitutes the full Ewald interaction sums with an arbitrarily chosen two-point potential.The potential is designed to retain basic properties of periodic electrostatic interactions.
  • D. Sine Matrix: The constructed potential is periodic in each crystal coordinate, equal for equivalent neighboring atoms, and divergent when the atoms coincide.
  • D. Sine Matrix: Figure 1 depicts the interaction magnitude for one atom at r1=(x,y) and a fixed atom at the origin across a two-dimensional lattice.The magnitude repeats across cells and diverges near repetitions of the origin atom.
  • D. Sine Matrix: Crystal-coordinate differences inside the sine function enforce periodicity with respect to the lattice vectors.The resulting representation is a straightforward M by M matrix depending only on one unit cell.
  • D. Sine Matrix: The representations have no proof of completeness or uniqueness and are presented as sensible extensions of the Coulomb matrix idea.

IV. IMPLEMENTATION DETAILS AND DATA SETS

The study implemented kernel ridge regression for molecular and crystal datasets, using Materials Project formation energies and grid-selected hyperparameters for 3000-crystal training sets.

  • Implementation: Kernel ridge regression was implemented with a Laplacian kernel for both molecular and crystal formation-energy representations.The implementation used Python with numpy, scipy, and pymatgen.
  • Data set: The crystal dataset contains 3938 periodic solids extracted from the Materials Project database.The systems were extracted without obvious order, and their element distribution roughly matches published materials in the literature.
  • Hyperparameter selection: Hyperparameters λ and σ were selected by evaluating MAEtest on 3000-crystal training sets over a two-dimensional logarithmic grid.The grid used spacing factors of 2 for σ and 10 for λ.
  • Hyperparameter selection: Table I reports the optimal λ and σ values for each representation using 3000 randomly drawn Materials Project crystals.The table is specifically defined for machines trained on this 3000-crystal sample.
  • Hyperparameter selection: MAEtest was relatively insensitive to the selected regularization and kernel-width parameters near the grid minima.The regions around the minima in the generated grids were relatively flat.

V. RESULTS

The results compare formation-energy prediction across crystal representations and against molecular atomization-energy prediction, using error as a function of training-set size.

  • Dataset comparison: The Materials Project and QM7 datasets are compared using different prediction targets: crystal formation energies versus molecular atomization energies.The comparison includes the regular Coulomb matrix for QM7 molecules.
  • Error versus training size: Figure 3 plots mean absolute generalization error against training-set size for Sine, Ewald, GCM, and QM7 Coulomb-matrix models.The crystal models predict formation energies in Materials Project, while QM7 measures molecular atomization energies.
  • PCA analysis: The MP data for sine and extended Coulomb representations are more uniformly spread in PCA space than the localized QM7 clusters.The Ewald representation has a central cluster and several points far from it.

VI. DISCUSSION

The discussion finds broadly similar representation performance, a slight advantage for the sine matrix, and systematic error reduction with larger training sets, while highlighting the greater diversity of periodic materials.

  • Representation performance: The sine matrix reaches lower generalization error and develops more favorably with training-set size than the other representations.Its performance advantage is described as slight.
  • Representation performance: The representations have roughly similar performance, so representation choice may not strongly affect generalization error for a given application.The sine matrix remains attractive because it requires the least computational expense.
  • Training-set size: Generalization error systematically decreases as training-set size increases, suggesting that larger training datasets could reduce error further.This trend is reported across all representations.
  • Dataset diversity: The MP chemical space is substantially broader than QM7, which contains fewer element types and small molecules with no more than seven non-hydrogen atoms.The PCA shows tight QM7 clusters compared with the broader periodic-system representations.
  • Future improvements: The paper identifies larger datasets, improved representations, and chemically restricted datasets as three routes for improving ML of periodic systems.Improved representations may reduce the need for larger training sets, while compact chemical spaces may yield more promising performance.

VII. SUMMARY AND CONCLUSIONS

The study evaluates three crystal-structure representations for formation-energy ML and finds the sine matrix most efficient and accurate, while attributing the molecule–solid gap partly to insufficient coverage of periodic chemical and structural diversity.

  • Main findings: The sine matrix simulates features of an infinite Coulomb sum while delivering the smallest generalization-error value among the tested representations.The paper also identifies it as the most efficient representation.
  • Scope and interpretation: The crystal representations achieve less favorable accuracy than molecular atomization-energy models because the training data insufficiently cover the diverse compositional and structural space.The paper states that the full potential of ML for periodic systems remains to be demonstrated.
  • Future direction: Larger or chemically restricted training subsets may enable more accurate machine models for periodic systems.This conclusion follows from the reported improvement of MAEtest with training-set size.
Loading 1503.07406v1…