Representation of compounds for machine-learning prediction of physical properties

Atsuto Seko; Hiroyuki Hayashi; Keita Nakayama; Akira Takahashi; Isao Tanaka

Source-linked AI summary

Representation of compounds for machine-learning prediction of physical properties

Atsuto Seko, Hiroyuki Hayashi, Keita Nakayama, Akira Takahashi, Isao Tanaka

arXiv:1611.08645v2Nov 26, 2016cond-mat.mtrl-sci

TL;DR

The paper tackles the challenge of constructing effective compound descriptors from limited elemental and structural information. It develops a systematic descriptor-generation procedure and evaluates it with kernel ridge regression and Bayesian optimization across three physical-property datasets. The cohesive-energy model reaches a prediction error of 0.041 eV/atom, near chemical accuracy, while the approach also shows good performance for lattice thermal conductivity and melting temperature.

Problem
Limited first-principles databases and the difficulty of combining elemental and structural information motivate systematic compound descriptors for physical-property prediction.
Method
The study generates same-dimensional, invariant descriptors from simple elemental and structural representations and evaluates them with kernel ridge regression and Bayesian optimization.
Results
The cohesive-energy KRR model achieves a prediction error of 0.041 eV/atom, while descriptor sets show good predictive performance across cohesive energy, lattice thermal conductivity, and melting temperature tasks.
Takeaways & Limitations
Systematic descriptor sets can support accurate physical-property prediction and efficient Bayesian optimization across diverse compound datasets.

Abstract

from arXiv · show

The representations of a compound, called "descriptors" or "features", play an essential role in constructing a machine-learning model of its physical properties. In this study, we adopt a procedure for generating a systematic set of descriptors from simple elemental and structural representations. First it is applied to a large dataset composed of the cohesive energy for about 18000 compounds computed by density functional theory (DFT) calculation. As a result, we obtain a kernel ridge prediction model with a prediction error of 0.041 eV/atom, which is close to the "chemical accuracy" of 1 kcal/mol (0.043 eV/atom). The procedure is also applied to two smaller datasets, i.e., a dataset of the lattice thermal conductivity (LTC) for 110 compounds computed by DFT calculation and a dataset of the experimental melting temperature for 248 compounds. We examine the performance of the descriptor sets on the efficiency of Bayesian optimization in addition to the accuracy of the kernel ridge regression models. They exhibit good predictive performances.

I. INTRODUCTION

The paper addresses the need for systematic compound descriptors that combine elemental and structural information while remaining comparable across diverse compounds. It introduces such a procedure and applies it to DFT and experimental datasets.

Descriptor quality is central to machine-learning models for predicting physical properties from large candidate spaces and existing data.
Limited first-principles databases motivate deriving descriptors from simple elemental and structural representations.
Combined elemental–structural descriptors are difficult to design systematically, creating a need for a general generation procedure.
The proposed approach generates same-dimensional descriptors across chemical compositions and crystal structures while satisfying translational, rotational, and other required invariances.
The approach is evaluated on cohesive energy for 18093 DFT compounds, lattice thermal conductivity for 110 DFT compounds, and experimental melting temperature for 248 compounds.

B. DFT lattice thermal conductivity (110 compounds)

The study includes a 110-compound DFT lattice-thermal-conductivity dataset and uses structural information and selected crystal-structure assumptions in its broader materials evaluation.

The LTC dataset contains 110 compounds whose lattice thermal conductivities were computed by DFT-based phonon methods.
The melting-temperature dataset contains 248 binary compounds spanning room temperature to 3273 K and excludes transition-metal compounds.
For melting-temperature compounds, candidate crystal structures come from the ICSD, with the lowest-energy structure selected when multiple structures exist.
Table I lists elements and valences in the cohesive-energy DFT dataset and shows adopted prototype structures selected for extensive ICSD registration.

III. REGRESSION METHODS

The regression method uses kernel ridge regression with an RBF kernel to predict properties from compound descriptors, fitting coefficients and hyperparameters from training data.

Kernel ridge regression expresses an observed property through similarities between a query descriptor and training descriptors.
The RBF kernel measures descriptor similarity using a length scale σ.
Training coefficients α are determined from the observations, kernel matrix, regularization parameter λ, and identity matrix through matrix operations.
Predictions for a new descriptor use its kernel similarities to training examples and depend on σ and λ selected by grid search.

B. Gaussian process regression

Bayesian optimization evaluates descriptor usefulness by searching known compounds for favorable properties with a Gaussian-process model using an RBF covariance.

Bayesian optimization measures how efficiently descriptors identify the compound with the best observed property among existing data.
The Gaussian-process model is specified by mean and covariance functions, with an RBF covariance for noise-free observations.
Its mean function at a query descriptor is the KRR prediction without the regularization term.

C. Bayesian optimization

Bayesian optimization builds a Gaussian-process model and iteratively samples compounds expected to improve the observed property. The descriptor pipeline represents compounds through atomic representations and distribution statistics, while PI and EI provide sampling criteria.

C. Bayesian optimization: A Gaussian-process model starts from two randomly selected observations and is repeatedly updated with each newly sampled observation.The process continues until all data points are sampled.
C. Bayesian optimization: PI selects the next point by maximizing the probability that its observation is lower than the best observed value for minimization.Here, ybest denotes the best observation among the sampled data.
C. Bayesian optimization: EI is the second acquisition option, and both PI and EI are applied to the LTC and melting-temperature datasets.The formulations use the Gaussian-process mean, standard deviation, cumulative distribution, and probability density functions.
A. Representation of compounds: Compound descriptors are generated by treating atomic representations as a distribution and characterizing it with quantities such as means, standard deviations, skewness, kurtosis, and covariance.The atomic collection is organized as a representation matrix X before distribution statistics are calculated.
A. Representation of compounds: The procedure’s performance depends on the elemental and structural representations and on the representative quantities used to characterize their distributions.The authors therefore propose generating many representations and selecting a useful subset.

B. Atomic representations

The study combines elemental representations with structural representations that describe local atomic environments. Four structural families are used, including radial, pairwise, bond-orientational, and angular representations.

B. Atomic representations: The elemental representation set includes intrinsic, periodic-table, electronic, geometric, and elemental-substance properties.Examples include atomic number, atomic mass, period, group, ionization energies, electronegativities, radii, melting point, and boiling point.
B. Atomic representations: Four structural representation types are introduced: PRDF, GRDF, BOP, and AFS.They are used as histogram representations of partial radial distribution, generalized radial distribution, bond-orientational, and angular Fourier information.
B. Atomic representations: PRDF uses histogram bin counts within specified bin widths and cutoff radii as structural representations.This formulation makes PRDF applicable across a dataset containing a wide range of compounds.
B. Atomic representations: GRDF generalizes PRDF by replacing rectangular histogram functions with pairwise functions of interatomic distance.The study employs Gaussian, trigonometric, and Bessel pairwise functions.
B. Atomic representations: BOP and AFS encode rotationally invariant local structure using spherical-harmonic quantities, with AFS including radial and angular dependencies.BOP uses Q_l and W_l up to a specified maximum l, while AFS uses bond angles and modified spherical harmonics with radial functions.

A. Cohesive energy

Cohesive-energy KRR models improve as descriptors incorporate structural information and covariances, while practical prototype structures retain useful predictive accuracy. With enough training data, the best structural descriptor set approaches chemical accuracy, but composition-only descriptors face an intrinsic structural-variation limit.

Elemental descriptors: 0.249 eV/atom is the error using only means of elemental representations, decreasing to 0.231 eV/atom when elemental standard deviations and covariances are included.Skewness and kurtosis were not important descriptors for prediction.
Intrinsic limitation: 0.211 eV/atom is the estimated intrinsic standard deviation from multiple crystal structures sharing each chemical composition, close to the errors of elemental-only models.This indicates that the elemental representation set is nearly complete for cohesive-energy prediction, while omitted structural information contributes intrinsic error.
Structural descriptors: 0.166 eV/atom is the lowest error using means of elemental and PRDF representations, while adding covariances reduces it to 0.106 eV/atom.The simple use of PRDF representations alone does not produce a good cohesive-energy model.
Structural descriptors: 0.045 eV/atom is achieved by the best model using elemental and trigonometric GRDF representations with standard deviations and covariances, approximately matching 43 meV/atom chemical accuracy.This is about half the error of the best PRDF model.
Training-data dependence: The prediction error decreases with increasing training data, and covariance descriptors improve prediction mainly when a large training dataset is available.The cohesive-energy model using 20 trigonometric GRDFs and 20 BOPs gives the best performance among the structural representations examined.
Practical structures: 0.07 eV/atom is the prediction error obtained with normalized prototype structures when covariances of elemental and structural representations are included.Using normalized prototype structures avoids requiring DFT-optimized structures for the structural representations; means alone give 0.166 eV/atom.

B. LTC

The LTC experiments evaluate descriptor quality through both kernel ridge prediction and Bayesian optimization on small datasets. Structural representations improve prediction, while descriptor choice strongly affects optimization efficiency.

Kernel ridge regression: 0.096 log-scaled LTC prediction error is achieved by the model using means and SDs of elemental and BOP representations.Including covariances reduces accuracy on the small training dataset.
Bayesian optimization: 5.0 average samples are required to find PbClBr with the GP model and BOP, versus 50 for random search.Bayesian optimization is therefore ten times more sample-efficient in this comparison.
Bayesian optimization: 15.1 and 9.1 average observations find CuCl and LiI with GP models using BOPs, compared with 40.5 and 48.6 using GRDFs.The BOP-based models outperform GRDF-based models despite similar RMSE for the latter.

C. Melting temperature

The melting-temperature experiments apply kernel ridge regression and Bayesian optimization to experimental data for 248 solids. Elemental descriptors provide the best prediction, while combined elemental and structural descriptors can improve search efficiency.

Experimental setup: The melting-temperature dataset contains 248 compounds, and optimization performance is averaged over 200 trials.AlN, SiC, and MgO are evaluated as target compounds in the search comparison.
Kernel ridge regression: 236 K is the prediction error of the best model, which uses means, SDs, and covariances of elemental information.BOP improves the model, whereas GRDF reduces accuracy; covariances help only with elemental representations.
Bayesian optimization: 22.5, 28.2, and 20.9 average samples find AlN, SiC, and MgO using means and SDs of elemental and BOP representations.These counts are about six times smaller than the corresponding random-search requirements.
Bayesian optimization: 27.6, 52.1, and 28.8 average samples find AlN, SiC, and MgO with the best-RMSE elemental model.The second-best-RMSE combined model requires fewer samples for all three targets.

VI. CONCLUSION

The study demonstrates systematic compound descriptors derived from simple atomic representations and evaluates them across cohesive energy, LTC, and experimental melting temperature. Performance is assessed through both kernel ridge accuracy and Bayesian optimization.

Scope and evaluation: The descriptor-generation approach is applied to three physical-property datasets: cohesive energy, LTC, and experimental melting temperature.The study evaluates descriptor sets using kernel ridge models and Bayesian optimization.
Cohesive energy: 0.041 eV/atom is the cohesive-energy prediction error obtained by the best model.This result is reported as the study’s cohesive-energy outcome.