PaperlayerView exact revision on arXiv

Source-linked AI summary

Machine learning with systematic density-functional theory calculations: Application to melting temperatures of single and binary component solids

Atsuto Seko, Tomoya Maekawa, Koji Tsuda, Isao Tanaka

arXiv:1310.1546v2cond-mat.mtrl-sci

TL;DR

The paper addresses prediction of melting temperatures across single and binary compounds using systematic DFT-derived information and regression. It compares four regression methods and demonstrates kriging for efficient discovery, finding that SVR performs best and kriging outpaces random selection.

  • Problem

    Predicting melting temperatures across a broad range of single and binary compounds requires models that can use systematic computational information beyond limited existing approaches.

  • Method

    The study combines systematic DFT calculations with OLSR, PLSR, SVR, and GPR, then demonstrates kriging for melting-temperature materials design.

  • Results

    SVR provides the best prediction among the four regressions, DFT-computed physical properties improve prediction, and kriging discovers the highest-melting compound faster than random designs.

  • Takeaways & Limitations

    The results support using DFT-derived physical properties and kriging in efficient materials design for melting-temperature applications.

Abstract

from arXiv · show

A combination of systematic density functional theory (DFT) calculations and machine learning techniques has a wide range of potential applications. This study presents an application of the combination of systematic DFT calculations and regression techniques to the prediction of the melting temperature for single and binary compounds. Here we adopt the ordinary least-squares regression (OLSR), partial least-squares regression (PLSR), support vector regression (SVR) and Gaussian process regression (GPR). Among the four kinds of regression techniques, the SVR provides the best prediction. In addition, the inclusion of physical properties computed by the DFT calculation to a set of predictor variables makes the prediction better. Finally, a simulation to find the highest melting temperature toward the efficient materials design using kriging is demonstrated. The kriging design finds the compound with the highest melting temperature much faster than random designs. This result may stimulate the application of kriging to efficient materials design for a broad range of applications.

I. INTRODUCTION

The study combines systematic DFT calculations with regression techniques to predict melting temperatures for single and binary compounds. It also motivates kriging as a strategy for efficient materials design.

  • Motivation: Systematic DFT calculations and machine learning can extract patterns from existing data to predict target physical properties.The approach is positioned within computational materials design and high-throughput screening.
  • Motivation: Existing melting-temperature theories are limited across bonding types and compound classes.The covalent-crystal scaling theory is not directly applicable to compounds with other chemical bondings.
  • Related work: Earlier machine learning predicted melting temperatures for 44 AB suboctet compounds using constituent-atom quantities and regularized linear regression.The study identifies systematic DFT predictors as a route toward more accurate models.
  • Study design: This study compares OLSR, PLSR, SVR, and GPR for prediction models covering a wider range of single and binary compounds.The models use systematic DFT calculations and advanced regression techniques.
  • Materials design: Kriging treats materials design as black-box optimization and selects measurements to increase the chance of discovering optimal compounds.The case study targets compounds with the highest melting temperature.
  • Regression methods: PLSR avoids multicollinearity by extracting response-relevant latent variables from predictor variables.These latent variables are linear combinations of the predictors and support linear prediction with many variables.

1. Support vector regression (SVR)

Support vector regression constructs nonlinear response functions with kernels, modeling predictor–response relationships in a higher-dimensional feature space while tolerating errors within an ε-insensitive margin.

  • Support vector regression (SVR): SVR constructs a nonlinear regression function using a kernel function.It is described as a regression version of support vector machines.
  • Support vector regression (SVR): In ε-SVR, the response is modeled from predictor variables, a weight vector, and a bias parameter.The training data consist of predictor vectors x_i and response variables y_i.
  • Support vector regression (SVR): The feature map φ(x) sends predictors into a higher-dimensional space and is defined implicitly through a kernel function.A valid mapping exists when the kernel is positive semidefinite; Gaussian and polynomial kernels are popular choices.
  • Support vector regression (SVR): The ε-insensitive loss ignores errors smaller than ε, while slack variables allow larger deviations during optimization.The formulation includes a positive regularization parameter C and constraints involving ξ_i and ξ*_i.
  • Support vector regression (SVR): The optimization can be solved in a dual formulation using Lagrange multipliers, with α and α* subject to constraints.The dual formulation uses a kernel matrix whose entries are k(x_i, x_j).

2. Gaussian process regression (GPR)

Gaussian process regression is a Bayesian approach for nonlinear estimation that predicts both a function value and its variance using a Gaussian kernel.

  • Gaussian process regression (GPR): GPR is a Bayesian regression technique used for nonlinear estimation problems.A Gaussian process generalizes the multivariate Gaussian probability distribution.
  • Gaussian process regression (GPR): At a new point x*, GPR provides both a prediction f(x*) and a prediction variance v(f*).These quantities are described using a Gaussian kernel function.
  • Gaussian process regression (GPR): The prediction uses kernel values between x* and the training examples, represented by the vector k*, together with the unit matrix I.The prior distribution is specified with variance σ^2.

C. Kriging

Kriging uses GPR to select compounds with the greatest probability of exceeding the current best melting temperature. The process begins with random samples and iteratively adds selected compounds until all melting-temperature data are included.

  • Kriging principle: Kriging applies GPR to estimate the probability of exceeding the current best target property across possible compounds.The next sampling point is chosen using this probability distribution.
  • Procedure: The initial training set is prepared by randomly choosing compounds.
  • Procedure: Each subsequent compound is selected from unobserved materials using the largest probability of exceeding the current best value.Because this probability increases monotonically with the z-score, the highest-z-score compound is selected.
  • Procedure: The selected compound’s melting temperature is observed and the compound is added to the training data.
  • Procedure: Steps (2)–(4) are repeated until all melting-temperature data are included in the training set.

III. RESULTS AND DISCUSSION

The study constructs prediction models from experimental melting temperatures and compound predictors, including elemental information and DFT-computed physical properties. The dataset contains 248 compounds spanning room temperature to 3273 K, excludes transition metals, and uses specified DFT calculations and crystal-structure inputs.

  • A. Data set: Prediction models are built from experimental melting temperatures and predictor variables.
  • A. Data set: 248 compounds are included, with melting temperatures ranging from room temperature to 3273 K.
  • A. Data set: The dataset excludes transition metals to avoid complexity in the DFT calculations.
  • A. Data set: The DFT-derived physical-property predictors are cohesive energy, bulk modulus, volume, and nearest-neighbor pair distance.These properties are used alongside elemental information as predictor variables.
  • A. Data set: Symmetric elemental predictors are used for binary compounds, yielding 23 total predictor variables.
  • A. Data set: DFT calculations use the PAW method within GGA as implemented in VASP, with convergence thresholds for total energies, forces, and relaxed structures.Crystal structures are taken from ICSD when available, including all listed structures when multiple structures exist.

B. Regressions

The study compares regression models for melting-temperature prediction using elemental predictors alone versus elemental predictors augmented with DFT-computed compound properties. Across the evaluated models, SVR performs best overall, while DFT-derived predictors generally reduce prediction error.

  • Predictor sets: Two predictor sets use elemental information alone or add DFT-computed physical properties of compounds.The second set includes symmetric elemental variables together with computed compound properties.
  • OLSR and PLSR: 473 K and 293 K are the OLSR CV scores for predictor sets (1) and (2), respectively.The corresponding test-data RMS errors are 472 K and 306 K.
  • OLSR and PLSR: 476 K and 291 K are the PLSR CV scores for predictor sets (1) and (2), respectively.These scores are almost the same as those of the OLSR models.
  • Predictor selection: The five-predictor stepwise OLSR model has a 320 K RMS error and uses Ecoh, χA + χB, B, cAcB and rNN.Three of these five predictors are DFT-computed physical properties; the ten-predictor minimum-AIC model has a 295 K RMS error.
  • SVR and GPR: 376 K and 265 K are the SVR CV scores for predictor sets (1) and (2), while GPR obtains 492 K and 334 K.For SVR, the corresponding test-data RMS errors are 364 K and 262 K; for GPR, they are 481 K and 306 K.
  • Model comparison: Among the four regression techniques, SVR provides the best CV scores and test-data RMS errors.With predictor set (2), however, SVR does not improve prediction substantially compared with linear regressions.

C. Prediction

The study predicts melting temperatures for nitrides and Group 14 elements using SVR and OLSR models trained on dataset I, comparing them with experimental data and extrapolated values. Predictions are generally close to experiment, but reliability decreases when predictions require extrapolation beyond the training data.

  • 262 K and 295 K are the RMS errors for test data from the SVR and OLSR models, respectively.The OLSR model uses ten predictors optimized by the stepwise method.
  • SVR and OLSR predictions are close to experimental melting temperatures for most compounds in dataset I.The largest discrepancy occurs for AlN, whose reported experimental values vary widely across databases.
  • Predictions agree well with dataset II when error bars are narrow, as for Mg3N2.Dataset II includes melting temperatures extrapolated from experimental solid-liquid phase boundaries in pressure-temperature phase diagrams.
  • Predictions are less reliable for compounds with wide error bars, such as C and BN, where SVR and OLSR estimates differ greatly.Their larger cohesive energies and bulk moduli place them outside the range represented in dataset I, requiring extrapolation.
  • Including compounds beyond dataset I in the training data should decrease model uncertainty and improve predictive power for extrapolative cases.

D. Kriging

The study uses kriging to search a compound pool for the highest-melting-temperature material and compares it with random compound selection. Across thirty repetitions, kriging required far fewer observed compounds on average.

  • 16.1 compounds were required on average with kriging, compared with 133.4 using random compound selection.Both simulations began with a dataset of 12 compounds and were repeated thirty times.
  • Kriging substantially improved the efficiency of discovering the compound with the highest melting temperature.Figure 7 reports the highest melting temperature among observed compounds as the number of compounds increases.

IV. CONCLUSION

The paper applies regression techniques to predict melting temperatures of single and binary compounds and demonstrates kriging for materials discovery. SVR performs best, especially when DFT-computed physical properties are included, while extrapolation limits predictive power.

  • SVR has the highest predictive power among the four regression techniques evaluated.The four techniques are OLSR, PLSR, SVR, and GPR.
  • 265 K and 262 K are the best model’s CV score and RMS error for test data, respectively.The best model is SVR with elemental information and DFT-computed physical properties as predictors.
  • DFT-computed physical properties improve the prediction models when combined with elemental information.
  • Extrapolation from the training dataset limits predictive power.
  • Kriging found the optimal compound using 16.1 compounds on average over thirty trials, fewer than random compound selection.

Appendix A: Melting temperatures of single and binary component solids

The appendix presents a dataset of melting temperatures for single and binary component solids. Table V contains 248 AxBy binary compounds quoted from Ref. 27.

  • Table V lists melting temperatures for single and binary component solids in the dataset.
  • Table V contains 248 AxBy binary compounds quoted from Ref. 27.
Loading 1310.1546v2…