PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning physical descriptors for materials science by compressed sensing

Luca M. Ghiringhelli, Jan Vybiral, Emre Ahmetcik, Runhai Ouyang, Sergey V. Levchenko, Claudia Draxl, Matthias Scheffler

arXiv:1612.04285v1cond-mat.mtrl-sciphysics.data-an

TL;DR

Separating rocksalt and zincblende structures from chemical formula alone has been a longstanding materials-science problem. The paper uses compressed sensing to discover physical descriptors and build predictive equations; it reports a smallest RMSE of 0.1520 eV and a likely qualitative prediction for extrapolated diamond.

  • Problem

    Separating rocksalt and zincblende structures from chemical formula alone has remained a key materials-science problem for over 50 years.

    Source: §I · p. 4

  • Method

    The paper applies compressed sensing to identify physical parameters that describe materials and their properties, then uses them in quantitative predictive models.

    Source: §VI · p. 34

  • Results

    The model achieves the reported smallest RMSE of 0.1520 eV; its LASSO+ℓ0 extrapolation is judged likely to be at least qualitatively predictive.

    Source: primary features · p. 16, C. Extrapolation: (re)discovering… · p. 30

  • Takeaways & Limitations

    The paper demonstrates a compressed-sensing methodology for discovering physically meaningful descriptors and quantitatively predicting materials-science properties.

    Source: §VI · p. 34

Abstract

from arXiv · show

The availability of big data in materials science offers new routes for analyzing materials properties and functions and achieving scientific understanding. Finding structure in these data that is not directly visible by standard tools and exploitation of the scientific information requires new and dedicated methodology based on approaches from statistical learning, compressed sensing, and other recent methods from applied mathematics, computer science, statistics, signal processing, and information science. In this paper, we explain and demonstrate a compressed-sensing based methodology for feature selection, specifically for discovering physical descriptors, i.e., physical parameters that describe the material and its properties of interest, and associated equations that explicitly and quantitatively describe those relevant properties. As showcase application and proof of concept, we describe how to build a physical model for the quantitative prediction of the crystal structure of binary compound semiconductors.

I. INTRODUCTION

Materials science data offer opportunities for scientific insight, but systematic descriptor selection remains a central challenge for predictive equations. The paper presents a compressed-sensing methodology for finding physically meaningful descriptors and demonstrates it by predicting binary-semiconductor crystal structures.

I. INTRODUCTION

  • Only a few of roughly 200,000 known materials have had basic properties determined, while possible material systems extend practically infinitely.The introduction notes that many relevant material properties remain unmeasured.Source: §I · p. 2
  • Ad hoc descriptor choices lack systematic validation, risking inefficient learning, hidden mechanisms, and unclear prediction reliability.The paper identifies good feature selection as central to a high-quality predictive statistically learned equation.Source: §I · p. 3
  • The proposed methodology discovers physically meaningful descriptors and predicts materials-science properties.The paper frames descriptors as physical parameters describing a material and properties of interest.Source: §I · p. 3
  • The proof of concept quantitatively predicts binary-compound semiconductor structures by modeling the energy difference between rocksalt and zincblende.The sign of the energy difference classifies the structure: negative for rocksalt and positive for zincblende.Source: §I · p. 4
  • Predicting that energy difference requires high accuracy because the rocksalt–zincblende difference can be 0.001% or less of a single atom’s total energy.The paper distinguishes quantitative prediction from classification by sign alone.Source: §I · p. 4

II. COMPRESSED-SENSING BASED METHODS FOR FEATURE SELECTION

This section sets up interpretable linear prediction from descriptor inputs and motivates sparse, low-dimensional models. It also notes that nonlinear transformations of the inputs are central to the paper’s approach.

II. COMPRESSED-SENSING BASED METHODS FOR FEATURE SELECTION

  • The target is an interpretable equation P = f(d) with a small number of input parameters, extended through nonlinear transformations of d.The paper identifies handling these nonlinearities as its main focus.Source: §II · p. 5
  • Least squares fits a linear relationship P = f(d) by minimizing the Euclidean-norm error between outputs and D c.The input matrix D and output vector P define the fit, with coefficient vector c to be determined.Source: §II · p. 5
  • Ridge regression adds a penalty that trades fit error against coefficient magnitude, with larger λ favoring smaller coefficient vectors.As λ approaches zero, its solutions converge to the least-squares solution; as λ grows without bound, they tend to zero.Source: §II · p. 6

A. Sparsity, NP-hard problems, and convexity of the minimization problem

The section formulates feature selection as finding a sparse coefficient vector, but direct minimization becomes computationally infeasible as the feature space grows. The resulting optimization problem is NP-hard.

A. Sparsity, NP-hard problems, and convexity of the minimization problem

  • The desired solution has most coefficients equal to zero, so the model depends on only a small number of descriptor components.The number of nonzero coordinates is denoted by the ℓ0 count, and a vector is k-sparse when that count is at most k.Source: A. Sparsity, NP-hard problems, and… · p. 6
  • Directly searching subsets to minimize the sparse objective is computationally infeasible when M is large.The naive search checks subsets with successively more elements.Source: A. Sparsity, NP-hard problems, and… · p. 6
  • The number of subsets grows quickly with M and sparsity level k, and the sparse minimization problem is NP-hard.The passage characterizes this search approach as not substantially improvable.Source: A. Sparsity, NP-hard problems, and… · pp. 6–7

B. Methods based on the ℓ1 norm

The section presents the ℓ1 norm as a tractable compromise: it keeps the optimization convex while promoting sparse solutions. Its penalty parameter controls the magnitude of the fitted coefficients.

B. Methods based on the ℓ1 norm

  • LASSO replaces ℓ0 regularization with the ℓ1 norm, combining a convex optimization problem with promotion of sparse solutions.The section describes this as a compromise between convexity and sparsity.Source: B. Methods based on the ℓ1 norm · p. 7
  • Increasing λ decreases the ℓ1 norm of the solution, while decreasing λ allows it to grow.The passage identifies a smallest λ threshold above which the solution is zero.Source: B. Methods based on the ℓ1 norm · p. 7

C. Compressed sensing

Compressed sensing (CS) supplies conditions under which sparse coefficient recovery by ℓ1 minimization can match the sparsest ℓ0 solution. Here, CS guides construction of the feature matrix, while LASSO selects a low-dimensional basis from the known materials data.

C. Compressed sensing

  • The ℓ0 problem seeks coefficients c with Dc = P and the fewest nonzero entries; its computational infeasibility motivates a convex reformulation.The ℓ1 problem approximates the ℓ0 problem, but their solutions need not coincide.Source: C. Compressed sensing · pp. 7–8
  • ℓ1 minimization recovers an Ω-sparse coefficient vector exactly if and only if D has the Null Space Property of order Ω.This exact-recovery condition is difficult to check for a specific matrix D.Source: C. Compressed sensing · p. 8
  • CS provides guidance on the admissible dimensions of D, but the supplied bound does not give a directly usable feature-count threshold.The section identifies M with the 82 compounds and N with the input features.Source: C. Compressed sensing · p. 8
  • Highly correlated columns of D can make the ℓ0 and ℓ1 solutions differ, especially when one column lies near the span of a few others.CS frames LASSO as selecting a small basis for known data, while its reconstruction setting emphasizes recovering a signal from few measurements.Source: C. Compressed sensing · pp. 8–9
  • With known material data but freedom to choose the basis, CS guides construction of D and LASSO selects a low-dimensional basis.The data are the energies of 82 materials; D has one row per compound and one column per input feature.Source: C. Compressed sensing · pp. 8–9

D. A simple LASSO example: the energy differences of crystal structures

This section uses crystal-structure energy differences to illustrate LASSO-based modeling and feature selection, including how correlated features can yield a suboptimal descriptor triplet and how explicit nonlinear feature mappings can be introduced before applying linear methods.

D. A simple LASSO example: the energy differences of crystal structures

  • The example targets quantitative DFT-LDA energy differences between ZB and RS structures for 82 octet binary semiconductors, whereas prior work predicted only the sign of ΔE.The model uses free-atom and gas-phase-dimer properties as primary features, including ionization potentials, electron affinities, Kohn–Sham levels, and orbital radii.Source: D. A simple LASSO example: the energy… · pp. 9–10

Appendix A, the two pairs of features are not interchangeable.

  • The zero-coefficient model provides a baseline RMSE of 0.4449 eV for predicting ΔE = 0 across all materials.This is derived from an MSE of 0.1979 eV².Source: Appendix A · p. 11
  • Among six single-feature linear models, rp(A) gives the smallest RMSE, consistent with LASSO selecting it first as the most useful feature.The first nonzero coefficient corresponds to rp(A).Source: Appendix A · p. 12
  • The pair (rp(A), rd(B)) yields the lowest least-squares RMSE among all 15 pairs, with RMSE 0.2927 eV.The second nonzero LASSO component is rd(B), following rp(A).Source: Appendix A · p. 12
  • The exhaustive best triplet, (rs(B), rp(A), rp(B)), achieves RMSE 0.2834, about 2% better than LASSO’s selected triplet (rp(A), rp(B), rd(B)) at 0.2897 eV.The authors attribute the mismatch to Eq. (6) being a convex proxy for the actual problem in Eq. (5).Source: Appendix A · p. 12
  • The exhaustive-search advantage reflects near-collinearity: rs(B) and rp(B) have Pearson correlation 0.996, and the optimal ℓ0 model uses them approximately as their difference.Their linear-expansion coefficients have nearly equal magnitudes, about 1.3, and opposite signs.Source: Appendix A · pp. 12–13
  • With highly correlated features, rs(B) − rp(B) has a coefficient of about 1.3, and L1 penalization can select a suboptimal triplet relative to L0.The large coefficient is penalized by the L1 term, so L1 may poorly approximate L0-regularized minimization in this example.Source: Appendix A · p. 13
  • Only for λ ≤ 0.00079 does the first triplet have smaller LASSO error; with all features included at that λ, every coefficient is nonzero.Source: Appendix A · p. 13
  • John and Bloch31’s descriptor pair has RMSE 0.3055 eV, while (rp(A), rd(B)) predicts the energy difference 4% better.Source: Appendix A · p. 13

E. A more complex LASSO example: non-linear mapping

primary features,

The section builds a feature space from known primary features, uses LASSO to shortlist candidates, and then enumerates subsets because LASSO may miss the optimal descriptor.

primary features,

  • LASSO’s nonzero-feature count can fluctuate as λ decreases because of linear correlations in the feature space.Features 2 and 3, and features 3 and 5, have covariance greater than 0.8, complicating feature identification.Source: primary features · p. 15
  • The pair of the second and third selected features achieved the smallest RMSE of 0.1520 eV, improving on John and Bloch’s descriptors [31].The full search covered 46·45/2 = 1035 pairs.Source: primary features · p. 16
  • LASSO can shortlist Θ relevant features, after which exhaustive search over their Ω-feature subsets identifies the optimal descriptor.The paper notes that LASSO may not directly find the best Ω-dimensional descriptor.Source: primary features · p. 16
  • Combining primary features through analytic formulas offers a strategy for enlarging the feature space, whose construction is not evident a priori.The paper’s approach assumes the basic ingredients, or primary features, are known.Source: primary features · pp. 16–17
  • The 2D model is the same when selected by RMSE or MaxAE, but the 1D models minimizing those errors differ.The section cautions that the 2D agreement need not hold in general.Source: primary features · p. 17

III. GENERATION OF A FEATURE SPACE

The authors construct a physically constrained feature space from atomic inputs and use LASSO plus exhaustive sparse selection to identify descriptors for predicting the RS–ZB energy difference. The resulting two-dimensional descriptor maps the 82 materials using physically meaningful quantities.

III. GENERATION OF A FEATURE SPACE

  • The authors generate as many primary-feature combinations as possible while excluding linear combinations of inhomogeneous quantities; for Ω = 2 and N = 82, they report an M range with a lower end of about 200.Source: §III · pp. 18–19
  • Candidate features are built iteratively from atomic inputs using sums, absolute differences, unary operators, and further combinations.The feature set described contains about 4,500 candidates.Source: §III · p. 19
  • LASSO’s top 30 nonzero features are followed by exhaustive ℓ0 minimization over subsets to select descriptors by lowest RMSE.This avoids brute-force searching all candidate pairs and the need to set a feature-correlation threshold.Source: §III · pp. 19–20
  • On smaller feature spaces where exhaustive pair and triple searches were feasible, the procedure returned the same result.Source: §III · p. 20
  • The procedure identifies best 1D, 2D, and 3D descriptors and equations for predicting the RS–ZB energy difference.The reported equation uses energies in eV and radii in Å.Source: §III · p. 20
  • The structure map places materials at coordinates given by the two descriptor components and shows predicted energy-difference isolevels alongside DFT-LDA reference coding.The isolevels are listed at -0.2, 0, 0.2, 0.5, and 1.0 eV.Source: §III · pp. 20–21
  • The two-dimensional descriptor combines physically meaningful quantities, including B’s band gap and valence s–p orbital size mismatch.Its components define the coordinates of a map of the 82 materials.Source: §III · pp. 20–21

IV. CROSS VALIDATION, SENSITIVITY ANALYSIS, AND EXTRAPOLATION

The authors assess prediction and descriptor stability with repeated leave-10%-out cross-validation, retraining the full selection procedure on each split. The 1D and 2D descriptors are stable, whereas higher-dimensional choices vary despite similar RMSE.

IV. CROSS VALIDATION, SENSITIVITY ANALYSIS, AND EXTRAPOLATION

  • Each leave-10%-out split trains on 75 of 82 data points, and the full LASSO+ℓ0 selection and model fitting are repeated for each split.The random split, training, and error evaluation were typically repeated 150 times.Source: §IV · p. 22
  • Standardization uses only each training set’s mean and standard deviation, preventing test-set information from entering training.Source: §IV · p. 22
  • Increasing descriptor dimensionality and feature-space size eventually reached a plateau with no significant cross-validation RMSE improvement.Source: §IV · p. 23
  • The 1D descriptor matched the all-data choice in 90% of training sets, and the 2D descriptor matched it in every case.Source: §IV · p. 23
  • Higher-dimensional descriptors often differed across training sets, but their RMSE changed little, indicating many competing models.Source: §IV · p. 23

A. Complexity of the feature space

The authors expand the feature space in tiers and find little RMSE gain beyond tier three. Larger tiers can make descriptor selection less stable and increase maximum absolute error.

A. Complexity of the feature space

B. Sensitivity Analysis

The section frames sensitivity analysis as a test of how perturbations affect a learned model and its relationship to the training data. It also asks whether a fitted relationship reflects numerical accuracy limits or a deeper physical relationship.

B. Sensitivity Analysis

1. Noise applied to the primary features

The descriptor is robust to noise on individual primary features, while perturbing all features or the target data can reduce recovery. Target-noise effects depend on whether the materials’ relative ordering is disturbed.

1. Noise applied to the primary features

  • Noise on the nine primary features absent from the 2D descriptor never changes its recovery, while perturbing descriptor features makes recovery noise-dependent.For isolated feature perturbations, the noiseless descriptor is recovered a majority of the time or more up to 5% noise.Source: §1 · p. 26
  • LOOCV RMSE rises from 0.09 eV (σ = 0.001) to 0.12 eV (σ = 0.3), while L-10%-OCV RMSE rises from 0.11 to 0.15 eV.The selected descriptor may vary at the largest noise level, but the authors report only a mild effect on RMSE, consistent with many competitive models in the feature space.Source: §1 · p. 27
  • When all primary features are perturbed, recovery of the 2D descriptor drops quickly, though it occurs almost always at noise levels up to 1%.The authors note that this low level could correspond to computational inaccuracies such as incompletely converged basis sets.Source: §1 · pp. 26, 28

2. Adding noise to P = ∆E

  • Recovery of the unperturbed 2D descriptor remains remarkably stable up to uniform target noise of ±0.1 eV, then drops rapidly.The target-noise test uses two feature spaces of sizes 2924 and 1568.Source: §2 · p. 28
  • Target-noise predictions become less accurate when noise starts mixing the materials’ relative order along the ∆E scale.When the noise preserves that ordering, the errors stay constant.Source: §2 · p. 28

C. Extrapolation: (re)discovering diamond

The extrapolation tests show that the model can qualitatively identify stable zinc-blende compounds, including carbon compounds excluded from training, despite substantial quantitative errors. The authors qualify this evidence as exemplary rather than general.

C. Extrapolation: (re)discovering diamond

  • Prediction errors of 1.2 eV for C and 0.34 eV for BN are large, yet the model still predicts both as the most stable ZB structures.C-diamond and BN were excluded from training in this test.Source: C. Extrapolation: (re)discovering… · p. 29
  • With all four carbon-containing materials excluded from training, the model predicts that they form ZB materials and predicts the relative magnitude of ∆E.The reported prediction errors are relatively large.Source: C. Extrapolation: (re)discovering… · pp. 29–30
  • The authors conclude that LASSO+ℓ0 may have good, at least qualitative, extrapolation capabilities, attributing this to linear-model stability and descriptors containing chemistry of the constituent species.They cite explicit analytic functions of primary features and robustness to perturbations as evidence that the models may have physical meaning.Source: C. Extrapolation: (re)discovering… · p. 30
  • The authors caution that these tests do not support general conclusions about the particularly robust performance of descriptors found by LASSO+ℓ0.They present the tests as a useful basis for assessing a found model’s robustness.Source: C. Extrapolation: (re)discovering… · p. 30
  • The fitted functional relationships cannot automatically be regarded as physical laws, since both the primary features and ∆E are determined by the Kohn-Sham equations.The passage notes that the physically relevant input consists only of atomic charges.Source: C. Extrapolation: (re)discovering… · pp. 30–31

V. COMPARISON TO GAUSSIAN-KERNEL RIDGE REGRESSION WITH VARIOUS DESCRIPTORS

This section evaluates Gaussian-kernel ridge regression across descriptor choices and contrasts its predictive quality and feature handling with LASSO+ℓ0. It also examines alternative and mechanically motivated feature constructions, primary-feature sets, descriptor stability when materials are excluded, and a scikit-learn Lasso implementation.

V. COMPARISON TO GAUSSIAN-KERNEL RIDGE REGRESSION WITH VARIOUS DESCRIPTORS

  • With several atomic-based descriptors, KRR fits reach RMSE levels comparable to or slightly better than the LASSO+ℓ0 fit, yet KRR performance does not improve with descriptor dimensionality.Source: §V · pp. 31–32

A. Prediction test with KRR

  • KRR’s prediction of selected materials depends strongly on the descriptor: extrapolation with descriptor CBN has poor quantitative accuracy, although it correctly predicts C and BN as very stable ZB materials.Other descriptors, including (rs(A), rp(A), rs(B), rp(B)), show good predictive ability in these examples.Source: A. Prediction test with KRR · p. 33
  • KRR is a viable alternative to LASSO+ℓ0 when a good descriptor is identified, but it cannot discard redundant or uninformative components as LASSO+ℓ0 does.Using a mixture of relevant and non-relevant features as a KRR descriptor decreases predictive quality.Source: A. Prediction test with KRR · pp. 33–34

VI. CONCLUSIONS

  • The proposed compressed-sensing method builds a large space of nonlinear functions from physically motivated primary features, then uses LASSO prescreening and ℓ0 minimization to select a few relevant features.The approach can handle linear correlations among features.Source: §VI · pp. 34–35

Appendix A: More details on the construction of the feature space

  • Among tested radii-derived features, the procedure selected features using radii at orbital probability-density maxima rather than average radii, and selected rs, rp, and rd rather than neutral, anion, or cation radii.The appendix compares feature spaces built from both radius sets and their combinations.Source: Appendix A · p. 39
  • Dimer-related features were never selected, and searches including C3, D3, and E3 led LASSO to select descriptor components with exponential forms replacing squared denominators.The descriptor was found through LASSO rather than by directly attempting that substitution.Source: Appendix A · pp. 39–40
  • In tests excluding group G, the best-performing descriptors yielded larger RMSEs than previously found; the 2D model uses A3 features, excludes A2, and has 0.109 eV RMSE versus 0.099 eV for Eq. (21).Source: Appendix A · pp. 40–41

Appendix B: Other feature spaces

  • Appendix B compares feature spaces built from different primary features to underscore the importance of the initial feature set.Once combination rules were established, new numerical tests required only the human time needed to tabulate the primary features.Source: Appendix B · p. 41

1. Primary features including valence and row of the PTE

  • Row and group coordinates alone did not contain enough information to build a predictive model with the algorithm.The test used primary features R(A), R(B), G(A), and G(B), with a feature space of size 1143.Source: §1 · pp. 41–42
  • Replacing |IP(B) − EA(B)| in Eq. (20) with |R(B) − G(B)| while keeping the denominator unchanged yields RMSE 0.13 eV versus 0.14 eV, but MaxAE 0.43 eV versus 0.32 eV.Source: §1 · p. 42

2. Adding John and Bloch’s rσ and rπ as primary features

  • Adding rσ and rπ to the primary features yielded the same 2D descriptor as Eq. (21), while L-10%-OCV selected a descriptor containing either quantity in about 10% of cases.The feature space was reduced because adding all combinations made the procedure unstable with only 82 data points.Source: §2 · p. 43

as primary features

  • The mechanical-stability feature space explored whether the RS–ZB energy difference depends on the stability of constituent structures, using elastic constants for tetrahedral AB4 and BA4 pentamers.The setup considered symmetric and antisymmetric stretching deformations.Source: as primary features · p. 43
  • The six primary features comprise four α features from AB4 and BA4 molecules and two α + 4β combinations; both atoms’ information is intermingled in them.Source: as primary features · p. 44
  • Feature construction uses operations such as A + B and |A − B|, but does not allow arbitrary linear combinations of the primary features.Source: as primary features · p. 44

Appendix C: Numerical tests excluding selected elements

  • Excluding C-diamond or C-diamond together with BN marginally reduces errors but changes the descriptor; the 3D descriptor remains remarkably stable in both tests.The tests used L-10%-OCV on the remaining materials.Source: Appendix C · pp. 45, 47, Table XIX · p. 46
  • Removing all carbon compounds produces behavior similar to the C-diamond exclusions, while carbon is an anomaly that also carries important information for overall-fit stability.Source: Appendix C · p. 47
  • Removing BN or all boron compounds leads to a fit similar to case 0).Source: Appendix C · p. 47

library

  • The library example fits scikit-learn’s Lasso to the data, scaling the ℓ2-norm by 1/(2N) and including the bias c0 by default.Source: library · p. 47

Related papers

Loading 1612.04285v1…