PaperlayerView exact revision on arXiv

Source-linked AI summary

MODNet -- accurate and interpretable property predictions for limited materials datasets by feature selection and joint-learning

Pierre-Paul De Breuck, Geoffroy Hautier, Gian-Marco Rignanese

arXiv:2004.14766v3cond-mat.mtrl-sci

TL;DR

Machine-learning models for materials properties often need more data than is available for computationally demanding problems. MODNet combines physically meaningful feature selection with a feedforward network and joint learning; it reports strong vibrational-entropy accuracy on a holdout set and improved performance from feature selection and joint learning.

  • Problem

    Graph-network accuracy depends on data quantity, yet computationally demanding materials properties often have limited datasets.

    Source: §I · p. 2

  • Method

    MODNet combines a feedforward neural network, relevance-redundancy feature selection, and a hierarchical architecture that jointly learns across properties.

    Source: §I · p. 3, A. The MODNet model · pp. 5–6

  • Results

    The vibrational-entropy MAE on S305K was 8.9 µeV/K/atom, four times lower than Legrain et al. and 25 times lower than Tawfik et al..

    Source: §III · p. 16

  • Takeaways & Limitations

    Feature selection highlights important complementary variables, including bond length and valence range for vibrational entropy and band-gap estimates and density for refractive index.

    Source: §III · p. 16

Abstract

from arXiv · show

In order to make accurate predictions of material properties, current machine-learning approaches generally require large amounts of data, which are often not available in practice. In this work, an all-round framework is presented which relies on a feedforward neural network, the selection of physically-meaningful features and, when applicable, joint-learning. Next to being faster in terms of training time, this approach is shown to outperform current graph-network models on small datasets. In particular, the vibrational entropy at 305 K of crystals is predicted with a mean absolute test error of 0.009 meV/K/atom (four times lower than previous studies). Furthermore, joint-learning reduces the test error compared to single-target learning and enables the prediction of multiple properties at once, such as temperature functions. Finally, the selection algorithm highlights the most important features and thus helps understanding the underlying physics.

I. INTRODUCTION

Machine learning can accelerate materials screening, but graph-model accuracy depends on data that is often scarce for computationally demanding properties. MODNet addresses this limited-data challenge by combining physically meaningful features, feature selection, and joint learning.

I. INTRODUCTION

  • Materials design matters to technological applications, and machine learning has emerged as a useful technique in materials science.Source: §I · p. 2
  • Surrogate models can predict complex properties in a fraction of the time of conventional quantum methods with almost the same accuracy, enabling faster materials screening.Source: §I · p. 2
  • Materials machine learning includes case-specific models with tailored descriptors and general graph models that process crystal structures through convolutional layers.Examples of graph models include CGCNN [10] and MEGNet [11].Source: §I · p. 2
  • Graph-model accuracy depends strongly on data quantity, while computationally demanding properties often have limited datasets.Examples include GW band gaps for 80 crystals, lattice thermal conductivity for 101 compounds, and vibrational properties for 1245 materials.Source: §I · pp. 2–3
  • MODNet combines physically meaningful features, feature selection, and joint learning to predict solid properties from small datasets and help identify important features.The authors place it among approaches bridging case-specific models and general graph models.Source: §I · p. 3

A. The MODNet model

MODNet combines a feedforward neural network with physically informed descriptors selected for relevance and low redundancy. Its hierarchical multi-property architecture shares representations across related targets, supporting predictions of parameter-dependent properties such as temperature curves.

A. The MODNet model

  • MODNet uses a feedforward neural network with selected descriptors, reducing the optimization space without requiring massive data.The descriptors encode physical, chemical, and geometrical information; the schematic describes a material encoder followed by target-specific decoding and re-encoding blocks.Source: A. The MODNet model · pp. 3–4
  • Feature selection begins with the descriptor having the highest NMI with the target, then selects features using a relevance-redundancy score.The selection stops at a chosen or error-optimized feature-count threshold; for multiple targets, it takes the union of relevant features.Source: A. The MODNet model · p. 5
  • The selection score balances relevance and redundancy with hyperparameters p and c, which are varied dynamically as the selected-feature set grows.The procedure is described as similar to mRMR [23], more flexible through p and c, less computationally expensive than CFS, and provides a global ranking.Source: A. The MODNet model · p. 5
  • Joint learning across multiple properties can support temperature-curve prediction, with the vibrational-property hierarchy sharing thermodynamic representations across several property and temperature predictors.The described targets include specific heat, entropy, enthalpy, and energy at various temperatures.Source: A. The MODNet model · p. 6
  • The hierarchical network shares early layers across properties and branches into target-specific predictors, using more samples to optimize shared layers and limiting overfitting.The authors report slightly improved accuracy over single-target prediction.Source: A. The MODNet model · p. 6

B. Performance assessment

MODNet is evaluated on single-property and multi-property prediction, with performance depending on dataset size and model design. The results also examine feature selection and predictions of temperature-dependent vibrational properties.

B. Performance assessment

  • Below ~4,000 training samples, MODNet outperforms MEGNet and SISSO; on large formation-energy and band-gap datasets, MEGNet has the lowest prediction error.The authors recommend feature-based models for small-to-medium datasets and graph-based models for large datasets.Source: B. Performance assessment · pp. 7–8
  • Joint-learning yields an approximately 8% accuracy gain over single-target learning, with a larger gain at lower training sizes.The reported lowest mean absolute error and variance are systematically found for MODNet models.Source: B. Performance assessment · p. 8
  • Against the AllNet baseline, feature selection’s performance gain decreases with training size: it is approximately 12% at 200 training samples and approximately 5% at 1,000 training samples.Source: B. Performance assessment · p. 9
  • For vibrational properties, m-MODNet predicts entropy, enthalpy, specific heat, and Helmholtz free energy at 25, 305, and 705 K, with errors summarized as MAE and MaxMAE.The reported comparison includes typical dataset values alongside errors; MaxMAE is the mean over the worst 5% of predicted samples.Source: Table II · p. 10, B. Performance assessment · p. 10
  • The benefit of sharing representations depends on architecture: separating dissimilar properties too little limits flexibility, while separating them too much removes shared representations that may aid generalization.The authors treat architecture distance as a hyperparameter because validation error has a minimum as distance increases.Source: B. Performance assessment · pp. 10–11

C. Feature selection

Feature-selection performance depends on how many features are chosen: MOD-selection performs best beyond 10 features, while SISSO is strongest when only a few are selected. Selection also improves accuracy and makes influential features interpretable.

C. Feature selection

  • Feature selection yields an average 12% error improvement, with gains especially pronounced on small datasets and expected to diminish as training size increases.The reported mechanism is reduced noise-to-signal ratio from removing irrelevant features.Source: C. Feature selection · p. 12
  • MOD-selection favors relevance while accounting for redundancy, and its selected features can reveal interpretable relationships with target properties.For vibrational entropy, AGNIFingerprint relates to inverse bond length, and a larger range of p-valence electrons is associated with higher entropy.Source: C. Feature selection · p. 13
  • With only the first 10 features and 1100 training samples, SISSO outperforms MOD-selection on vibrational entropy at 305 K.Source: C. Feature selection · p. 14, Figure 5 · p. 14
  • MODNet outperforms other selection algorithms, particularly at low sample sizes, while MOD-selection performs best when selecting more than 10 features.OMP performs poorly in this setting; SISSO is best when only a few features are selected but becomes computationally unaffordable above 10.Source: C. Feature selection · pp. 14–16, Figure 5 · p. 14
  • The 300-feature Jaccardian-similarity experiment finds that all methods’ selected-feature similarity changes by over 40% between 200 and 1000 samples, while OMP has low similarity across subset sizes.Higher similarity indicates convergence toward the features selected on the total dataset, but does not establish that those features are worthwhile.Source: C. Feature selection · p. 15

III. DISCUSSION

MODNet combines physically meaningful features, feature selection, and joint learning to improve predictions on limited materials datasets. Its feature selection also identifies variables associated with the underlying physics.

III. DISCUSSION

  • The multi-property m-MODNet provides a single model for multiple properties, speeding training and prediction.The approach also supports parameter-dependent functions such as temperature functions.Source: §III · pp. 16–17
  • On 200 samples, feature selection improved vibrational-thermodynamics performance by 12%, while joint learning added an 8% improvement on S305K.The 8% improvement is attributed to MODNet’s joint-learning mechanism.Source: §III · p. 16
  • 8.9 µeV/K/atom MAE and 12.0 µeV/K/atom RMSE were achieved for vibrational entropy on S305K’s 145-material holdout test set.The MAE is reported as four times lower than Legrain et al. and 25 times lower than Tawfik et al..Source: §III · p. 16
  • Feature selection identifies important complementary variables, including bond length and valence range for vibrational entropy and band-gap estimates and density for refractive index.The authors connect the vibrational-entropy valence range to bond ionicity.Source: §III · p. 16
  • Although this work predicts properties from structural primitives, MODNet can also be extended to composition-only tasks.The authors point to their GitHub repository.Source: §III · p. 16
  • Feature-based methods are more powerful below approximately 4,000 samples, while graph-based models are more powerful on larger datasets.The authors describe graph networks as powerful on large datasets but not scaling well on smaller ones.Source: §III · pp. 16–17

A. Datasets

The study uses datasets for formation energy, band gap, vibrational thermodynamics, and refractive index, with different sample sizes and coverage.

A. Datasets

  • 69,640 Materials Project crystals form the formation-energy and band-gap datasets, with 45,901 finite-band-gap crystals used for non-zero band-gap regression.The crystal set matches the MP-crystals-2018.6.1 dataset used for MEGNet benchmarking.Source: A. Datasets · p. 17
  • 1,245 inorganic compounds provide vibrational entropy, Helmholtz free energy, internal energy, and heat capacity values from 5 to 800 K in 5 K steps.The properties were computed using density functional perturbation theory in the harmonic approximation.Source: A. Datasets · p. 17
  • 4,040 compounds make up the refractive-index dataset, whose typical values range from 1 to 6, with 60% below 2.The calculations use density functional theory and high-throughput methods.Source: A. Datasets · p. 18
  • The thermodynamic and refractive-index datasets span multiple symmetry groups, space groups, and elements, but the refractive-index set is biased toward oxides.84% of refractive-index materials contain at least one oxygen atom.Source: A. Datasets · p. 18

B. Model training

Model training uses separate training, validation, and test sets, with hyperparameter tuning on validation data and test data reserved for generalization assessment.

B. Model training

  • Validation tunes hyperparameters and the test set estimates generalization; small datasets use 10-fold validation, and MAE is the standard criterion except for vibrational thermodynamics.A wider set of metrics is used for vibrational thermodynamics to assess its multiple learned properties.Source: B. Model training · p. 18
  • MODNet tunes the number of optimal input features in addition to neural-network depth, width, learning rate, batch size, activation, and loss.The final vibrational-property model uses two layers per block and 256, 128, 64, and 8 neurons in successive blocks.Source: B. Model training · pp. 18–19
  • The final vibrational-property MODNet uses four blocks, with formation energy added through a second-order block to the first block.The network uses min-max preprocessing, MSE loss, learning rate 0.01, batch size 256, and 600 Adam epochs.Source: B. Model training · p. 19, Figure 6 · p. 19

V. DATA AVAILABILITY

The paper identifies locations for generated features and datasets used in its analyses.

V. DATA AVAILABILITY

  • Generated features, NMI, and MP-2018.6 datasets are available through the listed Figshare project.The vibrational-thermodynamics and refractive-index datasets are available from Refs. and [30], respectively.Source: §V · pp. 19–20
  • The vibrational-thermodynamics dataset is available from Ref..Source: §V · p. 20
  • The refractive-index dataset is available from Ref. [30].Source: §V · p. 20

VIII. AUTHOR CONTRIBUTIONS

P.-P. D.B. designed, implemented, and benchmarked the model; P.-P. D.B., G. H., and G.-M. R. conceived the project and prepared the manuscript, with G.-M. R. supervising.

VIII. AUTHOR CONTRIBUTIONS

  • P.-P. D.B. designed, implemented, and benchmarked the model, while G.-M. R. supervised the project.P.-P. D.B., G. H., and G.-M. R. conceived the project and prepared the manuscript.Source: §VIII · p. 20

XI. FIGURE LEGENDS

The figure legends describe MODNet's feature-selection and hierarchical-network schematic, alongside captions for vibrational-entropy comparisons, temperature-dependent predictions, selected features, and model architecture.

XI. FIGURE LEGENDS

  • The schematic follows matminer feature selection with a hierarchical tree-like network that groups related properties and encodes materials before target-specific decoding and re-encoding.Properties may be parameter-dependent, with values available across different parameter settings.Source: §XI · pp. 23–24, Figure 1
  • The vibrational-entropy caption describes absolute-error distributions at 305 K across three training sizes and strategies, reporting the mean (MAE) and variance.It specifies Gaussian-kernel density estimation and gives the error units as µeV/K/atom.Source: §XI · p. 24
  • For Li2O, the figure compares MODNet predictions with DFPT values for thermodynamic quantities across temperature, with sample errors close to the test-set MAE.The caption identifies the material as Li2O (MPID: mp-1960).Source: §XI · p. 24
  • The feature-importance figure covers vibrational entropy at 305 K, refractive index, formation energy, and band-gap energy, with complementary features that are insufficient alone for accurate estimation.Source: §XI · p. 24
  • The selection-algorithm figures compare test error across selected-feature counts and training sizes, and measure Jaccardian similarity against the full dataset.The compared selectors are Pearson correlation, NMI, Random Forest, SISSO, and OMP.Source: §XI · p. 24
  • The vibrational-properties architecture uses four MODNet blocks, with formation energy added through a second-order block to the first block.Source: §XI · p. 25
Loading 2004.14766v3…