Source-linked AI summary
Opportunities and Challenges for Machine Learning in Materials Science
Dane Morgan, Ryan Jacobs
TL;DR
Materials science needs ways to use rapidly expanding machine-learning opportunities while judging model accuracy and applicability. This review surveys major ML applications, then examines supervised property-prediction workflows and common approaches to model assessment and uncertainty. It concludes that ML infrastructure and methods are advancing quickly, but error estimates are most reliable for data similar to the analyzed database and domain applicability remains important.
Problem
Rapid changes in materials-science ML make it difficult to understand the field’s broad opportunities and establish best practices for assessing model accuracy and applicability.
Method
The review surveys major ML application areas and discusses feature engineering, model development, uncertainty assessment, and domain applicability for supervised property prediction.
Results
ML is impacting diverse materials-science applications, while common assessment approaches provide useful error estimates but require attention to data similarity and model domain.
Takeaways & Limitations
Materials researchers can draw on expanding computing, software, data, and community infrastructure, while using model-development and assessment practices suited to their application.
Takeaways & Limitations
Resampling-based error estimates are expected to be accurate only for new data sufficiently similar to the analyzed database, and resampling does not clearly establish that similarity.
Abstract
from arXiv · showhide
Advances in machine learning have impacted myriad areas of materials science, ranging from the discovery of novel materials to the improvement of molecular simulations, with likely many more important developments to come. Given the rapid changes in this field, it is challenging to understand both the breadth of opportunities as well as best practices for their use. In this review, we address aspects of both problems by providing an overview of the areas where machine learning has recently had significant impact in materials science, and then provide a more detailed discussion on determining the accuracy and domain of applicability of some common types of machine learning models. Finally, we discuss some opportunities and challenges for the materials community to fully utilize the capabilities of machine learning.
1 Introduction
ML is expanding rapidly in materials science and engineering, supported by a growing ecosystem of computing, data, software, and community initiatives. This review surveys major application areas and examines model accuracy and applicability.
- 1 Introduction: ML includes unsupervised methods that learn data structure without human labels and supervised methods that typically use labeled data.Examples include clustering and principal component analysis for unsupervised learning, and image labeling for supervised learning.
- 1 Introduction: Billions of dollars per year in ML infrastructure investment have created opportunities for MS&E researchers to use advanced tools with modest investment.Open-source algorithms and relatively inexpensive computing resources, including GPUs, are widely available.
- 1 Introduction: Materials data from new instruments, simulations, and cloud resources can be integrated with the expanding ML ecosystem.Community initiatives including ICME, MGI, NOMAD, MaX, and MGE support MS&E computation and data infrastructure.
- 1 Introduction: ML research in materials science and engineering has grown rapidly, particularly since 2014.The publication trend is described as exponential, with an estimated doubling about every 1.6 years.
- 1 Introduction: The review combines a high-level overview of ML applications with detailed guidance on supervised property-prediction models.It focuses especially on model development, assessment, accuracy, and domain of applicability.
2 Some notation
The review represents supervised regression data as feature matrix X paired with target vector Y, then uses fitted models to map new features to predicted targets. Model type, parameters, and hyperparameters distinguish the model form, fitted values, and tuning choices.
- 2 Some notation: Supervised regression data are represented as a feature matrix X paired with a target-value vector Y.Each row of X describes one modeled system, such as a material structure and composition.
- 2 Some notation: A fitted model F+ maps new feature inputs X* to predicted target values Y+*=F+(X*).The notation is used for property prediction on new materials data.
- 2 Some notation: Model type specifies the functional form, while model parameters are fitted during training and hyperparameters control model development choices.Examples of model types include linear regression and neural networks.
3 Where and how is ML impacting MS&E?
Machine learning is being applied across MS&E to predict properties, discover and design materials, guide autonomous experiments, analyze characterization data, extract synthesis knowledge, and accelerate molecular simulations. These applications include active learning, generative models, machine vision, natural-language processing, and machine-learned potentials, while challenges remain in feature representation, data quality, and applicability assessment.
- Property prediction and materials discovery and design: ML predicts materials properties from existing databases, enabling rapid exploration of new systems and iterative optimization through active learning.The approach regresses properties on material feature vectors, but choosing those representations remains a critical challenge.
- Property prediction and materials discovery and design: ML has been used to predict stability, formability, superconducting temperatures, melting points, dielectric properties, bandgaps, and diffusion barriers across diverse materials classes.Reported applications span oxides, perovskites, polymers, intermetallics, metallic glasses, semiconductors, and metallic hosts.
- Property prediction and materials discovery and design: Active learning balances exploitation of promising predictions with exploration of under-sampled, uncertain regions to guide experiments or simulations toward target outcomes.Applications include a lead-free piezoelectric with the largest measured electrostrain in its BaTiO3 family and high-glass-transition-temperature polymers from five training materials.
- Property prediction and materials discovery and design: ML-guided autonomous experimentation combines robotic high-throughput synthesis with on-the-fly decisions, enabling guided searches with limited human intervention.For polyoxometalate crystallization, a machine-based search covered about 6 times more parameter space and achieved prediction accuracy about 5% higher than humans.
- Property prediction and materials discovery and design: Generative models support inverse materials design by suggesting and evaluating candidates from desired characteristics, including stable hydrides and high-melting-point organic molecules.CrystalGAN generated and screened hydrides for hydrogen storage, while ORGANIC predicted high-melting-point organic molecules; RANC outperformed ORGANIC for novel-molecule discovery.
- Materials characterization: ML processes increasingly large and complex characterization data, matching or exceeding domain-specific experts for some microscopy defect-identification tasks.Applications include image denoising, correction, clustering, spectral demixing, microstructure classification, defect identification, and X-ray diffraction analysis.
- Materials characterization: Natural-language processing extracts chemical entities, synthesis parameters, structure-property relationships, and writing practices from scientific publications.Tools such as ChemDataExtractor parse chemical formulas and domain-specific terms, while studies have provided synthesis guidance for nanotubes, perovskites, and oxide systems.
- Molecular simulation: Machine-learned potentials and charge-density models accelerate molecular and materials simulations while retaining reported accuracy for selected systems.Examples include accelerated AIMD for aluminum, accurate energetics for silicon surface reconstructions, models with up to 11 elemental components, and DFT-level charge-density predictions at orders-of-magnitude greater speed.
4 Some challenges and best practices for ML in MS&E
Reliable materials-science ML requires careful workflow design, validation, uncertainty assessment, and attention to applicability domain. The review emphasizes nested cross-validation, feature and model choices, and the limits of resampling-based error estimates.
- ML workflow: A supervised-regression workflow spans data preparation, feature engineering, model fitting and optimization, uncertainty and applicability assessment, and final prediction.These stages frame the review’s best-practice discussion.
- Data splitting: Training data fit model parameters, validation data select models and hyperparameters, and testing data independently assess the final optimized model.Test data should remain unused during optimization to provide a meaningful final-model error estimate.
- Cross-validation: Nested cross-validation repeats model development and testing across splits, reducing bias from relying on one fixed train-validation-test partition.Level 1 splits act as test sets, while nested level 2 splits optimize the model using the included data.
- Applicability and uncertainty: Applicability domains are best framed as data regions where prediction uncertainty can be quantified at a desired level, not merely where errors are assumed acceptable.Feature-distance methods provide only qualitative guidance because their metrics and cutoffs may be problem-dependent.
- Cross-validation: k-fold cross-validation with k near 10 best matched test-data error in the illustrated example, while smaller k generally increases error through smaller training sets.The review describes k values from 3–10 as a commonly useful compromise.
- Limitations: Resampling errors may be overly optimistic when validation points resemble training points, and small datasets warrant checks against simple naïve references.These limitations tie estimated accuracy to the analyzed database and its similarity structure.
- Applicability and uncertainty: GPR and RFDT uncertainty estimates behaved differently: GPR errors were accurate on average but weakly correlated pointwise with residuals, whereas RFDT errors correlated strongly up to a transition region.For RFDTs, predicted-error behavior became comparatively noisy above roughly the dataset standard deviation.
5 Machine learning tools and software for materials
Open-source software packages are streamlining ML adoption in materials science and engineering, supporting both community best practices and ease of use.
- Open-source ML software packages are increasingly aimed at streamlining and accelerating materials informatics research.The review provides a list of packages and the ML-related analyses they enable.
6 Future opportunities and ongoing challenges of ML in MS&E
ML opportunities in MS&E include building shared infrastructure, improving data practices, and changing how scientific research is conducted. The authors argue that these developments remain at an early stage despite substantial progress.
- A living informatics ecosystem could unify materials data, MS&E-focused ML tools, and model generation, analysis, and dissemination.The proposed infrastructure is intended to support automated workflows and algorithm experimentation, particularly for researchers who are not ML specialists.
- Effective model infrastructure should make ML models discoverable, reproducible, reusable, and accessible through both human and machine interfaces.The passage specifically identifies application programming interfaces as a route to incorporating models into complex workflows.
- Curated open data and consistent metadata can reduce data siloing and enable more informed comparisons across materials datasets.The authors also identify unreported null results as valuable training information that can reduce bias toward traditionally reported outcomes.
- Automation and AI are beginning to shift laboratory scientists toward analyzing large volumes of automatically produced data.The review presents autonomous robot scientists as an early example of this change.
- The full potential of ML and AI in MS&E remains far from realized, although their interaction is already changing materials discovery, design, and deployment.The conclusion characterizes this interaction as being in its nascent stages.
7 Data Availability
The review identifies its supporting datasets and source materials as available through Figshare, while reporting no conflicts of interest.
- The diffusion activation energy dataset used in Section 4.4.4 is available on Figshare.The dataset was taken from the work of Lu et al.
- Training, validation, and testing subsets, original figure data, and the current supporting information are available on Figshare.
- The authors report no affiliations, memberships, funding, or financial holdings known to affect the review’s objectivity.
8 Supplemental Section 1: Recent reviews of ML in MS&E
The supplement compiles recent reviews, perspectives, tutorials, and related resources covering diverse applications of ML and materials informatics. It also lists software and reviews spanning materials discovery, modeling, characterization, manufacturing, and related domains.
- The resource list reflects the rapid growth of reviews, progress reports, perspectives, and tutorials on ML in MS&E.The authors state that numerous such publications appeared within the preceding few years.
- The cited literature covers materials discovery, design, property prediction, and learning accelerated discovery.Examples include reviews by Liu et al., Tanaka, and Yosipof et al.
- Additional reviews address atomistic calculations, computational chemistry, quantum properties, interatomic potentials, and molecular dynamics.The list includes work by Ward and Wolverton, Goh et al., Lilienfeld, Chan et al., and Lamoureux et al.
- The supplement describes open-source software as increasingly important for maintaining best practices and easing ML adoption in MS&E.It organizes software into eight groups, ranging from ML environments to MS&E-specific tools.
10 Supplemental Section 3: Journals publishing ML in MS&E studies
The section identifies journals that frequently publish machine-learning studies in materials science and engineering, including papers with a methodological focus.
- Some journals are particularly well represented in machine-learning studies in materials science and engineering.
- These journals may publish papers with a relatively methodological focus.
- Such papers may contain limited new materials insights.
11 Supplemental Section 4: Types of Machine Learning Models
The section reviews standard machine-learning models used in materials science and engineering, then discusses baseline checks and methods for assessing prediction errors and applicability domains. It emphasizes model choice, uncertainty calibration, and identifying predictions that may fall outside the model domain.
- Model formulation: The review formulates supervised regression as learning a function F(X) from feature matrix X and target vector Y, with noise ε.Predictions are written as Y+ from the fitted model F+(X).
- Model types: MVLR provides a rapid, robust baseline, but nonlinear materials-property behavior can make it inaccurate, overfit, or unreliable for shifted data.The section therefore says MVLR is typically not the method of choice for most MS&E machine-learning problems.
- Model types: RFDTs form ensembles of decision trees using bootstrap sampling and altered split criteria, producing prediction distributions whose moments estimate values and variance.
- Model types: Deep neural networks reduce complex inputs into essential features and use highly flexible weights that can support multiple prediction tasks.The weights may number in the millions, requiring extensive training.
- Model checks: Permuting target values provides an overfitting check; the permuted model should perform substantially worse, ideally with RMSE/s » 1 and R2 » 0.
- Error assessment: GPR and RFDT r-distributions broadly match a normal reference, but GPR has about 2% of errors above three predicted standard deviations and RFDT error bars are often too large for r < 2.The analysis excludes errors beyond 2 3 × σM/IDO to avoid values likely outside the model domain.