Recent Advances and Applications of Deep Learning Methods in Materials Science

• Linking material structures and phenomena across atomic, microscale, mesoscale, image, and spectral representations remains challenging because their length and time scales vary substantially.Source: §1 · p. 2
• The exponential growth of experimental and computational materials datasets, supported by MGI and FAIR principles, creates a need for automated analysis using machine learning.Source: §1 · p. 2
• Deep learning can extract higher-level features from raw data, complement physics-based methods, and remain feasible for small datasets through transfer learning, multi-fidelity modelling, and active learning.Source: §1 · pp. 2–3
• Deep learning can substantially accelerate conventional scientific computing and, in some applications, achieve accuracy comparable to physics-based or computational models.Source: §1 · p. 3
• Despite these advantages, deep learning’s black-box nature and often thousands to millions of parameters can hinder physical insight and make interpretation difficult.Source: §1 · p. 3
• The review presents deep-learning principles, recent materials-science application trends, and evolving publicly available tools and datasets through an updateable GitHub repository.Source: §1 · p. 4

2.1 General machine learning concepts

• Deep learning is a subset of machine learning within artificial intelligence that uses multilayer neural networks, especially for unstructured data where manual feature extraction is time-consuming, brittle, and difficult to scale.Traditional machine learning commonly trains generalized models on structured data, whereas deep learning learns representations from inputs such as images, sounds, text, and graphs.Source: §2.1 · p. 4, Figure 1 · p. 5

2.2.1 Perceptron

• A perceptron is the basic neural-network unit: it multiplies inputs by weights, incorporates biases, sums them, and forwards the result; common training packages include PyTorch, Tensorflow, and MXNet [56].Artificial neural networks can approximate functions to arbitrary accuracy according to the universal approximation theorem [50, 51].Source: §2.2.1 · p. 4, §2.1 · p. 4

2.2.2 Activation function

• Activation functions such as sigmoid, tanh, ReLU, leaky ReLU, and Swish provide nonlinear transformations that let neural networks learn complex functions.Sigmoid maps real numbers to (0, 1) and is often used in binary-classifier output layers; activation choice affects training efficiency and final accuracy.Source: §2.2.2 · p. 5

2.2.3 Loss function, gradient descent and normalization

• Neural networks minimize a task-specific loss by iteratively updating weights with gradient descent and back-propagation, using automatic differentiation to compute parameter derivatives.Common optimizers include SGD, Adam, and Adagrad; BCE and NLLL suit classification, while MSE suits regression, and normalization improves training stability.Source: §2.2.3 · p. 6

2.2.4 Epoch and mini batches

• Training proceeds over epochs, with each epoch passing through the full dataset while mini-batches make gradient computation practical for large deep-learning datasets.Mini-batches are small subsets used for forward passes before repeated updates across multiple epochs until weights converge.Source: §2.2.4 · p. 6

2.2.5 Underfitting, overfitting, regularization and early stopping

• Training, validation, and test sets separate model fitting from evaluation, while underfitting reflects insufficient capacity and overfitting reflects poor validation performance despite strong training performance.Regularization methods such as dropout and early stopping improve generalization by discouraging memorization and halting training when validation accuracy flattens or decreases.Source: §2.2.5 · p. 6

2.3 Convolution neural networks

• CNNs combine learnable convolution filterbanks, nonlinear activations, spatial coarsening, and a prediction module operating on a global representation.Convolutional filters use shared trainable parameters and can be one-, two-, or three-dimensional.Source: §2.3 · pp. 6–7
• CNN filters slide across inputs to automatically extract local high-level features as feature maps, which are passed to hidden or fully connected layers.The filters convolve with fixed strides, using a relatively small number of rows and columns.Source: §2.3 · pp. 6–7
• Nonlinear activations such as ReLU support complex data, while pooling preserves spatial invariance, downsamples feature maps, and reduces dimensionality.Pooling may be maximum, minimum, average, or sum pooling; CNN outputs are typically reduced to a one-dimensional global representation.Source: §2.3 · p. 7

2.4.1 Graphs and their variants

• CNNs are based on regular-grid Euclidean data, whereas many real-world structures, including atomic and molecular systems, are non-Euclidean and motivate graph-based representations.Examples also include social networks, image segments, word-vectors, and recommender systems.Source: §2.4.1 · p. 7
• A graph G = (V, E, X) represents non-Euclidean data with nodes, edges, and node features, while adjacency matrices encode node connections.Nodes and edges may have attributes, and graphs may be directed or undirected, weighted or unweighted, homogeneous or heterogeneous, static or dynamic, or multigraphs.Source: §2.4.1 · p. 7

2.4.2 Types of GNNs

• GNNs operate on graph domains and capture graph dependencies through message passing, first aggregating neighbor information and then updating nodes and/or edges.Aggregation is permutation invariant, and propagation multiplies input node features by adjacency and weight matrices before nonlinear activation.Source: §2.4.2 · pp. 7–8
• Propagation and aggregation choices yield GCN, GAT, Relational-GCN, GRN, GIN, and LGNN variants, with graph convolutional neural networks identified as the most popular.GNNs are described as methods for predicting materials properties from structural information.Source: §2.4.2 · p. 8

2.5 Sequence-to-sequence models

• Sequence-to-sequence models replace fixed-length representations with contextual processing and outputs of arbitrary length.This contrasts with bag-of-words representations, which count words into fixed-length vocabulary vectors.Source: §2.5 · p. 8
• In named entity recognition, sequence-to-sequence models map word sequences such as chemical abstracts to category sequences assigning every word an entity label.Source: §2.5 · p. 8
• RNNs use feedback connections and shared weight matrices, with common variants including GRURNN, LSTM, and CW-RNN.Source: §2.5 · p. 8
• Traditional RNNs are difficult to parallelize and preserve long-range context because of vanishing gradients, although LSTMs support materials-domain NER applications.Source: §2.5 · p. 8
• Transformer models such as BERT [70] use attention to access previous words simultaneously, improving parallelization and long-term context preservation.Source: §2.5 · pp. 8–9

2.6 Deep generative models (VAE and GAN)

• Generative models create varied samples for augmentation, learn generalized latent features, and support applications such as virtual try-on.VAEs explicitly define and learn data likelihood, whereas GANs directly generate samples from a learned distribution without defining a density function.Source: §2.6 · p. 9

2.7 Deep reinforcement learning

• Deep reinforcement learning combines deep learning with trial-and-error decision-making, using high-dimensional states such as images or spectra as inputs.Agents act within Markov decision processes to receive scalar rewards and learn policies maximizing returns; architectures may be model based or model free.Source: §2.7 · p. 9

3 Applications of DL methods

• Successful materials-science DL applications require large, balanced, diverse datasets, suitable DL approaches and input representations, and performance metrics aligned with scientific goals.Datasets often contain 10000 data points or more; repositories and datasets support reproducibility and extensibility.Source: §3 · p. 9

3.1 Atomistic and chemical representations

• Graph neural networks have surpassed earlier hand-crafted atomistic descriptors, enabling deep-learning workflows for force fields, property prediction, and materials screening.The review also covers generative adversarial and complementary methods for atomistic applications.Source: §3.1 · p. 10

3.1.1 Databases and software libraries

• Atomistic databases and software packages span molecular, solid-state, and protein systems, while incorporating bond angles into graph neural networks can substantially improve performance at comparable computational cost.Such databases often take years to generate with conventional methods, whereas deep learning offers lower-cost predictions with reasonable accuracy.Source: §3.1.1 · p. 10

Force field development

• Force-field models predict interactions hundreds to thousands of times faster than conventional approaches, but require large datasets generated with computationally expensive methods.Neural and graph-based force fields extend simulations beyond 0 K, while hybrid models combine neural networks with traditional potentials to mitigate extrapolation issues.Source: Force field development · pp. 10, 12

Direct property prediction from atomistic configurations

• ALIGNN achieves mean absolute errors of 0.022 eV/atom for solid formation energies and 0.002 eV for molecular internal energies at 0 K.SchNet, CGCNN, iCGCNN, DimeNet, ALIGNN, and MEGNet predict up to 50 molecular and crystalline-material properties, with some models capturing many-body interactions through graph constructions.Source: Direct property prediction from… · p. 12

Fast materials screening

• Deep-learning property predictors enable faster prescreening than DFT for stable perovskites, ThCr2Si2-type materials, thermoelectrics, metal-organic frameworks, and molecular properties such as solubility and toxicity.They are used as prescreening tools because DFT high-throughput calculations are limited to a few thousand compounds and are time-consuming.Source: Fast materials screening · p. 13
• When precise atom positions are unavailable, symmetry-based descriptors and Bayesian optimization provide alternatives, but coarse representations have higher errors than graph-based models; BOWSR discovered and enabled experimental synthesis of ReWB and MoWC2.Autoencoders, reinforcement learning, and GANs also support inverse design by identifying compounds with desired properties, including in solid materials.Source: Fast materials screening · p. 13, Direct property prediction from… · p. 13

3.2.1 SMILES and fragment representation

• SMILES encodes elemental composition, bonding, branching, chirality, and molecular sequences as ASCII strings that can be converted into sparse feature matrices for deep learning.Fixed-length matrices and additional attributes such as aromaticity can be used during training.Source: §3.2.1 · pp. 14–15
• SMILES-based deep learning supports novel molecule design with VAE, GAN, and RNN methods and molecular screening using datasets such as ZINC, Tox21, and PubChem.Generated molecules may be physically invalid, so chemical intuition is used for further screening.Source: §3.2.1 · p. 15
• Fragment models such as DeepFrag and DeepFrag-K address SMILES validity limitations by predicting suitable substituents after removing a ligand or receptor complex.More generally, descriptor-based representations remain useful for prescreening theoretical and experimental data despite lacking explicit atomic structure.Source: §3.2 · p. 14, §3.2.1 · p. 15

3.2.2 Chemical formula representation

• Chemical formula representations range from elemental fractions and alloy weight percentages to descriptors built from physical attributes and statistical operations, including MagPie features.These representations contain tens to hundreds of composition-based features and work with both traditional machine learning and deep learning.Source: §3.2.2 · p. 15
• ElemNet significantly outperformed Random Forest using only raw elemental fractions, learned chemistry such as element similarity and charge balance, and predicted phase diagrams for unseen systems.Further depth reduced accuracy because of vanishing gradients.Source: §3.2.2 · p. 15
• IRNet uses residual learning to alleviate vanishing gradients, enable deeper learning, and improve accuracy over plain deep networks and traditional methods across diverse materials datasets when big data are available.Roost provides another graph-based approach that can outperform similar techniques.Source: §3.2.2 · p. 16
• Composition-based models predict formation energy, band gap, magnetization, superconducting temperatures, and elastic moduli across DFT and experimental datasets, while transfer learning improves accuracy on small data.Applications include quick prescreening and transfer across datasets and source-target properties.Source: §3.2.2 · p. 16
• Composition-based representations are especially useful when experimental atomic structures are unknown, but cannot distinguish polymorphs and may predict formation energy accurately without matching DFT accuracy for stability.Descriptor libraries include MatMiner and DScribe.Source: §3.2.2 · p. 16

3.3.1 Databases and software libraries

• Spectral databases are expanding rapidly, but computational collections substantially exceed experimental datasets and data consistency remains a challenge.Materials Project hosts more than 400000 K-edge and 90000 L-edge XANES site-wise spectra, whereas experimental XAS collections contain only hundreds of spectra.Source: §3.3.1 · pp. 17–18

3.3.2 Applications

• CNNs trained on simulated XRD patterns classify crystal structures, while PDF models can remain effective on experimental data despite resolution and line-shape variation.Reported XRD accuracies reached 81.14%, 83.83%, and 94.99% for space-group, extinction-group, and crystal-system classification, respectively; another study achieved 90.02% and 79.82% for crystal systems and space groups.Source: §3.3.2 · pp. 19–20
• Deep learning extracts structural and chemical information from XAS and vibrational spectra, including coordination numbers, radial distribution functions, topology, mixture concentrations, and material classes.Random forests outperformed MLPs and CNNs for XANES atomic-environment labels, while CNNs outperformed classical models such as SVMs for Raman classification.Source: §3.3.2 · pp. 21–23
• Models also learn forward mappings from structures or images to spectra and from electronic density-of-states spectra to adsorption energies.An E(3)NN predicted phonon DOS and heat capacity from 1200 calculations on 65 elements, while an MPNN reproduced prominent XANES peaks with 90% within 1 eV of FEFF calculations.Source: §3.3.2 · pp. 22–23

3.4 Image based models

• Image-based models support classification, property regression, hyperparameter tuning, segmentation, superresolution, object tracking, and microstructure representation learning.These tasks generalize across optical, electron, scanning probe, and transmission electron microscopy modalities.Source: §3.4 · p. 23
• Microscopy images capture atomic-to-mesoscale structures, defects, and defect dynamics that are critically linked to material functionality and performance.Aberration-corrected STEM has made atomic-scale imaging widespread and near-routine.Source: §3.4 · p. 23
• Growing image datasets create an analysis bottleneck, while conventional pipelines often remain brittle and dependent on manual feature identification.Non-DL image analysis has advanced quantitative microscopy but is not broadly applicable without substantial manual intervention.Source: §3.4 · pp. 23–24
• Deep learning is therefore presented as the most promising route to high-performance, high-throughput automated image analysis in materials characterization.Computer vision has produced a broad portfolio of applications aimed at accelerating and improving image-based material characterization.Source: §3.4 · pp. 23–24

Databases and software libraries

• Materials image datasets can come from experiments, simulations, or the literature.Source: §3.4.1 · p. 24
• Software libraries can generate materials images such as STM/STEM images.Source: §3.4.1 · p. 24
• Rapid growth in materials image-learning tasks has produced several useful software packages, with representative datasets and libraries listed in Table 4.Source: §3.4.1 · p. 24

Image classification and regression

• Image classification and regression enabled broad material-feature recognition, achieving 90 % SEM-image accuracy and 16 % lower EBSD disorientation error than dictionary-based indexing.DL also classified diffraction symmetries despite defects, reached average F1 scores around 0.9 for five 2D Bravais lattices, and improved crystal-plasticity accuracy from 68.24 % to 92.48 % [270, 235, 260].Source: Image classification and regression · pp. 24, 26

Pixelwise learning

• Pixelwise models segmented steel phases with 94 % accuracy, while PixelNet achieved 86.5 % precision and recall for phase segmentation and 91.1 % for spheroidite particles.Additional models distinguished dislocations from γ−γ′ interfaces and segmented noisy dendritic-growth data despite small labeled datasets.Source: Pixelwise learning · p. 27

Object/entity recognition, localization, and tracking

• Deep learning detected and tracked atoms and defects across microscopy images, including WS2 sequences under deformation and WSe2 atomic-site classification with accuracy up to 98 %.Models trained on simulated graphene images transferred successfully to experimental images, while DefectSegNet produced more accurate and consistent measurements than human experts [274, 253].Source: Object/entity recognition,… · p. 28

Superresolution imaging and auto-tuning experimental parameters

• GAN super-resolution reduced undetected nanoparticle gaps from 13.9 % to 3.7 % and cut electron-beam interaction time four-fold [281].Other systems reconstructed high-resolution STEM images from sampling areas reduced by up to 100, automatically reconditioned SPM tips with 99 % accuracy, and detected manufacturing defects across printers.Source: Superresolution imaging and… · p. 30

Microstructure representation learning

• Microstructure representation learning improved transfer learning when source images were visually similar, while unsupervised representations classified steel defects and fracture orientations with 98 % and 88 % performance.Triplet networks outperformed ordinary convolutional classifiers, and customized CNNs supported homogenization and localization across scales [288, 259, 261, 285, 262].Source: Microstructure representation learning · pp. 31–32
• GAN-based microstructure design produced structures with up to 17 % better optical absorption than randomly sampled microstructures and enabled inverse modeling for desired properties.GAN-generated fuel-cell and high-energy-material microstructures also matched experimental statistics and visual features better than alternative synthetic approaches [290, 291].Source: Microstructure representation learning · pp. 32–33

Mesoscale modeling applications

• Mesoscale GNNs predicted magnetostriction with around 10 % error and abnormal grain growth with 75 % accuracy, outperforming image-only prediction at 60 % [293].Graph representations captured neighboring-grain interactions, while two neighborhood shells sufficed for maximum AGG performance.Source: Mesoscale modeling applications · p. 33

3.5 Natural language processing

• Materials NLP offers a route from largely unstructured text, tables, and images to structured information, but resolving dependencies across sentences and paragraphs remains unsolved.Applications span information extraction, search, synthesis prediction, and materials discovery.Source: §3.5 · pp. 33–34, §3.5.3 · p. 35

3.5.1 Data sets for NLP

• Relevant NLP datasets include articles, patents, preprints, and online sources, but copyright restrictions, publisher agreements, unreliable PDF parsing, and verification concerns limit access and usability.Automatically converting documents into well-structured HTML with few residual errors would substantially improve the field.Source: §3.5.1 · p. 34

3.5.2 Software libraries for NLP

• Materials NLP requires domain-specific preprocessing and entity extraction because chemical formulas, scientific notation, element symbols, and abbreviations cause errors in general-purpose libraries, while neural architectures are usually unchanged.Materials-specific word embeddings can be more informative than embeddings trained on larger generic corpora.Source: §3.5.2 · pp. 34–35, Table 5 · p. 35

3.5.3 Applications

• Deep-learning models extract synthesis information from text to predict synthesis maps for titania nanotubes, binary and ternary oxides, and perovskites.Automated and semi-automated tools also extract Curie and Néel temperatures, battery properties, UV-vis spectra, and metal-organic-framework surface and pore characteristics.Source: §3.5.3 · pp. 35–36
• NLP-derived databases support materials discovery, including HoBe2 magnetocaloric materials and dye-sensitized solar cells whose selected dye combination matched an existing standard material.In the dye study, 9431 literature-extracted dyes were reduced to 309 candidates, then 33, and finally 5 experimentally evaluated dyes.Source: §3.5.3 · pp. 36–37
• Materials-literature word embeddings can directly predict composition–application links, retrospectively identify thermoelectric materials, and propose compositions later tested computationally or experimentally [299, 322–328].The same approach has been used to search for understudied areas of metallocene catalysis, although direct materials prediction remains challenging.Source: Figure 5 · p. 36, §3.5.3 · p. 37

Practical limitations

• Most UQ implementations require retraining or model redesign; Bayesian NNs avoid this limitation but are computationally expensive and increasingly inefficient with larger datasets.Current research therefore targets scalable uncertainty estimation for trained networks without retraining, including output-variance methods and batch-normalization approximations to Bayesian inference.Source: §4 · p. 37

Motivation

• UQ improves reliability, guides training-data expansion, and enables out-of-distribution rejection where materials datasets are small, extrapolation is poor, or samples are ambiguous.A UQ-motivated reject option substantially improves the remaining material data by detecting out-of-distribution samples.Source: §4 · p. 38

Uncertainty types

• Epistemic uncertainty reflects insufficient training data, whereas aleatory uncertainty reflects unmodeled parameters; ideal methods distinguish and quantify both separately.Epistemic uncertainty is especially important because DL models perform well on interpolation but poorly on extrapolation.Source: §4 · p. 38

Methods

• Common DL-UQ methods include Monte-Carlo dropout, deep ensembles, quantile regression, and Gaussian processes, each producing predictive distributions or intervals through different mechanisms.Monte-Carlo dropout randomizes network subsets during prediction, while ensembles combine multiple models and initializations to estimate output variance.Source: §4 · pp. 38–39

Quantile regression

• Quantile regression estimates confidence intervals from selected quantiles, but traditionally requires three model runs; TensorFlow can obtain multiple quantiles simultaneously.Using amin = 0.1 and amax = 0.8 yields intervals covering 70 % of the population.Source: §4 · p. 39

Gaussian processes

• Gaussian processes provide predictive means and variances intrinsically, while deep Gaussian processes extract higher-level features but require careful optimization and can incur prohibitive cost on large datasets.Standard kernels may under-perform, motivating neural-network feature mappings and multilayer Gaussian-process models.Source: §4 · p. 39

• Dataset reliability and quality assessment remain difficult because ground truth, global comparison metrics, and reproducibility are often lacking [346].Source: §5 · p. 39
• Chemical-formula representations support compounds without known structures but miss structure-dependent phenomena, including phase transitions and sharply contrasting diamond and graphite properties.Source: §5 · pp. 39–40
• Atomistic graph models have mainly been tested on bulk materials, remain unvalidated for defects and multidimensional phase-space exploration, and still require lower errors to approach chemical accuracy.Input features must retain information predictive of the output labels; ALIGNN has improved accuracy over earlier atomistic models, but residual errors remain.Source: §5 · p. 40
• Experimental images and spectra are typically noisy and require substantial preprocessing, whereas noise-free simulated data fail to represent realistic scenarios.Source: §5 · p. 40
• Materials DL needs better regression split, normalization, and augmentation practices to prevent overly similar training and test materials from overstating generalization.Uncertainty quantification remains underdeveloped, while tools such as GNNExplainer seek to reduce black-box behavior and build community trust.Source: §5 · p. 40
• Closed-loop autonomous materials design remains a major technological challenge, although self-driving laboratory demonstrations have autonomously adjusted copper-oxide synthesis toward a target oxidation state.Source: §5 · p. 40