PaperlayerView exact revision on arXiv

Source-linked AI summary

Constrained crystals deep convolutional generative adversarial network for the inverse design of crystal structures

Teng Long, Nuno M. Fortunato, Ingo Opahle, Yixuan Zhang, Ilias Samathrakis, Chen Shen, Oliver Gutfleisch, Hongbin Zhang

arXiv:2007.11228v2physics.comp-phcond-mat.mtrl-sci

TL;DR

Existing crystal-structure discovery relies heavily on high-throughput calculations and has limited training data for deep generative models. This paper develops a constrained generative framework that optimizes formation energy and generates unreported Bi-Se structures with low formation energy.

  • Problem

    Deep crystal-structure generation is constrained by insufficient known training structures, motivating methods for inverse materials design.

  • Method

    The CCDCGAN framework combines a generator, discriminator, and constraint model to optimize formation energy during crystal-structure generation.

  • Results

    CCDCGAN generated unreported Bi-Se crystal structures with low formation energy and reproduced phases near the convex hull after screening and relaxation.

  • Takeaways & Limitations

    The framework supports inverse design of crystal structures for unknown compositions within the studied binary Bi-Se system.

  • Takeaways & Limitations

    Four of seven non-included Bi-Se phases—Bi8Se9, BiSe, Bi8Se7, and Bi4Se3—were not reported as generated structures.

Abstract

from arXiv · show

Autonomous materials discovery with desired properties is one of the ultimate goals for materials science, and the current studies have been focusing mostly on high-throughput screening based on density functional theory calculations and forward modelling of physical properties using machine learning. Applying the deep learning techniques, we have developed a generative model which can predict distinct stable crystal structures by optimizing the formation energy in the latent space. It is demonstrated that the optimization of physical properties can be integrated into the generative model as on-top screening or backwards propagator, both with their own advantages. Applying the generative models on the binary Bi-Se system reveals that distinct crystal structures can be obtained covering the whole composition range, and the phases on the convex hull can be reproduced after the generated structures are fully relaxed to the equilibrium. The method can be extended to multicomponent systems for multi-objective optimization, which paves the way to achieve the inverse design of materials with optimal properties.

Data for generative adversarial network · Results

Because binary systems often lack enough known structures for DCGAN training, the study constructed a Bi–Se DFT database from converged prototype structures. The database contained 155 stable and 707 metastable structures under the stated energy and convex-hull criteria.

  • Results: DFT calculations expanded the training data because Materials Project provides only 17 known Bi_xSe_y materials, insufficient for deep learning.The motivation was that binary systems typically contain only tens of known crystalline phases.
  • Results: Starting from 10,981 prototype structures, substituting Bi and Se atoms yielded 9,810 converged cases for the Bi–Se training database.The resulting database is referred to as the Bi–Se database.
  • Results: Eight of 15 experimentally achievable phases—including BiSe3, Bi2Se3, and Bi4Se5—were included in the database.The included phases were BiSe3, Bi2Se3, Bi4Se5, Bi3Se2, Bi5Se3, Bi2Se, Bi7Se3, and Bi3Se.
  • Results: Seven experimentally achievable compounds, including BiSe2, Bi3Se4, and Bi4Se3, were excluded by the selection criteria.The excluded compounds were BiSe2, Bi3Se4, Bi8Se9, BiSe, Bi8Se7, Bi6Se5, and Bi4Se3.
  • Results: 707 of 9810 Bi–Se structures were metastable, with negative formation energy and distance to the convex hull below 150 meV/atom.Metastable structures represented 7.2% of the database.

Continuous representation for crystal structures

The method converts discontinuous CIF information into a continuous, homogeneous latent representation that preserves chemical and structural information. This reversible representation reconstructs Bi-Se crystal structures with 96% success and negligible training–test loss differences.

  • Representation design: CIF data are converted into a continuous, homogeneous representation because discontinuous and heterogeneous formats are unsuitable for generative modeling.The representation includes both chemical and structural information.
  • Representation design: Lattice constants and atomic positions are voxelized and encoded by autoencoders into a continuous 2D crystal-graph latent space.The process is reversible, allowing 2D crystal graphs to be reconstructed into real-space crystal structures.
  • Reconstruction performance: 9420 of 9810 (96%) Bi-Se crystal structures are successfully reconstructed from the latent representation.The reconstruction process is reversible, enabling crystal structures to be generated from random 2D crystal graphs.
  • Reconstruction performance: Negligible differences between training-set and test-set losses indicate that the latent 2D crystal graphs retain adequate reconstruction information without overfitting.This behavior is shown by the learning curves of both the lattice and sites autoencoders.
  • Latent-space diversity: Twelve typical 2D crystal graphs demonstrate diversity in the latent space, including graphs corresponding to Bi4Se4, Bi2Se3, Bi4Se8, and Bi8Se6.Four representative structures and their corresponding graphs are shown in Fig. 1(d).

Construction and prediction of DCGAN

The DCGAN generates valid crystal structures without prescribing a latent-space distribution, achieving broad composition coverage and discovering distinct structures beyond the known Bi-Se phase space.

  • Construction and prediction of DCGAN: Unlike VAE, GAN generates data without assuming a latent-space distribution, making the generation process more robust for this work.GAN uses mutually competitive generator and discriminator networks to make generated data statistically resemble the training data.
  • Construction and prediction of DCGAN: The DCGAN produced 2832 crystal structures from 13000 generated 2D crystal graphs and covered a large composition range.The model’s success rate is defined as generated crystals divided by generated 2D crystal graphs.
  • Construction and prediction of DCGAN: 89.9% of generated structures had negative formation energies after DFT relaxation, compared with 46.8% in the Bi-Se database.This corresponded to 2548 of 2832 generated structures versus 4588 of 9810 database structures.
  • Construction and prediction of DCGAN: 476 generated structures were distinct from the Bi-Se database and each other, including 73 metastable and 15 stable structures.Overall, 1233 generated structures were metastable and 58 were stable.
  • Construction and prediction of DCGAN: DCGAN explored a much larger latent-space phase region than the Bi-Se database, generating distinct crystal structures beyond known structures.The database structures were concentrated in a small latent-space region, whereas DCGAN covered a much larger phase space.

Training of constraint model

A CNN was trained to predict formation energy from 2D crystal graphs in the latent space, enabling formation-energy optimization for inverse design. On the Bi-Se database, it achieved about 85% R2 and 0.019 eV/atom MAE, outperforming CGCNN’s 0.021 eV/atom MAE with marginal overfitting.

  • Training of constraint model: The constraint CNN maps 2D crystal graphs in latent space to formation energies, using thermodynamic stability as the target property.This property predictor is trained to support formation-energy optimization in the generative model’s latent space.
  • Training of constraint model: The latent-space crystal graphs serve as effective physical-property descriptors, while the training curve indicates that overfitting is only marginal.The reported accuracy and learning curve support the descriptor’s suitability for forward prediction.

Construction and prediction of DCGAN + constraint

DCGAN with an add-on formation-energy constraint screens generated structures without retraining another model and supports broad physical-property optimization. It increases negative-formation-energy and metastable-structure ratios, but reduces the number of distinct and stable structures.

  • Construction and prediction of DCGAN + constraint: The add-on constraint requires no additional model training and can incorporate existing forward-prediction models using crystal graphs or chemical and structural descriptors.This enables optimization across a wide spectrum of physical properties.
  • Construction and prediction of DCGAN + constraint: Because it primarily selects DCGAN outputs, the constraint cannot automatically search specific latent-space regions for local optima.Its application screens unwanted latent-space points, shrinking the explored region.
  • Construction and prediction of DCGAN + constraint: 94.8% of selected structures have negative formation energy, versus 89.9% for DCGAN, while the metastable ratio rises to 56.2% from 43.5%.The constraint selects 2148 structures from 2832 generated structures; 2037 have negative formation energy and 1145 are metastable.
  • Construction and prediction of DCGAN + constraint: The constraint reduces distinct structures to 247 after DFT calculations, compared with 476 obtained by DCGAN, and leaves 36 generated stable structures.The reduction is attributed to structural changes during DFT relaxation and imperfect formation-energy prediction by the CNN constraint.
  • Construction and prediction of DCGAN + constraint: The constraint produces 67 distinct metastable and 12 distinct stable structures after comparison with the Bi-Se database.Examples include Bi3Se and Bi2Se2, with formation energies of -0.065 and -0.202 eV/atom, respectively.

Construction and prediction of CCDCGAN

CCDCGAN integrates a formation-energy constraint into DCGAN backpropagation to optimize the latent space for automated inverse design. It generates more efficient and distinct stable crystal structures, reproduces known and experimentally achievable Bi-Se phases, and can predict structures at unseen compositions.

  • Construction and prediction of CCDCGAN: CCDCGAN requires training from scratch for 18 hours, two hours longer than DCGAN + constraint, and four excluded phases remain unreproduced under the applied constraints.The limitations are suspected to arise from restrictions on unit-cell dimensions and atom counts in the training database.
  • Construction and prediction of CCDCGAN: CCDCGAN regenerated 11 of 15 experimentally achievable phases, including all 8 phases in the Bi-Se database and 3 of 7 excluded phases exactly.The reproduced excluded phases were BiSe2, Bi3Se4, and Bi6Se5; four other excluded phases could not be reproduced.
  • Construction and prediction of CCDCGAN: Removing BiSe3, Bi2Se3, Bi2Se, and Bi3Se from training still allowed CCDCGAN to regenerate their experimental crystal structures, demonstrating prediction for unknown compositions.This capability is attributed to the model's predictive performance and could accelerate discovery of distinct crystal phases.

Discussion

The CCDCGAN inverse-design framework generates unreported low-formation-energy Bi–Se crystal structures using continuous crystal-graph latent representations. Physical-property optimization can be integrated as constraints or back propagators and extended to multicomponent, multi-objective design, although achieving mechanical and dynamical equilibrium remains challenging.

  • Discussion: CCDCGAN successfully designs unreported crystal structures with low formation energy for the binary Bi–Se system.
  • Discussion: 2D crystal graphs construct a continuous latent space that represents known structures, models physical properties, and decodes into distinct crystal structures.
  • Discussion: The inverse-design model can be generalized to multicomponent systems, as demonstrated by prior work.
  • Discussion: Formation-energy optimization can enter the generative model as explicit constraints or back propagators, enabling multi-objective optimization through modified objective functions.
  • Discussion: A remaining challenge is bringing generated structures into mechanical and dynamical equilibrium, potentially through relaxation with machine-learning interatomic potentials.

Methods

The method builds a Bi-Se database from substituted Materials Project binaries, evaluates formation-energy stability, and represents crystal structures with continuous 3D voxel grids. Separate 3D convolutional autoencoders encode site and lattice voxels into one-dimensional latent vectors for reconstruction and generation.

  • Data for generative adversarial network: The Bi-Se database is constructed by substituting binary Materials Project structures, with formation-energy calculations used to evaluate thermodynamic stability and determine the convex hull.The high-throughput workflow optimizes structures and evaluates both database and generated structures.
  • Data for generative adversarial network: The database-screening criteria exclude several stable phases near the convex hull, including BiSe, Bi4Se3, and Bi8Se7.The selection limits unit-cell size to fewer than 20 atoms and a maximum length below 10 Angstrom.
  • Continuous representation for crystal structures: Binary crystal structures are represented by continuous 3D voxel grids: separate site voxels encode the two elements’ atomic positions, while lattice voxels encode lattice constants.Gaussian probability densities are used for voxel construction, and inverse transformation is direct for lattice voxels but requires image filtering for site voxels.
  • Continuous representation for crystal structures: Separate 3D convolutional autoencoders encode site and lattice voxel data into one-dimensional latent vectors and decode them back using information-reconstruction and regularization losses.Both autoencoders use a 90%/10% training/test split and achieve high reconstruction ratios under pymatgen structure comparison.
  • Continuous representation for crystal structures: The site autoencoder uses 10 3D convolutional layers, whereas the lattice autoencoder uses 8, with leakyRELU activations and Adam optimization.The stated Adam learning rate is 0.003, with λ = 0.000001, β1 = 0.9, and β2 = 0.99.

Training of DCGAN

The DCGAN alternately trains a discriminator to distinguish generated from original crystal graphs and updates the generator through backpropagation until generated structures become indistinguishable from originals. Both networks use Adam optimization and convolutional, dropout, batch-normalization, and specified activation components.

  • Training procedure: DCGAN alternates discriminator training on generated versus original crystal graphs with generator updates through discriminator feedback, progressively producing structures indistinguishable from the originals.The generator and discriminator are trained repeatedly, with backpropagation updating the generator toward the original structures.
  • Model configuration: Both generator and discriminator use Adam with learning rate 0.002, β1=0.5, and β2=0.999, alongside 2D convolution, dropout, and batch normalization.The models use RELU, leakyRELU, tanh, and sigmoid activations.
  • Training setup: The DCGAN is trained for 1,000,000 steps over 16 hours on a Quadro P2000 GPU.Generator and discriminator designs are listed in Supplementary Tables 3 and 4.

respectively. Latent space of the GAN model has 200 dimensions.

The constraint model uses a convolutional and fully connected architecture trained with Adam and mean squared error, with regularization applied after convolutional layers.

  • Training of constraint model: The constraint model comprises four convolutional 2D layers and six fully connected layers, implemented in Keras.Its detailed design is provided in Supplementary Table 5.
  • Training of constraint model: Training uses Adam with a 0.002 learning rate, β1=0.5, β2=0.999, and mean squared error loss.
  • Training of constraint model: LeakyReLU activations are used, while dropout and batch normalization are applied after each convolutional layer to prevent overfitting.

Training of CCDCGAN

CCDCGAN retains the previous model’s generator, discriminator, constraint, and DCGAN parameters, but changes the optimization objective to Eq. (2) with formation-energy loss weight 0.1. Training takes about 18 hours under DCGAN’s conditions, with generated structures reported in Supplementary Table 6 and Supplementary Data 2.

  • Training of CCDCGAN: CCDCGAN retains the previous model’s generator, discriminator, constraint, and DCGAN parameters while using Eq. (2) as its optimization objective.The formation-energy loss weight is 0.1.
  • Training of CCDCGAN: Training takes about 18 hours under the same conditions as DCGAN.
  • Training of CCDCGAN: Typical generated structures are listed in Supplementary Table 6, with additional data available in Supplementary Data 2.
Loading 2007.11228v2…