PaperlayerView exact revision on arXiv

Source-linked AI summary

CrystalFlow: A Flow-Based Generative Model for Crystalline Materials

Xiaoshan Luo, Zhenyu Wang, Qingchang Wang, Jian Lv, Lei Wang, Yanchao Wang, Yanming Ma

arXiv:2412.11693v4cond-mat.mtrl-sciphysics.comp-ph

TL;DR

CrystalFlow addresses the challenge of efficiently sampling the vast crystal configuration space for crystal structure prediction. It combines Continuous Normalizing Flows, Conditional Flow Matching, symmetry-aware representations, and equivariant graph networks to generate crystal structures. The model achieves comparable or superior performance to state-of-the-art methods across standard benchmarks and supports conditional generation under pressures or material-property constraints.

  • Problem

    Crystal structure prediction is difficult because crystal-space sampling scales unfavorably as the potential-energy surface becomes higher-dimensional and develops exponentially more local minima.

    Source: §I · p. 1

  • Method

    CrystalFlow uses Continuous Normalizing Flows trained with Conditional Flow Matching and symmetry-aware graph representations to generate lattice parameters, coordinates, and/or atom types.

    Source: §I · p. 2, §4 · p. 5

  • Results

    CrystalFlow achieves comparable or superior performance to state-of-the-art models across standard generation metrics and generates structures optimized for external pressures or material properties when trained with appropriate labels.

    Source: §I · p. 2, A. CSP performance on MP-20 and… · p. 6, B. CSP performance on MP-CALYPSO-60… · p. 7

  • Takeaways & Limitations

    CrystalFlow provides a versatile structure-sampling tool for crystal structure prediction and materials-design workflows.

    Source: §IV · p. 9

    The full potential of flow-based models remains to be comprehensively investigated, including the effects of integration strategies, integration lengths, and prior distributions.

    Source: §IV · p. 9

Abstract

from arXiv · show

Deep learning-based generative models have emerged as powerful tools for modeling complex data distributions and generating high-fidelity samples, offering a transformative approach to efficiently explore the configuration space of crystalline materials. In this work, we present CrystalFlow, a flow-based generative model specifically developed for the generation of crystalline materials. CrystalFlow constructs Continuous Normalizing Flows to model lattice parameters, atomic coordinates, and/or atom types, which are trained using Conditional Flow Matching techniques. Through an appropriate choice of data representation and the integration of a graph-based equivariant neural network, the model effectively captures the fundamental symmetries of crystalline materials, which ensures data-efficient learning and enables high-quality sampling. Our experiments demonstrate that CrystalFlow achieves state-of-the-art performance across standard generation benchmarks, and exhibits versatile conditional generation capabilities including producing structures optimized for specific external pressures or desired material properties. These features highlight the model's potential to address realistic crystal structure prediction challenges, offering a robust and efficient framework for advancing data-driven research in condensed matter physics and material science.

I. Introduction

CrystalFlow addresses the scaling challenge of crystal structure prediction with a symmetry-aware flow-based generative model. It uses continuous normalizing flows and graph-based equivariant networks to generate lattice parameters, coordinates, and atom types while supporting conditional structure generation.

I. Introduction

  • Crystal structure prediction remains difficult because the potential-energy surface grows linearly in dimensionality while its local minima increase exponentially with unit-cell size.This unfavorable scaling motivates more efficient and robust sampling methods for crystal space.Source: §I · p. 1
  • Graph-based crystal models explicitly capture crystal symmetries, whereas string-based models scale to larger and more diverse data but generally do not model those symmetries explicitly.The comparison situates CrystalFlow within two major representation strategies for crystal generation.Source: §I · p. 2

II. Method

  • CrystalFlow uses Continuous Normalizing Flows within Conditional Flow Matching to transform simple prior distributions into complex crystalline-material distributions.The model can jointly generate lattice parameters, fractional coordinates, and/or atom types.Source: §I · p. 2, §II · p. 2
  • CrystalFlow incorporates permutation, rotation, and periodic-translation symmetries through an appropriate representation and graph-based equivariant message-passing network.This design targets data-efficient learning and high-quality sampling for crystalline systems.Source: §I · p. 2

I. Introduction

  • CrystalFlow supports conditional generation for constraints such as external pressure or desired material properties.The authors report that appropriately labeled training data enables structures optimized for these application-driven conditions.Source: §I · p. 2, §II · p. 2

A. Continuous Normalizing Flow.

  • The model architecture connects random structures sampled from priors with dataset structures through time-dependent flows predicted from intermediate crystal states and conditioning variables.The graph neural network receives intermediate lattice, coordinate, and atom-type representations and outputs vector fields.Source: Figure 1 · p. 3, A. Continuous Normalizing Flow · p. 3

B. Conditional Flow Matching.

Conditional Flow Matching makes CrystalFlow trainable without directly simulating an intractable marginal vector field. It constructs tractable conditional probability paths and extends the framework to conditioning variables for crystal generation.

B. Conditional Flow Matching.

  • Conditional Flow Matching replaces direct treatment of an intractable marginal vector field with simulation-free training based on tractable conditional vector fields.The marginal probability path is represented as a mixture of conditional paths indexed by an auxiliary variable.Source: B. Conditional Flow Matching · p. 3
  • The framework extends naturally to conditional generation by describing the flow and training objective with an additional conditioning variable.The neural vector field is parameterized as a time-dependent function of t, x, and y.Source: B. Conditional Flow Matching · p. 3
  • CrystalFlow uses independent coupling, defining the auxiliary variable as the initial and terminal points that determine an interpolating probability path.The prior and target distributions appear as the path’s boundary conditions, while the neural network learns the corresponding vector field.Source: B. Conditional Flow Matching · p. 3

C. Joint equivariant flow.

CrystalFlow jointly evolves lattice representations and fractional coordinates while preserving the periodic-E(3) symmetries of crystals. Its invariant lattice parameterization separates rotational degrees of freedom from structural information and supports flexible flow design.

C. Joint equivariant flow.

  • CrystalFlow jointly evolves lattice parameters and fractional coordinates using a flow designed to preserve permutation, rotation, and periodic-translation invariance.The symmetry-aware construction is intended to improve data efficiency, computational efficiency, and sampling quality.Source: C. Joint equivariant flow · pp. 3–4
  • A geometric graph neural network handles permutation invariance, while rotation-invariant lattice and fractional-coordinate representations make periodic-E(3) invariance tractable.The design combines an equivariant flow with an invariant prior.Source: C. Joint equivariant flow · p. 4
  • The lattice matrix is represented by a six-dimensional rotation-invariant vector derived from polar decomposition, decoupling rotational and structural information.This compact representation provides flexibility in defining the prior and conditional probability path.Source: C. Joint equivariant flow · p. 4

1. Flow on lattice.

  • For the lattice flow, CrystalFlow uses a Gaussian prior and a time-dependent path whose mean linearly interpolates between initial and terminal lattice representations.Setting the lattice path’s standard deviation to zero reduces it to deterministic interpolation consistent with Rectified Flow [52].Source: §1 · p. 4

2. Flow on fractional coordinates.

The fractional-coordinate flow is constructed to remain invariant under periodic translations. CrystalFlow combines an invariant uniform prior with a wrapped Gaussian path whose interpolation respects the unit-cell periodicity.

2. Flow on fractional coordinates.

  • CrystalFlow enforces periodic-translation invariance for fractional coordinates using an invariant prior and a pairwise invariant conditional probability path.This construction ensures the resulting marginal probability path preserves the required crystal periodicity.Source: §2 · p. 4
  • The fractional-coordinate prior is uniform, and the conditional path is a wrapped Gaussian connecting initial and terminal coordinates under the minimum-image convention.The mean uses a wrapping operation to keep interpolated coordinates within periodic equivalence classes.Source: §2 · p. 4
  • The conditional vector field and probability path are pairwise invariant under periodic translations, and the coordinate noise scale is set to zero for consistency with Rectified Flow [52].The construction preserves periodicity while using deterministic interpolation in the coordinate-flow component.Source: §2 · p. 4

3. Loss function.

CrystalFlow trains neural vector fields to match conditional flow-matching targets for lattice and coordinate representations. The total loss combines the corresponding lattice and fractional-coordinate terms with tunable weights.

  • CrystalFlow’s total training loss combines lattice and fractional-coordinate vector-field regression terms.The interpolating structure is represented by (k_t, F_t), and the predicted vector fields are parameterized by learnable neural networks.Source: §3 · pp. 4–5

4. Flow on chemical composition.

CrystalFlow extends its conditional flow framework to generate atom types for de novo materials design while using a symmetry-aware graph neural network for vector-field prediction. The architecture encodes atomic and conditioning information and preserves periodic translational invariance.

4. Flow on chemical composition.

  • De novo generation adds a chemical-composition CNF that generates atom types alongside fractional coordinates and lattice variables.CrystalFlow models p(x|y), where x=(F,L,A) and y can specify target properties such as formation energy.Source: §4 · p. 5
  • Atomic types are represented as compact 28-dimensional vectors formed by concatenating one-hot row and column encodings from a 13×15 periodic-table grid.The composition flow uses a Gaussian prior in R^28, and its conditional probability path and vector field are explicitly formulated.Source: §4 · p. 5

D. Model architecture.

  • The model architecture uses a graph neural network with message passing to predict vector fields while enforcing periodic translational invariance.Node features incorporate time, atomic identity, and user-specified conditions; the network uses L layers and reads out the vector field after message passing.Source: D. Model architecture · p. 5

E. Model inference.

During inference, CrystalFlow integrates a trained vector field from a random prior state to obtain a crystal structure. Its evaluation covers standard generation metrics and CSP measures including match rate and normalized RMSE.

E. Model inference.

  • Inference starts from a random prior state containing lattice parameters, fractional coordinates, and atom types for de novo generation, then solves the learned ODE over t∈[0,1].A numerical ODE solver performs the integration using the trained vector field.Source: E. Model inference · p. 5
  • The default inference scheme uses 100 integration steps, with anti-annealing applied only to fractional coordinates for CSP and disabled for de novo generation.The step size is Δt=1/100; the anti-annealing parameter is s′=5 for CSP and s′=0 for DNG.Source: E. Model inference · p. 5

III. Results

  • The evaluation benchmarks CrystalFlow against existing crystal-generation methods using standard metrics and analyzes generated structures with DFT calculations.Table I reports CSP match rate for reconstructed held-out structures and normalized RMSE averaged over matched structures.Source: §III · p. 6, Table I · p. 6

A. CSP performance on MP-20 and MPTS-52 datasets.

CrystalFlow achieves state-of-the-art crystal structure prediction performance on MP-20 and MPTS-52, while also supporting pressure-conditioned generation on MP-CALYPSO-60. On the latter dataset, its generated structures better match target pressures and generally require fewer optimization steps than Cond-CDVAE.

A. CSP performance on MP-20 and MPTS-52 datasets.

  • CrystalFlow achieves state-of-the-art performance across MP-20 and MPTS-52, matching FlowMM on MP-20 while outperforming CDVAE and DiffCSP, and leading all four models on MPTS-52.Evaluation uses match rate and normalized positional RMSE at k = 1, 20, and 100.Source: A. CSP performance on MP-20 and… · p. 6

B. CSP performance on MP-CALYPSO-60 dataset.

  • On MP-CALYPSO-60, CrystalFlow is conditioned on chemical composition and external pressure to generate structures across pressure conditions.The dataset contains 657,377 structures spanning 86 elements and 79,884 unique chemical compositions.Source: B. CSP performance on MP-CALYPSO-60… · p. 6
  • CrystalFlow structures align significantly better with target pressures than Cond-CDVAE structures across integration steps S = 100, 1000, and 5000, except for a few outliers.Most CrystalFlow samples also have lower enthalpy before optimization, although optimization narrows the enthalpy difference between models.Source: B. CSP performance on MP-CALYPSO-60… · p. 7, Figure 2 · p. 7
  • At S = 5000, CrystalFlow requires 39.82 average ionic steps versus 45.91 for Cond-CDVAE, corresponding to a 13.3% reduction in computational cost.CrystalFlow generally has a higher convergence rate, and its required ionic steps decrease as integration steps increase.Source: B. CSP performance on MP-CALYPSO-60… · pp. 7–8, Table II · p. 8
  • For 200 SiO2 structures at 0 GPa, CrystalFlow produces lower-energy structures than Cond-CDVAE, with S = 5000 requiring 31.99 versus 44.36 average ionic steps.The 31.99-step result is about 27.9% fewer steps than Cond-CDVAE.Source: B. CSP performance on MP-CALYPSO-60… · p. 8

C. DNG performance on MP-20.

CrystalFlow is evaluated for de novo crystal generation on MP-20, including unconditional sampling and formation-energy-conditioned generation. The section also discusses its broader promise, limitations of flow-based modeling, and symmetry properties of the conditional vector field.

C. DNG performance on MP-20.

  • CrystalFlow’s MP-20 DNG performance is evaluated against other models using structural validity, compositional validity, coverage, and property statistics.Structural validity is the percentage of generated structures whose pairwise atomic distances all exceed 0.5 Å; compositional validity is the percentage with overall neutral charge. Property statistics compare density and element count using Wasserstein distances.Source: C. DNG performance on MP-20 · p. 8
  • CrystalFlow achieves performance comparable to existing models across DNG metrics, with the smallest density Wasserstein distance despite marginally lower compositional validity.The result indicates that its generated structures have particularly reasonable lattice-parameter distributions.Source: C. DNG performance on MP-20 · p. 8, Table III · p. 9
  • Conditioning on formation energy produces distributions centered near target values, and geometric optimization brings them into closer alignment with the specified energies.For targets better represented in the training data, the generated distributions agree more closely with target values.Source: C. DNG performance on MP-20 · pp. 8–9, Figure 3 · p. 8

IV. Discussion

  • CrystalFlow is presented as a versatile structure-sampling tool for efficient materials-space exploration and potential integration into CSP and materials-design workflows.The discussion connects its physical constraints, target conditioning, and benchmark performance to this broader application scope.Source: §IV · p. 9
  • The work identifies flow-based modeling as promising but leaves the effects of integration strategies, integration lengths, and prior distributions for future study.The authors characterize this study as an initial exploration whose full potential remains unexamined.Source: §IV · p. 9

S1 Symmetry of conditional vector field uF

  • The supplementary symmetry analysis establishes that the conditional probability path is invariant under pairwise periodic translations.This follows from the corresponding translation invariance of the conditional velocity field and wrapped-normal construction.Source: S1 Symmetry of conditional vector… · p. 12

S2 Encoding of atom type for de novo generation

For de novo generation, CrystalFlow represents atomic types using a compact grid-based encoding of the periodic table. Each element is mapped to row and column indices and represented by concatenated one-hot vectors.

S2 Encoding of atom type for de novo generation

S3 Pressure distribution for MP-CALYPSO-60 dataset

The supplementary material documents the MP-CALYPSO-60 pressure dataset, computational settings, coverage definitions, model hyperparameters, and inference-time comparisons. These details specify the evaluation data, implementation choices, and metrics used across the study.

S4 DFT computational details

  • DFT calculations use the PBE functional, projector-augmented-wave methods, a 520 eV cutoff, and 0.25 Å−1 Monkhorst-Pack sampling.Optimization is limited to 150 ionic steps or 20 hours.Source: S4 DFT computational details · p. 15

S5 Coverage metric for de novo generation

  • Coverage recall measures how many test materials are matched by generated structures, whereas coverage precision measures how many generated materials are high quality.Both use structural and compositional fingerprint distances with predefined thresholds.Source: S5 Coverage metric for de novo… · p. 16

S6 Hyperparameters table for CrystalFlow

S7 Inference time compared to other generative model

  • The supplementary material includes a table comparing inference times for generating 10,000 materials against other generative models.The comparison includes records for FlowMM and FlowLLM from Ref. [39].Source: Table S7 · p. 18

Related papers

Loading 2412.11693v4…