Source-linked AI summary
MatRIS: Toward Reliable and Efficient Pretrained Machine Learning Interatomic Potentials
Yuanchang Zhou, Siyu Hu, Xiangyu Zhang, Hongyu Wang, Guangming Tan, Weile Jia
TL;DR
Equivariant MLIPs can achieve high accuracy, but their tensor-product operations and high-degree representations are computationally costly, raising the question of whether compact models can exploit growing QM datasets. MatRIS is an invariant MLIP that uses attention to model three-body interactions; across benchmarks it reports accuracy comparable to leading equivariant models, including a Matbench-Discovery F1 of 0.847 at lower training cost.
Problem
Equivariant MLIPs achieve strong benchmark accuracy but their tensor products and high-degree representations are costly, prompting the question of whether compact models can exploit expanding QM-based datasets.
Source: §1 · p. 2Method
MatRIS is an invariant MLIP that uses attention to model three-body interactions, with separable attention designed to handle source and target roles independently.
Source: §1 · p. 2, §3.2 · p. 5Results
Across diverse benchmarks MatRIS reports accuracy comparable to leading equivariant models; on Matbench-Discovery, MatRIS-L achieves an F1 of 0.847.
Source: §1 · p. 3Takeaways & Limitations
The authors conclude that MatRIS offers a route toward accurate and efficient MLIPs across multiple benchmarks.
Source: §5 · p. 10The authors identify scaling to larger QM-based datasets and incorporating long-range electrostatics as future work.
Source: §5 · p. 10
Abstract
from arXiv · showhide
Foundation MLIPs demonstrate broad applicability across diverse material systems and have emerged as a powerful and transformative paradigm in chemical and computational materials science. Equivariant MLIPs achieve state-of-the-art accuracy in a wide range of benchmarks by incorporating equivariant inductive bias. However, the reliance on tensor products and high-degree representations makes them computationally costly. This raises a fundamental question: as quantum mechanical-based datasets continue to expand, can we develop a more compact model to thoroughly exploit high-dimensional atomic interactions? In this work, we present MatRIS (\textbf{Mat}erials \textbf{R}epresentation and \textbf{I}nteraction \textbf{S}imulation), an invariant MLIP that introduces attention-based modeling of three-body interactions. MatRIS leverages a novel separable attention mechanism with linear complexity $O(N)$, enabling both scalability and expressiveness. MatRIS delivers accuracy comparable to that of leading equivariant models on a wide range of popular benchmarks (Matbench-Discovery, MatPES, MDR phonon, Molecular dataset, etc). Taking Matbench-Discovery as an example, MatRIS achieves an F1 score of up to 0.847 and attains comparable accuracy at a lower training cost. The work indicates that our carefully designed invariant models can match or exceed the accuracy of equivariant models at a fraction of the cost, shedding light on the development of accurate and efficient MLIPs.
1 INTRODUCTION
Equivariant MLIPs achieve strong accuracy but incur substantial computational costs, motivating the question of whether compact invariant models can exploit growing QM datasets. MatRIS addresses this with attention-based three-body interactions and reports competitive benchmark accuracy at lower training cost.
1 INTRODUCTION
- Equivariant models’ accuracy gains come with high computational and memory costs; eSEN-30M-MP and eqV2 S DeNS require 335 and 228 GPU days, respectively.The passage attributes the expense to tensor products, large parameter counts, and prolonged training schedules.Source: §1 · p. 2
- As QM-based datasets expand, the paper asks whether equivariance remains indispensable and whether a more compact architecture can capture high-dimensional atomic interactions.The motivation follows the reported growth in QM-based reference datasets.Source: §1 · p. 2
- MatRIS is an invariant MLIP using O(N) attention for three-body interactions, addressing the stated insufficiency of element types and pairwise interactions alone.The authors identify attention as a promising route to expressiveness and scalability.Source: §1 · p. 2
- MatRIS-L achieves an F1 of 0.847 and RMSD of 0.0717; MatRIS-S/M match eqV2 S DeNS/eSEN-30M-MP accuracy while improving training efficiency by 13.0×/6.4×, respectively.The reported Matbench-Discovery and cross-model comparisons indicate accuracy alongside lower training cost.Source: §1 · p. 3
2 RELATED WORKS
Related work spans invariant, equivariant, and unconstrained MLIPs, which differ in how they encode geometric symmetries and trade expressiveness against computational demands. The paper situates MatRIS within efforts to increase invariant models’ expressiveness through higher-order interactions.
Invariant MLIPs
- Invariant MLIPs use rotation- and translation-invariant internal coordinates, with later approaches adding angular, dihedral, or higher-order geometric features.Examples include directional message passing in DimeNet, dihedral angles in GemNet, and efficient four-body-angle extraction in SphereNet and ComENet.Source: §2 · p. 3
Equivariant MLIPs
- Equivariant MLIPs encode symmetry through restricted operations or irreducible representations, often using tensor products to combine feature degrees and remaining computationally demanding.The passage notes that such models typically restrict representations to low degree to avoid excessive complexity.Source: §2 · p. 3
Unconstrained MLIPs
- Unconstrained MLIPs learn symmetries from data or auxiliary losses, offering flexible architectures and inference efficiency comparable to invariant models.The cited approaches include rotation augmentation, force and torque removal, and an Equigrad loss.Source: §2 · p. 3
- Unconstrained models show competitive benchmark accuracy, but studies also report errors on certain property-prediction tasks.The passage presents this as a qualified limitation rather than a general failure.Source: §2 · p. 4
3 MATRIS
MatRIS combines atom- and line-graph representations with attention and refinement to model higher-order interactions. Its architecture includes graph generation, feature embedding, graph attention, refinement, and readout.
3.1 LINE–ATOM GRAPH INTERACTION
- The line graph represents three-body interactions by treating atom-graph edges as nodes and connecting nodes whose corresponding edges share an atom.The resulting line-graph edges encode angular information from three atoms.Source: §3.1 · p. 4, Figure 2 · p. 4
- Line-graph edge features are updated to encode angles, then propagated back to the atom graph so atomic features incorporate higher-order information.This describes the information exchange between the two graph representations.Source: §3.1 · p. 4
3.2 GRAPH ATTENTION
- Dim-wise Softmax computes attention scores independently for each feature dimension, unlike shared-weight approaches that assume equal importance across dimensions.The method is intended to preserve feature-dimension independence while weighting neighbors.Source: §3.2 · pp. 4–5
- Separable Attention uses independent source and target attention branches to represent potentially asymmetric influence between neighboring atoms.The source branch models neighbors’ effects on the central atom; the target branch models the central atom’s effects on its neighbors.Source: §3.2 · p. 5
3.3 OVERALL ARCHITECTURE
- For crystals, graph generation periodically repeats the structure and constructs atom and line graphs using radius cutoffs.The atom graph uses atoms and bonds; the line graph uses bonds and angles.Source: §3.3 · p. 6
- Feature embedding encodes atomic numbers with trainable embeddings, distances with a learnable radial Bessel basis, and angles with a learnable Fourier basis.A linear layer produces initial node, edge, and three-body representations.Source: §3.3 · p. 6
- Refinement processes graph-attention outputs with a gMLP and envelope, aggregates neighboring interactions, and produces residual node and edge features.The envelope is used to ensure smoothness.Source: §3.3 · p. 6
- The readout predicts atomic energies and magnetic moments from final-layer node features, sums atomic energies into total energy, and computes forces and stress.Force and stress are computed by differentiation as described in the cited passage.Source: §3.3 · p. 6
4 EXPERIMENTS
The experiments compare MatRIS across materials and molecular benchmarks, then examine its modules, training strategies, and computational efficiency. Evaluation includes compliant Matbench-Discovery, MatCalc, MDR phonons, and molecular zero-shot tasks.
4.1 MATBENCH DISCOVERY
- MatRIS-L achieves SOTA performance across all compliant Matbench-Discovery metrics, with F1 0.847 and RMSD 0.0717; MatRIS-S/M are comparable to leading models at lower cost.The benchmark evaluates formation energies for 256k WBM test structures, with MPTrj required in the compliant setting.Source: §4.1 · pp. 6–7, Table 1 · p. 7
4.2 MATCALC BENCHMARK
- MatRIS achieves the best overall MatCalc performance on MatPES-PBE, leads formation-energy prediction, reaches SOTA or near-SOTA on 83% of MPTrj metrics, and performs robustly across OAM metrics.On MPTrj, the authors report stable near-equilibrium performance despite PES softening.Source: Table 2 · p. 7, §4.2 · p. 8
4.3 MDR PHONON BENCHMARK
- MatRIS performs competitively on MPTrj-trained MDR phonon models, while MatRIS-10M-OAM reaches SOTA on most metrics and MatRIS-10M-OMat reports MAEs of 7.08, 7.12, 2.12, and 1.91.The metrics are ωmax, S, F, and CV; the benchmark evaluates approximately 10,000 structures using 0.01 Å displacements at 300 K.Source: Figure 4 · p. 8, §4.3 · p. 8
4.4 MOLECULAR ZERO-SHOT BENCHMARK
- MatRIS-M performs well across four molecular downstream datasets, reducing TorsionNet-500 energy error by about 22.2%–33.3% versus DPA3 and ranking first on five of six metrics across the other three datasets.The models are evaluated zero-shot on Ac-Ala3-NHMe datasets from MD22, ANI-1x, and AIMD-Chig.Source: Table 3 · p. 9, §4.4 · p. 9
4.5 ABLATION STUDY
- Replacing dimension-wise softmax raises formation-energy MAE to 28.4 meV/atom, while one-way attention aggregation and a fixed envelope also degrade performance.Training ablations report gains from denoising pretraining, magmom prediction, and graph-level loss reduction.Source: Table 4 · p. 9, §4.5 · pp. 9–10
4.6 EFFICIENCY-ACCURACY ANALYSIS.
- MatRIS inference is faster than eqV2 and eSEN but slower than MACE-L; on 500 WBM structure relaxations, it balances speed with energy accuracy.The inference-time comparison excludes graph construction; MACE-L uses two layers, versus four in MatRIS-S and six in MatRIS-M.Source: Figure 5 · p. 10, §4.6 · p. 10
5 CONCLUSION AND FUTURE WORK
MatRIS is presented as an invariant MLIP that attains state-of-the-art results across multiple benchmarks. The authors identify validation on larger QM-based datasets, student-model distillation, and long-range electrostatics as future work.
5 CONCLUSION AND FUTURE WORK
- Across multiple benchmarks, MatRIS attains state-of-the-art results.Source: §5 · p. 10
- The authors plan to test MatRIS on larger QM-based datasets, develop a student-MLIP distillation strategy, and add long-range electrostatics for more complex downstream tasks.Source: §5 · p. 10
A.1 EXPLICIT THREE-BODY MODELING IN MLIPS
MatRIS explicitly models three-body interactions with attention and presents a distinct update strategy from other three-body MLIPs. Its details span graph construction, feature embedding, attention, refinement, and readout.
A.1 EXPLICIT THREE-BODY MODELING IN MLIPS
- MatRIS updates three-body features with O(N) attention and uses a learnable envelope to smooth their contributions during aggregation, with both choices reported to improve performance.Source: §A.1 · p. 19
- MatRIS-M achieves accuracy comparable to eSEN-30M-MP with significantly fewer parameters and substantially lower training cost.Source: §A.1 · pp. 19–20
A.2 ATTENTION-BASED MLIPS
- Unlike full-attention MLIPs with O(N^2) cost that typically restrict interactions to two bodies, MatRIS uses O(N) attention for both two- and three-body interactions.Source: §A.2 · p. 20
- MatRIS's separable attention explicitly models three-body interactions, with dimension-wise weights and separate source-to-target and target-to-source attention branches.Source: §A.2 · p. 20
B DETAILS OF MATRIS
- The model builds atom and line graphs, embeds atom types, distances, and three-body angles, then updates the line graph before incorporating its output into atom features.Source: B DETAILS OF MATRIS · pp. 20–21
- A refinement layer applies a learnable envelope to node updates, and the readout uses final atom features to predict total energy and magnetic moments.Source: B DETAILS OF MATRIS · p. 22
C TRAINING STRATEGIES
The training strategies address uneven computational load across GPUs, uneven node-level loss contributions, and denoising pretraining for non-equivariant models on non-equilibrium structures. Load balancing and batch expansion yield reported cumulative speedups.
C.1 LOAD-BALANCE STRATEGY
- Load balancing yields 1.35–1.75× cumulative speedup, rising to 2.05–2.63× with batch expansion.Source: §C.1 · p. 23
C.2 LOSS-BALANCE STRATEGY
- Because large samples can dominate node-level force or magnetic-moment losses, graph-level loss reduction first averages within each graph and then across the batch.Source: §C.2 · p. 23
C.3 DETAILS OF DENOISING PRETRAINING
- For denoising pretraining on non-equilibrium structures, MatRIS projects forces onto relative-position edges to create scalar features, then predicts added Gaussian noise from the noisy structure, projected force, and timestep.Source: §C.3 · pp. 23–24
D.1 MATBENCH-DISCOVERY BENCHMARK (NON-COMPLIANT)
In the non-compliant Matbench-Discovery setting, MatRIS-10M-OAM is evaluated across full, unique, and 10k most stable test splits and is reported to perform competitively despite having fewer parameters than most models.
D.1 MATBENCH-DISCOVERY BENCHMARK (NON-COMPLIANT)
- MatRIS-10M-OAM delivers competitive performance across the full, unique, and 10k most stable splits despite having fewer parameters than most models.Source: §D.1 · p. 24, Table 5 · p. 24
D.2 LAMBENCH BENCHMARK
LAMBench evaluates MatRIS-10M-OAM across force-field prediction and property-calculation tasks spanning molecules, inorganic materials, and catalysis. It performs consistently well across both task groups, with best or highly competitive results.
D.2 LAMBENCH BENCHMARK
- For force-field prediction, MatRIS-10M-OAM achieves the best overall accuracy and generally outperforms other models in force prediction for molecules and inorganic materials.The tasks predict energies, forces, and virials across molecules, inorganic materials, and catalysis.Source: §D.2 · p. 25, Table 8 · p. 25
- In elastic-constant and Wiggle150 conformer-energy prediction, MatRIS-10M-OAM achieves the best performance, while its OC20-NEB reaction-barrier and reaction-energy predictions are worse than other models.Its OC20-NEB success rate remains relatively high; the authors speculate that MAE energy loss without extra outlier weighting amplifies outlier effects in this task.Source: §D.2 · pp. 25–26, Table 9 · p. 26
- Across LAMBench force-field and property tasks, MatRIS-10M-OAM achieves either the best or highly competitive results.The evaluation covers force-field prediction and property calculation.Source: Table 8 · p. 25, §D.2 · p. 26, Table 9 · p. 26
D.3 ZEOLITE BENCHMARK
On the Zeolite benchmark, MatRIS achieves state-of-the-art performance overall. On DPA2, MatRIS-M improves energy and force accuracy overall across diverse domains, although some models have lower force errors in certain systems.
D.3 ZEOLITE BENCHMARK
- On the Zeolite dataset, MatRIS achieves state-of-the-art performance overall.The dataset comprises 16 zeolite structures, with 48,000 training, 16,000 validation, and 16,000 test structures per zeolite.Source: §D.3 · p. 26, Table 10 · p. 26
D.4 DPA2 TEST SET
- DPA2 combines 18 domain-specific subsets generated with different DFT software, with training data ranging from 6K to 2,000K per subset.MatRIS-M uses the same training configurations and training/test sets as Zhang et al. (2024) for comparability.Source: §D.4 · pp. 26–27
- On DPA2 test sets, MatRIS-M achieves better energy and force accuracy overall, though eqV2 and GemNet-OC have lower force errors in certain systems.The authors attribute that advantage as likely related to non-conservative force prediction, which fits energy and forces separately at the expense of physical consistency.Source: §D.4 · p. 27, Table 11 · p. 27
D.5 MD STABILITY EVALUATION
MatRIS-M conserves energy over long NVE simulations, including tests on organic molecules and inorganic materials described as out-of-distribution for its MPTrj training. The evaluation runs for 80 ps at a 1 fs time step.
D.5 MD STABILITY EVALUATION
- The stability evaluation initializes atomic velocities at 300 K using the Maxwell-Boltzmann distribution before running NVE molecular dynamics.Energy drift and thermostatted stability are the stated evaluation metrics.Source: §D.5 · p. 27
- In 80 ps NVE simulations, MatRIS-M shows no large energy drift, with total energy and kinetic temperature fluctuating around their initial values.The simulations use a 1 fs time step on organic molecules and inorganic systems out-of-distribution for MatRIS-M trained on MPTrj.Source: §D.5 · p. 27
E TRAINING DETAILS AND HYPER-PARAMETERS
Training configurations vary model depth and include a cosine-annealed learning rate. Distributed training uses load balancing and graph-level loss reduction, while Matbench-Discovery MatRIS-M and MatRIS-L models receive denoising pretraining.
E TRAINING DETAILS AND HYPER-PARAMETERS
- MatRIS-S, MatRIS-M, and MatRIS-L use 4, 6, and 10 layers, respectively, on MPTrj, with learning rates following cosine annealing to a minimum at the last epoch.Table 12 summarizes model hyperparameters across datasets and versions.Source: E TRAINING DETAILS AND HYPER-PARAMETERS · p. 27, Table 12 · p. 28
- Distributed training uses load balancing to reduce synchronization overhead and graph-level loss reduction to avoid bias toward larger systems.The reported global batch size is an average because samples per GPU may vary.Source: E TRAINING DETAILS AND HYPER-PARAMETERS · p. 28
- For Matbench-Discovery, MatRIS-M and MatRIS-L are pretrained with a denoising process to enhance generalization.Table 12 distinguishes OMat configuration values from sAlex+MPTrj fine-tuning values for MatRIS-OAM; unparenthesized values are shared.Source: E TRAINING DETAILS AND HYPER-PARAMETERS · p. 28, Table 12 · p. 28