Source-linked AI summary
PYTHIA, a transferable machine learning framework for simultaneous defect and adsorption site optimization
Ioannis Kouroudis, Zhaosu Gu, Poonam, Leon Muller, Alessio Gagliardi
TL;DR
Adsorption-site and defect optimization can demand extensive DFT data, limiting machine learning on understudied systems. PYTHIA transfers features from a frozen pretrained model through a Transformer and GP, then uses Bayesian optimization; it reports strong results on two systems, while targeting limited-data problems.
Problem
Mapping adsorption energy landscapes requires many DFT calculations, and the cost rises across compositions and defects; new systems can require substantial data generation before model training.
Source: §1 · p. 1Method
PYTHIA freezes pretrained MACE, adapts its features with a Transformer, uses a Gaussian process for energy and uncertainty, and applies Bayesian optimization to defects and adsorption positions.
Source: §1 · p. 2Results
PYTHIA outperformed AUGUR and fully trained MACE-GP on two adsorption-optimization cases, and identified defect groups absent from training with high accuracy.
Source: §1 · pp. 2–3Takeaways & Limitations
The authors conclude that PYTHIA combines complex-model accuracy with lower data demand and can identify adsorption sites with energies many times better than those in the training set.
Source: §1 · p. 3, §4 · p. 10PYTHIA targets limited datasets; substantially more complex systems may require batch Gaussian processes, which can handle larger datasets at somewhat decreased performance.
Source: §2.4 · p. 5
Abstract
from arXiv · showhide
Machine learning has permeated material optimization, providing high value. Nevertheless, its re- liance on a high number of samples limits its applicability to relatively explored material families. In this paper we extend the applicability of machine learning to novel and understudied materials through the PYTHIA (Probabilistic Yet Transferable Hierarchical Interaction Architecture) framework. Pythia utilizes two of the most powerful machine learning concepts, foundational models and transfer learning. We employ a complex foundational model trained on a large dataset of chemically similar molecules, on a target that is physically close to our prediction target. Then, we freeze the weights and perform subspace alignment with the use of a transformer layer and add uncertainty quantification with gaussian processes. These two layers can be trained on a limited dataset yet allow us to leverage the gained knowl- edge and the studied materials and complex architectures. The resulting model is used as a surrogate for a Bayesian Optimization scheme. To showcase the robustness of the framework, we applied it to the investigation of adsorption and defect optimization of two distinct systems. Not only were we able to correctly and efficiently identify the optimal adsorption sites of even complex a
1 Introduction
Searching adsorption structures becomes costly as compositions and defects vary, because mapping energy landscapes requires many DFT calculations. PYTHIA transfers pretrained atomic features to target systems and uses uncertainty-aware Bayesian optimization; it is evaluated on two adsorption problems.
1 Introduction
- DFT energy-surface mapping requires many molecular positions and orientations, and becomes more costly when multiple compositions and defect structures are studied.Adsorption structures depend on atomic composition and local arrangement.Source: §1 · p. 1
- For new adsorption systems, generating enough DFT training data can account for much of the total computational cost, sometimes requiring a full investigation before model training..Source: §1 · p. 2
- Structural-coordinate representations may need redesign when molecular composition, substrate, or defects change, motivating representations learned directly from atomic structures..Source: §1 · p. 2
- PYTHIA combines pretrained MACE features, a Transformer translation layer, Gaussian-process prediction and uncertainty, and Bayesian optimization for adsorption-site and defect searches.MACE remains frozen while the Transformer and GP are trained on target-system data.Source: §1 · p. 2
- PYTHIA is tested on simultaneous defect and adsorption-position optimization for Zn on defective Chini clusters and formaldehyde on buckled silicene, outperforming AUGUR and fully trained MACE-GP by a significant margin.The introduction describes the two systems as distinct local chemical environments.Source: §1 · p. 2
- PYTHIA predictions are reported as highly accurate even for defect groups absent from training, and optimization identifies adsorption sites with energies many times better than those in the training set.The authors present data-efficient transfer as a way to reduce time-consuming data acquisition.Source: §1 · p. 3
2 Methods
The methods combine a predictive pipeline with physically constrained candidate generation and Bayesian optimization. A frozen foundational model is adapted through a translation layer and GP to predict adsorption energies and uncertainty from limited target data.
2.1 PYTHIA pipeline
- The PYTHIA pipeline predicts an interaction-energy surface and corresponding uncertainty, which inform whether another DFT round is worthwhile.The full optimization also combines prediction with defect and placement permutations.Source: §2.1 · p. 3, Figure 1 · p. 3
2.2 Bayesian Optimization
- Bayesian optimization combines a stochastic predictor with an acquisition function that balances candidate optimality and information gain, shifting from exploration toward exploitation.The authors generate and predict 10,000 systems to maintain strict control over physical feasibility.Source: §2.2 · p. 4
2.3 Point generation
- Candidate generation samples defect substitutions, adsorption positions on physically bounded spheres, and random adsorbate rotations, then rejects geometries that penetrate the cluster or violate distance tolerances.Accepted systems are added to the sample set for evaluation.Source: §2.3 · pp. 4–5
2.4 Gaussian Processes
- Gaussian processes provide predictions with inherent uncertainty by modeling observations with Gaussian prior and likelihood distributions and a kernel-defined covariance.Kernel parameters may be optimized using marginal log likelihood.Source: §2.4 · p. 5
2.5 Foundational Model
- The framework uses MACE because the chemical space is too large for a simple GNN, while GP methods struggle with high-dimensional feature spaces.Other architectures, including NequIP, could also be integrated.Source: §2.5 · p. 6
2.6 Translation Layer
- A Transformer translation layer adapts rich intermediate MACE representations to the target system; its output feeds a GP for energy and uncertainty predictions while MACE stays frozen.This avoids retraining MACE, which the authors describe as computationally expensive and data inefficient.Source: §2.6 · p. 6
3 Results
PYTHIA jointly optimizes defect choices and adsorption positions across Chini clusters and silicene, outperforming comparator models while identifying stronger-binding configurations. The results include accurate predictions on unseen defect configurations and full energy-surface estimates.
3.1 Zn adsorption on Chini Clusters
- Across both cases, optimization jointly varied adsorbate position and defect choices; the Chini search considered Ni and Pd substitutions in clusters with n = 1 and 2.PYTHIA combines frozen pretrained MACE features with a Transformer and Gaussian process.Source: §3 · p. 6, §3.1 · p. 7
- R2 0.83, MAE 0.33 eV, and RMSE 0.37 eV: PYTHIA most closely matched reference energies, improving 80–100% over the next-best model depending on the metric.Figure 2 compares predictions for defect configurations absent from training.Source: Figure 2 · p. 7, §3.1 · p. 7
- −6.1 eV: after one optimization run, PYTHIA identified a top or symmetric-bottom Zn site for Pd3, versus a dataset optimum of −4.74 eV.The model also supplied the full energy surface and uncertainty estimates.Source: §3.1 · p. 7, Figure 3 · p. 8
3.2 Formaldehyde adsorption on Silicene
- Silicene’s search included multiple dopant classes and simultaneous substitutions; numerical stability constrained the tested adsorbents to at most three defects.The authors state this defect limit is adjustable and not an inherent PYTHIA limitation.Source: §3.2 · pp. 8–9
- PYTHIA significantly outperformed AUGUR and fully trained MACE on silicene adsorption-energy prediction; AUGUR failed in the complex search, while limited data precluded full MACE training.Figure 4 compares prediction accuracy for the defective Silicene-CH2O system.Source: §3.2 · pp. 8–9, Figure 4 · p. 9
- -1.12 eV versus -0.332 eV: in two iterations, PYTHIA found a best silicene adsorbent with a reported improvement of more than 230% over the original dataset.The full energy surface is also available in minutes; its visual discontinuities arise from formaldehyde rotation and do not affect optimization.Source: §3.2 · p. 9, Figure 5 · p. 10
4 Discussion
The discussion presents PYTHIA as a modular, transferable framework for interaction-site optimization, with potential applications beyond the two demonstrated systems when adjacent, richly labeled data are available.
4 Discussion
- PYTHIA outperformed benchmarks in both tested systems and was accurate for defect configurations absent from training, supporting transfer beyond the observed configurations.The framework combines a frozen pretrained MACE model, a Transformer translation layer, and a Gaussian Process head.Source: §4 · p. 10
- The authors suggest broader use in catalyst design, gas sensing, and surface functionalization where DFT data are scarce but adjacent, richly labeled datasets exist.They also state that MACE can be replaced by newer models and that the framework can target solvents as well as defects and adsorption sites.Source: §4 · p. 10
- The proposed scope includes settings where large datasets do not exist and prediction accuracy matters alongside optimization results.Source: §4 · p. 11