Source-linked AI summary
Message-passing neural networks for high-throughput polymer screening
Peter C. St. John, Caleb Phillips, Travis W. Kemper, A. Nolan Wilson, Michael F. Crowley, Mark R. Nimlos, Ross E. Larsen
TL;DR
High-throughput molecular screening is constrained by the cost of DFT calculations and the optimized 3D geometries often used by molecular graph models. The study develops an OPV dataset and evaluates MPNNs with and without 3D inputs; their predictions are similar, and transferred weights improve performance on another DFT functional, even for weakly correlated tasks.
Problem
DFT calculations and optimized 3D geometries are costly for high-throughput screening, while hand-engineered molecular features may not capture all variability in chemically feasible compounds.
Source: §I · p. 1Method
The study builds a 91,000-molecule OPV dataset and trains MPNNs on molecular structures with and without 3D information, also transferring weights between DFT functionals.
Source: §I · p. 2Results
2D-only MPNNs achieved similar error distributions to 3D models and closely approached conformer-based limits; 2D single-task εHOMO MAE was 32.1 meV, and transferred weights improved predictions even for weakly correlated tasks.
Source: A. Prediction performance on… · p. 4, §IV · p. 6Takeaways & Limitations
For the molecules studied, near-optimal predictions can be obtained without optimized 3D geometry, and transferred representations can improve prediction with limited data.
Source: §I · p. 2, §IV · p. 6
Abstract
from arXiv · showhide
Machine learning methods have shown promise in predicting molecular properties, and given sufficient training data machine learning approaches can enable rapid high-throughput virtual screening of large libraries of compounds. Graph-based neural network architectures have emerged in recent years as the most successful approach for predictions based on molecular structure, and have consistently achieved the best performance on benchmark quantum chemical datasets. However, these models have typically required optimized 3D structural information for the molecule to achieve the highest accuracy. These 3D geometries are costly to compute for high levels of theory, limiting the applicability and practicality of machine learning methods in high-throughput screening applications. In this study, we present a new database of candidate molecules for organic photovoltaic applications, comprising approximately 91,000 unique chemical structures.Compared to existing datasets, this dataset contains substantially larger molecules (up to 200 atoms) as well as extrapolated properties for long polymer chains. We show that message-passing neural networks trained with and without 3D structural information for these molecules achieve similar accuracy, comparable to state-of-the-art methods on existing benchmark datasets. These results therefore emphasize that for larger molecules with practical applications, near-optimal prediction results can be obtained without using optimized 3D geometry as an input. We further show that learned molecular representations can be leveraged to reduce the training data required to transfer predictions to a new DFT functional.
I. INTRODUCTION
DFT remains too slow for screening vast molecular libraries, while 3D geometries required for top-performing molecular graph models are costly to obtain. This study applies MPNNs to a large OPV dataset and investigates 2D-only prediction and transfer learning.
I. INTRODUCTION
- Machine learning predictions are nearly six orders of magnitude faster than DFT, but DFT remains too slow to screen vast chemical space.The comparison is O(10^-3s) for ML versus O(10^3s) for DFT on molecules with approximately 30 heavy atoms.Source: §I · p. 1
- MPNNs learn from molecular graphs, but achieving optimal QM9 accuracy has typically required optimized 3D geometries that are costly to obtain.That requirement limits high-throughput applications, particularly for larger molecules.Source: §I · pp. 1–2
- The study applies MPNNs to a new dataset of 91,000 OPV molecules to predict optoelectronic properties directly from 2D structure.The pipeline avoids requiring DFT-optimized 3D structures as model inputs.Source: §I · p. 2
- The authors report near-equivalent performance with and without spatial information, and improved predictions on another DFT functional using transferred weights, including for weakly correlated tasks.They also make the dataset, which includes 2D and 3D structures, available for future graph-network development.Source: §I · p. 2
A. Dataset preparation
The dataset combines OPV-relevant molecular structures and DFT properties, including polymer-limit extrapolations. Its preparation defines the evaluation splits and a conformer-based error reference for models using connectivity alone.
A. Dataset preparation
- The database includes monomer optoelectronic properties, polymer-limit extrapolations, and soluble small molecules, with small molecules excluded from validation and test sets.The two largest functional/basis subsets are B3LYP/6-31g(d) and CAM-B3LYP/6-31g.Source: A. Dataset preparation · p. 2
- The B3LYP/6-31g(d) dataset contains approximately 91,000 unique molecules, while the CAM-B3LYP/6-31g transfer-learning dataset contains approximately 32,000.For each, validation and test molecules were selected from entries with polymer properties; training data came from the remaining small-molecule and monomer results.Source: A. Dataset preparation · p. 2
- Conformer-pair MAE establishes a lower-limit accuracy reference for connectivity-only models, which cannot distinguish molecules with identical SMILES but different 3D conformations.A single optimization, rather than conformational sampling, was performed for each molecule or oligomer to support screening more molecules.Source: A. Dataset preparation · pp. 2–3
B. Message Passing Architecture
The MPNN represents molecules using atom and bond features derived from SMILES, passes learned messages along graph connectivity, and converts final atom states into property predictions.
B. Message Passing Architecture
- SMILES-derived inputs encode atom types, bond types, and connectivity for molecules as large as 201 atoms and 424 bonds.Atom and bond classes incorporate chemical attributes including aromaticity, bonding, conjugation, and ring membership.Source: B. Message Passing Architecture · p. 3
- In message passing, learned messages travel between neighboring atoms, and a gated recurrent unit implements the node-state update.The atom-state dimension is 128, with three message-passing layers.Source: B. Message Passing Architecture · p. 3
- The readout uses final atom states to generate a graph-level feature vector, which is passed through fully connected layers to predict properties.The graph vector has dimension 1024, followed by layers of dimensions 512 and 256; the 3D SchNet-like models use an average readout.Source: B. Message Passing Architecture · p. 3
C. Software
The study implements message passing with Keras and TensorFlow and uses Scikit-learn and RDKit for target scaling and molecular encoding.
C. Software
- Keras and TensorFlow implement the message-passing operations.Source: C. Software · p. 3
- Scikit-learn scales prediction targets, while RDKit encodes atoms and bonds as integer classes.Source: C. Software · p. 3
- The MPNN implementation is available on GitHub and installable via pip; datasets, scripts, and trained model weights are also made available.Source: C. Software · p. 3
D. Hyperparameter optimization
The 2D message-passing model pools atom representations into molecular predictions, with hyperparameters selected to meet the target error while fitting on one GPU.
D. Hyperparameter optimization
- Model size was increased until training error fell below the target while fitting on one Tesla K80 GPU with batch size 100.The best learning rate was 1E-3; models trained for 500 epochs with a 2E-6 per-epoch decay.Source: D. Hyperparameter optimization · p. 3
- Validation loss was checked every 10 epochs, and early stopping retained the model with the lowest validation loss.Dropout and l2 regularization did not decrease validation loss.Source: D. Hyperparameter optimization · p. 3
- Atom and bond types initialize message-passing layers; summing atom outputs creates molecule-level features for dense layers that produce predictions.The figure shows multi-task layer dimensions; single-task models differ only in the final output dimension, which is 1.Source: Figure 1 · p. 4
A. Prediction performance on B3LYP/6-31g(d) results
For B3LYP/6-31g(d) results, 2D models approached optimized-3D accuracy and outperformed models using approximate UFF geometries; more training data brought performance toward the estimated optimum.
A. Prediction performance on B3LYP/6-31g(d) results
- For εHOMO, the 2D single-task MAE was 32.1 meV versus 36.7 meV for QM9's best model; 2D and optimized-3D errors were similar overall.Table I reports test-set MAEs; the conformer column represents an optimal error rate for 2D models.Source: Table I · p. 4, A. Prediction performance on… · p. 4
- Optimized-3D errors were smaller for monomer and dimer properties but slightly larger for extrapolated polymer properties than 2D errors.The authors suggest polymer properties may depend less on the monomer's precise 3D configuration.Source: A. Prediction performance on… · p. 4
- UFF-geometry models were less accurate than 2D models, indicating that poor-quality geometries performed worse than omitting 3D features.UFF re-optimization was used to obtain approximate 3D coordinates at lower computational cost.Source: A. Prediction performance on… · p. 4
- As training-set size increased, the 2D model's out-of-sample performance improved and asymptotically approached the optimal error rate.The model architecture was held constant across the subsampled-data experiments.Source: A. Prediction performance on… · pp. 4–5
B. Transfer learning to an alternate DFT functional
Transfer learning from B3LYP/6-31g(d) tasks improved prediction of CAM-B3LYP/6-31g polymer band gaps, especially with strongly correlated pretraining and limited training data.
B. Transfer learning to an alternate DFT functional
- The transfer experiment used a smaller CAM-B3LYP/6-31g polymer-band-gap dataset, with validation and test sets of 2,000 polymer species reserved.Transferred weights came from B3LYP/6-31g(d) polymer- or monomer-band-gap models and were compared with random initialization.Source: B. Transfer learning to an alternate… · p. 5
- At roughly 200 training molecules, weakly correlated transferred weights produced substantially lower test error than the estimated upper error bound.The weakly correlated source task was monomer band gap.Source: B. Transfer learning to an alternate… · p. 5
- Strongly correlated transfer achieved RMSE values of 100 meV with nearly an order of magnitude less data and outperformed other models at every training-set size.The strongly correlated source task was polymer band gap; its correlation with the target was 0.93, compared with 0.48 for monomer band gap.Source: B. Transfer learning to an alternate… · pp. 5–6
- With larger training sets, both transfer models fell below their estimated upper error bounds; at the largest sizes, all models approached the optimal error rate.The authors report that knowledge in transferred weights is eventually replaced by knowledge from the new training data.Source: B. Transfer learning to an alternate… · p. 6
IV. CONCLUSIONS
The conclusions report near-equivalent MPNN prediction accuracy from 2D and 3D features, close to the estimated 2D lower-bound error. They also report improved predictions after transferring weights between DFT functionals, especially with limited data.
IV. CONCLUSIONS
- 2D- and 3D-feature MPNNs achieved near-equivalent prediction accuracy, both close to the estimated 2D lower-bound error from conformational optimization.The authors contrast this with QM9, where accurate predictions have required 3D coordinates.Source: §IV · p. 6
- QM9's 3D-coordinate advantage may reflect its smaller molecules (≤29 atoms) and complex structures, unlike OPV database molecules reaching 201 atoms.The authors suggest the public OPV database may be a more representative molecular-learning benchmark for electronic-structure calculations.Source: §IV · p. 6
- Transfer gains depended on task correlation, but weights from even weakly correlated DFT tasks improved related-functional prediction, especially with limited data.The authors identify prediction on experimental data and other targets with limited data as future work.Source: §IV · p. 6