Source-linked AI summary
SMILES2Vec: An Interpretable General-Purpose Deep Neural Network for Predicting Chemical Properties
Garrett B. Goh, Nathan O. Hodas, Charles Siegel, Abhinav Vishnu
TL;DR
Chemical-property prediction is limited by reliance on engineered features and opaque models, despite the need for accurate and interpretable methods. The paper develops SMILES2vec, which learns directly from SMILES and uses Bayesian optimization to tune a general-purpose neural architecture. It reports performance across toxicity, activity, solvation energy, and solubility, with an explanation mask reaching 88% top-3 accuracy on solubility interpretation.
Problem
Chemical-property modeling relies heavily on engineered features and black-box models, limiting learned representations and making predictions difficult to interpret.
Method
SMILES2vec is a deep neural network that learns features directly from SMILES, with Bayesian optimization for architecture selection and an explanation mask for prediction interpretation.
Results
SMILES2vec predicted toxicity, activity, solvation energy, and solubility, while its solubility explanation mask achieved 88% top-3 accuracy and outperformed typical engineered-feature MLP models.
Takeaways & Limitations
SMILES2vec provides a general-purpose, interpretable approach for learning chemical properties from chemical text without feature engineering.
Takeaways & Limitations
Validation and test metrics correlated at 0.54 for Tox21 and 0.78 for FreeSolv, indicating dataset-dependent differences during hyperparameter optimization.
Abstract
from arXiv · showhide
Chemical databases store information in text representations, and the SMILES format is a universal standard used in many cheminformatics software. Encoded in each SMILES string is structural information that can be used to predict complex chemical properties. In this work, we develop SMILES2vec, a deep RNN that automatically learns features from SMILES to predict chemical properties, without the need for additional explicit feature engineering. Using Bayesian optimization methods to tune the network architecture, we show that an optimized SMILES2vec model can serve as a general-purpose neural network for predicting distinct chemical properties including toxicity, activity, solubility and solvation energy, while also outperforming contemporary MLP neural networks that uses engineered features. Furthermore, we demonstrate proof-of-concept of interpretability by developing an explanation mask that localizes on the most important characters used in making a prediction. When tested on the solubility dataset, it identified specific parts of a chemical that is consistent with established first-principles knowledge with an accuracy of 88%. Our work demonstrates that neural networks can learn technically accurate chemical concept and provide state-of-the-art accuracy, making interpretable deep neural networks a useful tool of relevance to the chemical industry.
1 INTRODUCTION
Chemical design remains constrained by incomplete chemical understanding, while feature-engineered and black-box models limit learnable representations and interpretability. SMILES2vec addresses these gaps by learning directly from chemical text and adding an explanation mask for broad property prediction.
- Chemical design remains a bottleneck because computational methods face technical limitations and incomplete understanding of underlying chemical concepts.
- QSAR/QSPR models commonly predict chemical properties from engineered molecular descriptors and fingerprints developed from chemical knowledge.
- Feature engineering can limit neural networks’ search space, especially when domain knowledge is incomplete or engineered features are inadequate.
- Black-box models are difficult to interpret, although explanations matter for regulated chemical products and can support chemical hypothesis formation.
- SMILES2vec learns from chemical text, adds an explanation mask, and targets chemical-property prediction across multiple industries without explicit feature engineering.
2 METHODS
SMILES2vec encodes SMILES strings as chemical text and uses neural-network training, Bayesian architecture optimization, and standardized evaluation across four chemical-property datasets. The study evaluates toxicity, activity, solvation energy, and solubility while controlling data splits and class imbalance.
- SMILES encodes chemical structure as compact text using atom symbols, bond characters, ring numbers, and parentheses for side chains.
- The study evaluates SMILES2vec on Tox21, HIV, FreeSolv, and ESOL for toxicity, activity, solvation free energy, and solubility.
- SMILES strings longer than 250 characters were excluded because this length encompassed 99.9% of ChEMBL entries.
- The preprocessing mapped unique characters to one-hot encodings, zero-padded strings to 250 characters, and added 10 zeros on each side.
- A separate test set was held out, while the remaining data underwent random 5-fold cross-validation; minority classes were oversampled after stratification.
- Bayesian optimization selected network topologies using validation AUC for classification and RMSE for regression.
- Reported performance is the mean metric across five runs in 5-fold cross-validation, using AUC for classification and RMSE for regression.
3 EXPERIMENTS
The experiments optimize SMILES2vec’s architecture across chemical-property tasks, then develop an explanation mask that identifies influential SMILES characters while preserving the base model’s output.
- Architecture exploration: SMILES2vec models SMILES as sequence-to-vector prediction, requiring architecture designs specialized to chemical text rather than conventional NLP methods.The explored designs combine embedding layers with bidirectional GRU or LSTM layers, optionally preceded by a 1D convolution.
- Architecture exploration: Four architectural classes were investigated: GRU, LSTM, CNN-GRU, and CNN-LSTM, with separate Bayesian optimization of their hyperparameters.The search varied embedding, recurrent, and convolutional unit counts while using fixed convolution size and stride.
- Bayesian optimization: 60 trials and 6 manually seeded designs were used for each architectural class, while optimization was performed separately on Tox21 classification and FreeSolv regression.The separate task optimizations addressed the goal of selecting a reusable general-purpose network under limited computing resources.
- Bayesian optimization: For Tox21, convolutional architectures improved performance over their recurrent counterparts, with CNN-LSTM best and CNN-GRU slightly behind; for FreeSolv, GRU-based networks outperformed LSTM-based networks.CNN-GRU was selected for the remainder of the work with generalization to other chemical properties in mind.
- Bayesian optimization: Validation–test metric correlations were 0.54 for Tox21 and 0.78 for FreeSolv, indicating different degrees of alignment during Bayesian optimization.The lower Tox21 correlation may reflect optimizing AUC while training used crossentropy loss.
- Interpretability: The explanation mask is trained with a frozen SMILES2vec base network to retain near-identical predictions while masking as much input data as possible.The mask is produced over the embedded SMILES input, and L2 plus entropy regularization discourages uniform masks.
4 PERFORMANCE
SMILES2vec learns interpretable representations from SMILES and generalizes across chemical-property datasets. Its explanation mask aligns attention with hydrophobic and hydrophilic groups, while performance extends beyond solubility to toxicity, activity, and solvation energy.
- 4.1 Interpreting Chemical Solubility: 88% top-3 accuracy was achieved when the explanation mask identified expected atoms associated with soluble and insoluble compounds.The evaluation used soluble compounds with values > -1.0 and insoluble compounds with values < -5.0.
- 4.1 Interpreting Chemical Solubility: Low-solubility molecules received greater attention on c, C, and Cl, whereas high-solubility molecules received greater attention on O and N.These characters correspond to hydrophobic and hydrophilic groups, respectively.
- 4.1 Interpreting Chemical Solubility: The mask localized appropriate atoms without explicit chemical information or further feature engineering, indicating that SMILES2vec learned representations corresponding to known chemistry concepts.The base network was frozen while a separate network learned a mask preserving the base prediction while masking input data.
- 4.2 Generalization of SMILES2vec Models: SMILES2vec generalized to Tox21, HIV, and FreeSolv with validation AUCs of 0.80 and 0.78 and RMSE of 1.4 kcal/mol, respectively.Pre-training slightly improved these results to AUCs of 0.81 and 0.80 and RMSE of 1.2 kcal/mol.
- 4.2 Generalization of SMILES2vec Models: Bayesian optimization used approximately 500 trials, fewer than the approximately 10,000 trials typically required by alternative black-box architecture-design methods.The authors conclude that the optimized design functions as a general-purpose SMILES2vec network for other chemical properties.
- 4.2 Generalization of SMILES2vec Models: Against contemporary models, SMILES2vec outperformed engineered-feature MLPs, outperformed image CNNs on classification, and matched convolutional graphs on Tox21 and HIV classification.It slightly underperformed image CNNs on regression tasks and either matched or exceeded convolutional graph performance depending on the task.
5 CONCLUSION
SMILES2vec is a general-purpose neural network that predicts chemical properties directly from SMILES while providing an explanation mask for interpretability. It achieved strong performance across toxicity, activity, solvation energy, and solubility, and its explanations identified chemically meaningful functional groups.
- 5 CONCLUSION: SMILES2vec uses chemical text data and an explanation mask to predict chemical properties while improving interpretability.The network architecture was selected through Bayesian optimization experiments.
- 5 CONCLUSION: Validation performance reached AUC 0.81 for Tox21, AUC 0.80 for HIV, RMSE 1.2 kcal/mol for solvation energy, and RMSE 0.63 for solubility.On solubility, the explanation mask localized hydrophilic and hydrophobic groups with 88% top-3 accuracy.
- 5 CONCLUSION: SMILES2vec outperformed engineered-feature MLPs, outperformed convolutional graph networks on regression tasks, and matched them on classification tasks.The results support learning technically accurate chemical concepts directly from SMILES.