Source-linked AI summary
Chemception: A Deep Neural Network with Minimal Chemistry Knowledge Matches the Performance of Expert-developed QSAR/QSPR Models
Garrett B. Goh, Charles Siegel, Abhinav Vishnu, Nathan O. Hodas, Nathan Baker
TL;DR
The paper asks whether deep learning can reduce reliance on expert-engineered chemistry features in QSAR/QSPR modeling. It develops Chemception, a deep CNN trained on 2D molecular drawings with minimal explicit chemistry knowledge, and reports comparable performance across activity, solvation, and toxicity tasks. The authors conclude that deep neural networks can plausibly assist computational chemistry research by learning much of the feature-engineering process.
Problem
The paper addresses whether deep learning can reasonably replace human intelligence and expert feature engineering in computational chemistry applications.
Method
Chemception is a deep convolutional neural network that predicts chemical properties from 2D molecular structures without additional explicit chemistry knowledge.
Results
Chemception achieved a validation/test RMSE of 1.51/1.74 kcal/mol for free-energy-of-solvation prediction, close to physics-based methods with RMSE ~1.5 kcal/mol.
Takeaways & Limitations
The results support using deep neural networks as computational-chemistry tools that can perform feature engineering primarily through learned representations.
Takeaways & Limitations
The authors note that further optimization may be possible because more features could be learned at some network levels than others.
Abstract
from arXiv · showhide
In the last few years, we have seen the transformative impact of deep learning in many applications, particularly in speech recognition and computer vision. Inspired by Google's Inception-ResNet deep convolutional neural network (CNN) for image classification, we have developed "Chemception", a deep CNN for the prediction of chemical properties, using just the images of 2D drawings of molecules. We develop Chemception without providing any additional explicit chemistry knowledge, such as basic concepts like periodicity, or advanced features like molecular descriptors and fingerprints. We then show how Chemception can serve as a general-purpose neural network architecture for predicting toxicity, activity, and solvation properties when trained on a modest database of 600 to 40,000 compounds. When compared to multi-layer perceptron (MLP) deep neural networks trained with ECFP fingerprints, Chemception slightly outperforms in activity and solvation prediction and slightly underperforms in toxicity prediction. Having matched the performance of expert-developed QSAR/QSPR deep learning models, our work demonstrates the plausibility of using deep neural networks to assist in computational chemistry research, where the feature engineering process is performed primarily by a deep learning algorithm.
1. Introduction
Deep learning can learn representations and features directly from input data, motivating its use in chemistry to reduce reliance on expert-engineered descriptors and fingerprints. Chemception applies this approach to 2D molecular drawings and achieves performance comparable to expert-developed QSAR/QSPR models across toxicity, activity, and solvation prediction.
- Motivation: Deep learning learns hierarchical representations and task-relevant features from input data, rather than relying entirely on manually engineered features.These learned representations are conceptually similar to molecular descriptors and fingerprints in chemistry.
- Motivation: Chemistry QSAR/QSPR models traditionally use expert-developed molecular descriptors and fingerprints as inputs to machine-learning algorithms.Deep neural networks have also been used within this traditional feature-engineering paradigm.
- Contribution: The paper proposes using deep learning as a machine-intelligence tool to assist or potentially replace resource-intensive human-led feature engineering.The proposed paradigm uses raw 2D molecular drawings requiring no more than high-school-level chemistry knowledge to create.
- Contribution: Chemception predicts physical, biochemical, and physiological chemical properties from molecular images while learning its own representations without advanced chemistry knowledge.The evaluated properties include solvation free energies, in vitro HIV activity, and in vivo toxicity.
- Results: Chemception performs on par with expert-developed QSAR/QSPR models based on molecular descriptors, fingerprints, and other engineered features.The architecture is reported to achieve comparable performance for toxicity, activity, and solvation predictions, with virtually zero effort spent developing an appropriate feature set.
- Implications: The same architecture performs similarly across different chemical properties without requiring network-design tuning for each property, suggesting a general-purpose neural network.The authors also anticipate that the domain-agnostic approach may transfer to other research fields when suitable pictorial representations exist.
2. Methods
Chemception converts molecular representations into image arrays and trains a convolutional neural network to predict chemical properties. The workflow uses 2D chemical structures and measured properties without additional chemistry-inspired descriptors or fingerprints.
- Computational Framework Overview: The Chemception workflow converts SMILES strings into 2D molecular structures and maps them onto an input array for supervised CNN training.The workflow requires minimal chemical knowledge beyond generating the 2D structures.
- Computational Framework Overview: The model uses chemical images and measured chemical properties while excluding additional chemistry-inspired features such as molecular descriptors and fingerprints.The network is trained to predict toxicity, activity, and solvation properties.
2.2 Data Preparation
Data preparation decodes SMILES strings into 2D molecular structures, maps atoms and bonds onto an 80 x 80 grid, and encodes the resulting image for CNN training.
- Data Preparation: SMILES strings are compact molecular-structure representations that are decoded into corresponding 2D structures using cheminformatics software.The preparation uses OpenBabel, Pybel, and RDKit through the Cinfony interface.
- Data Preparation: Each 2D molecular structure is mapped onto an 80 x 80 grid with 0.5 Å pixel resolution.The resulting array represents the discretized molecular image used for subsequent processing.
- Data Preparation: The molecular grid is greyscale-coded according to atoms and bonds, with atom values based on atomic mass units.Bonds and atoms are assigned numerical values, while empty grid regions default to 0 and non-element-associated mapped portions receive 2.
- Data Preparation: The resulting discretized molecular image is parsed into a deep convolutional neural network for training.This completes the transformation from molecular structure to CNN input.
2.3 Chemistry Knowledge Used
Chemception uses molecular images without explicit chemistry-inspired features such as descriptors, fingerprints, valency, or periodicity. Its predictions therefore depend on learning relevant representations and features directly from the image data.
- The data-preparation workflow used open-source cheminformatics software, but its role was limited to preparing data and supporting database compatibility.
- No molecular descriptors or fingerprints were used to train Chemception.
- Chemception was not explicitly provided basic chemical concepts such as valency or periodicity.
- The molecular representation places atom information, bond information, and empty space in the same image channel.
- Chemception must learn relevant representations and automatically engineer features analogous to molecular descriptors for predicting chemical properties.
2.4 Dataset Description
Chemception was evaluated on public datasets spanning toxicity, activity, and physical or biochemical properties. The datasets included Tox21, HIV, and FreeSolv, with sizes and targets differing across tasks.
- Chemception was evaluated using several publicly available datasets, summarized in Table 1.
- Tox21 contains 8014 compounds measured across 12 toxicity-related endpoints, including stress-response pathways and nuclear receptors.
- Tox21 defines a binary classification task labeling molecules as toxic or non-toxic.
- The study also examined datasets covering physical and biochemical properties, including HIV activity and hydration free energy.
- FreeSolv comprises 643 compounds with measured small-molecule hydration free energies ranging from –25.5 to 3.4 kcal/mol.
2.5 Dataset Preprocessing
The preprocessing protocol used held-out test sets, random five-fold cross-validation, and stratification for classification tasks. Minority classes were oversampled after stratification without overlap across dataset splits.
- Chemception training used a 5-fold cross-validation protocol, with validation performance guiding evaluation and early stopping.
- Separate test sets were used to assess generalizability.
- The Tox21 and HIV datasets reserved 1/6th for testing, while FreeSolv reserved 1/10th because of its smaller size.
- The remaining data underwent random 5-fold cross-validation, with stratification enforced for the Tox21 and HIV classification tasks.
- Minority classes were oversampled after stratification to address class imbalance.
- Repeated observations could occur within a split, but data across different training, validation, and test sets did not overlap.
2.6 Deep Neural Networks
Deep neural networks learn representations by adjusting neuron weights to minimize prediction error, using nonlinear layers and backpropagation. CNNs specialize this process for images, and Chemception applies an Inception-ResNet-style architecture to molecular drawings.
- Deep neural networks: A neural network trains by adjusting neuron weights to minimize prediction error.
- Deep neural networks: Neural-network layers represent inputs as vectors, apply matrix-vector multiplication, and then use nonlinear activation functions.
- Deep neural networks: Chemception uses ReLU activations, whose functional form enables training many layers as a deep neural network.
- Deep neural networks: Backpropagation works backward from the predicted output, using gradient descent to minimize the loss function and update network weights.
- Convolutional neural networks: CNNs process image data with local filters and shared weights, preserving spatial structure while detecting features in different locations.
- Chemception architecture: Chemception uses a stem segment, sequential Inception-ResNet segments, and reduction segments, with architectural details specified for its blocks.
2.7 Convolutional Neural Network Design
Chemception adapts the Inception-ResNet v2 image-processing architecture to chemical images, retaining its multi-scale and residual design while reducing and tuning its structure for molecular data.
- Chemception is based on the Inception-ResNet v2 architecture, combining Inception modules with deep residual learning.
- The network processes inputs through a stem, Inception-ResNet A blocks, reduction blocks, B and C blocks, global pooling, and a task-specific output layer.Classification uses softmax, whereas regression uses a linear output layer.
- The design retains Inception-ResNet block regularities while optimizing block counts and layer widths for chemical images.A reference layer determines the convolutional filter counts of other layers within each block.
- Chemception replaces Inception-ResNet v2’s seven-layer stem with one convolutional layer using size (4, 4) and stride (2, 2).
- The study constructs a shortest baseline, Chemception T1, and tests nine additional architecture iterations with varying depths and widths.
2.8 Chemception Training Protocol
Chemception uses a two-stage optimization and validation protocol with regularization, dataset-specific training scales, and task-specific evaluation metrics.
- 5-fold cross validation reports the mean and standard deviation of each performance metric across five runs.
- Chemception is trained in two stages: RMSprop for 50 epochs followed by momentum SGD fine-tuning for another 50 epochs.The initial learning rate is 10^-3 in both stages, with exponential decay during fine-tuning.
- Training sizes range from approximately 580 FreeSolv compounds to approximately 66,000 HIV compounds, with Tox21 tasks using approximately 8,000–12,000 compounds.
- Early stopping saves the best validation-loss model when validation loss fails to improve for 25 epochs, reducing overfitting risk.
- Tox21 and HIV use AUC, while FreeSolv uses RMSE as the evaluation metric.
2.8 Data Augmentation on Input Images
Data augmentation increases training variation while preserving molecular information, primarily through rotation rather than potentially destructive cropping.
- Molecule images are sparse, with more than 90% of pixels equal to zero and useful information localized to a small fraction of the image.
- Approximately 10 pixels can encode a hydroxyl group and its local chemical environment, representing only 0.156% of an 80 x 80 image.
- Random cropping is excluded because it may remove part of a molecule and reduce data fidelity.
- Each image is randomly rotated between 0 and 180 degrees before being processed by Chemception.
- Rotation augmentation also facilitates learning rotational invariance of chemical structures.
3. Results and Discussion
Chemception applies computer-vision architecture to sparse molecular images and achieves competitive results across toxicity, activity, and solvation tasks, despite minimal explicit chemistry knowledge.
- Architecture rationale: Chemception uses Inception modules to link representations across spatial scales, from atoms and functional groups to fragments and whole molecules.
- Tox21: Tox21 performance spans validation AUC values from 0.702 to 0.834 across individual toxicity measurements, with T3 reaching 0.768 overall.
- Tox21: Chemception achieves validation/test AUC of 0.768/0.773 on Tox21 versus 0.777/0.799 for a multi-task MLP DNN using engineered features.
- HIV: On HIV, Chemception achieves validation/test AUC of 0.744/0.752, outperforming a single-task MLP DNN at 0.742/0.715.
- FreeSolv: On FreeSolv, finalized Chemception reaches validation/test RMSE of 1.51/1.75 kcal/mol, approaching physics-based performance but remaining less accurate.
4. Conclusion
Chemception predicts physiological, biochemical, and physical molecular properties from 2D molecular images without explicit chemistry features, using modest datasets. Across three tasks, its accuracy matches engineered-feature MLP networks, outperforming them on two properties while trailing slightly on toxicity.
- Chemception uses only 2D molecular drawings and no explicit chemistry knowledge, descriptors, or fingerprints as inputs.The architecture is presented as a general-purpose neural network for learning multiple molecular properties.
- It predicts toxicity, activity, and free energy of solvation across datasets containing approximately 600 to 40,000 compounds.These tasks cover physiological, biochemical, and physical properties.
- Across three tasks, Chemception matches MLP deep neural networks trained on engineered features such as ECFP fingerprints.The conclusion reports that Chemception outperforms its deep neural network counterparts on two of the three properties.
- 0.744/0.752 validation/test AUC was achieved for HIV activity prediction.The reported values are validation/test AUCs for the HIV activity task.
- 1.51/1.74 kcal/mol validation/test RMSE was achieved for free energy of solvation, close to physics-based simulation accuracy of approximately 1.5 kcal/mol.This comparison concerns free energy of solvation prediction.
- For Tox21 toxicity, Chemception's validation/test AUC of 0.768/0.773 trails an ECFP-fingerprint MLP's 0.777/0.799, possibly reflecting multi-task learning benefits.The stated explanation is qualified as possible rather than established.
- The domain-agnostic design minimizes chemistry knowledge and is anticipated to transfer to problems with limited understanding and feature engineering.The authors also emphasize deep neural networks' higher processing speed and consistency than humans.