Source-linked AI summary
Toxicity Prediction using Deep Learning
Thomas Unterthiner, Andreas Mayr, Günter Klambauer, Sepp Hochreiter
TL;DR
Toxicity testing cannot feasibly cover all chemicals because biological experiments are costly and limited, creating a need for scalable computational prediction. The paper applies deep neural networks to toxicity prediction using chemical descriptors and multitask learning, finding that the approach learned toxicophore-like representations and won the Tox21 panel and overall challenges.
Problem
Biological toxicity testing is too costly and limited to assess the toxicity of all chemicals, motivating scalable computational methods.
Method
The paper applies deep neural networks that predict multiple toxicity types from chemical descriptors in a multitask setting.
Results
The approach won 8 of 15 challenges, achieved the best average AUC in both panels and overall, and won the Nuclear Receptor, Stress Response, and overall Tox21 challenges.
Takeaways & Limitations
Deep networks learned representations containing known toxicophore structures and outperformed methods tailored for toxicological applications.
Abstract
from arXiv · showhide
Everyday we are exposed to various chemicals via food additives, cleaning and cosmetic products and medicines -- and some of them might be toxic. However testing the toxicity of all existing compounds by biological experiments is neither financially nor logistically feasible. Therefore the government agencies NIH, EPA and FDA launched the Tox21 Data Challenge within the "Toxicology in the 21st Century" (Tox21) initiative. The goal of this challenge was to assess the performance of computational methods in predicting the toxicity of chemical compounds. State of the art toxicity prediction methods build upon specifically-designed chemical descriptors developed over decades. Though Deep Learning is new to the field and was never applied to toxicity prediction before, it clearly outperformed all other participating methods. In this application paper we show that deep nets automatically learn features resembling well-established toxicophores. In total, our Deep Learning approach won both of the panel-challenges (nuclear receptors and stress response) as well as the overall Grand Challenge, and thereby sets a new standard in tox prediction.
1. Introduction
Toxicity testing is important but difficult to scale because biological assays are costly and limited, motivating computational prediction. Deep learning is well suited to this setting because it can learn hierarchical chemical features and share representations across multiple toxicity tasks.
- More than 30% of drug candidates fail in clinical trials because of undetected toxic effects.
- Tox21 seeks better toxicity assessment methods because existing approaches may not scale with growing testing demand.
- Tox21 tested only a few thousand compounds for as few as twelve toxic effects despite a multi-million-dollar effort.
- Deep learning architectures automatically construct complex features and support multi-task learning, matching common toxicity-prediction requirements.
- Toxicological reasoning uses hierarchical chemical abstractions in which structural properties combine into toxicophores associated with toxicological effects.
- A shared multitask setting can use relations among tasks and shared hidden representations, especially when some tasks have few measurements.
2. Methods
The method uses deep neural networks to predict multiple binary toxicity outcomes from chemical descriptors, while integrating heterogeneous features and handling missing task labels. Its inputs include ECFP4 fingerprints, toxicophore and scaffold similarities, molecular descriptors, and regularization-oriented training choices.
- DNN Architecture: The system predicts several binary toxicity effects simultaneously from a numerical compound descriptor.
- DNN Architecture: Training uses a weighted sum of cross-entropies across tasks, with missing labels masked during backpropagation.
- DNN Architecture: The network combines ReLU hidden layers with sigmoid output units, using one output unit for each classification task.
- Input Features: Feature combinations, normalization, sparsity thresholds, network dimensions, learning rates, dropout, and L2 weight decay were treated as hyperparameters.
- Input Features: ECFP4 fingerprints encode approximately 30,000 sparse chemical-substructure features, with sparsity filtering used to remove noninformative inputs.
- Input Features: Additional inputs include similarities to 2,500 known toxicophores, similarities to 200 common substructures, and 5,057 topological and physical descriptors.
- The implementation handled up to 40,000 input features using sparse storage, mini-batches of 512 samples, and a single GPU with 12 GB of RAM.
3. Experimental Results
The study evaluates deep multi-task networks on the 12-task Tox21 benchmark after chemical-data cleaning and task-aware validation. Multi-task learning generally outperformed single-task networks, while hidden units learned representations correlated with known toxicophores.
- Tox21 Data Challenge Data: The Tox21 benchmark contained approximately 12,000 compounds across 12 toxicity-prediction tasks spanning Nuclear Receptor and Stress Response panels.Most compounds were measured on multiple tasks, enabling task-correlation analysis.
- Data preprocessing: The training data were cleaned by identifying redundant or combined structures, yielding 8,695 distinct compound fragments.Chemical standardization also made hydrogens explicit, unified aromatic-bond and tautomer encoding, and normalized salts.
- Data preprocessing: Cross-validation included compounds labeled on at least eight tasks so validation more closely matched the expected fully labeled test set.Hyperparameters were selected using AUC independently for each task.
- Multitask Learning: Multi-task networks significantly outperformed single-task networks on almost all tasks.Both approaches failed on one task with only three positive leaderboard examples.
- Learning Toxicophore Representation: Using only ECFP4 fingerprints, hidden-unit activations showed highly significant correlations with known toxicophore features.Lower layers tended to represent smaller functional groups, whereas higher layers correlated with larger toxicophore clusters.
4. Results
The paper compares its deep-network approach with leading Tox21 Challenge teams using a held-back evaluation set and AUC-based scoring. The approach achieved consistently high rankings and won the overall challenge as well as both panel challenges.
- Challenge evaluation: The final evaluation used a held-back set of 643 compounds, with each sub-challenge scored by the AUC of final submissions.Participants received no performance feedback on these final compounds.
- Challenge results: The paper’s network never ranked below fifth in any task and won 8 of the 15 challenges.Table 3 reports detailed comparisons with leading teams.
- Challenge results: The approach achieved the best average AUC in both the Nuclear Receptor and Stress Response panels and across all sub-challenges.It was declared the winner of both panel challenges and the overall Tox21 Grand Challenge.
5. Conclusion
Deep neural networks learned effective chemical representations for toxicity prediction, including recognizable toxicophores. Using these representations, the approach outperformed toxicology-specific methods and set a new state of the art.
- Deep neural networks learned highly effective representations of chemical compounds for toxicity prediction.
- The learned representations captured toxicophores that experts had previously handcrafted over decades.
- The approach outperformed methods specifically tailored for toxicological applications.
- The method set a new state of the art in toxicity prediction in the Tox21 Data Challenge.
- The models were considered suitable for deployment in leading-edge toxicological research because of their high quality.