Source-linked AI summary
Automatic chemical design using a data-driven continuous representation of molecules
Rafael Gómez-Bombarelli, Jennifer N. Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez-Lengeling, Dennis Sheberla, Jorge Aguilera-Iparraguirre, Timothy D. Hirzel, Ryan P. Adams, Alán Aspuru-Guzik
TL;DR
Discovering and testing compounds is costly, motivating automated exploration of chemical space. The paper introduces a learned continuous molecular representation with property prediction and shows it can generate new molecules and improve optimization outcomes.
Problem
Discovering and testing new compounds is costly and time consuming, while data-driven representations can exploit large chemical-compound datasets.
Method
A learned latent representation couples molecular generation with property prediction and supports latent-space molecule optimization.
Results
The model generates molecules that are new relative to the training-data distribution and consistently produces molecules with higher percentile scores through latent-space optimization.
Takeaways & Limitations
Continuous latent representations support automated generation and optimization of novel chemical structures within the demonstrated molecular domains.
Takeaways & Limitations
Euclidean distance in the latent space may not directly correspond to chemically meaningful notions of molecular similarity, and weaker predictive power can limit optimization when training data are smaller.
Abstract
from arXiv · showhide
We report a method to convert discrete representations of molecules to and from a multidimensional continuous representation. This model allows us to generate new molecules for efficient exploration and optimization through open-ended spaces of chemical compounds. A deep neural network was trained on hundreds of thousands of existing chemical structures to construct three coupled functions: an encoder, a decoder and a predictor. The encoder converts the discrete representation of a molecule into a real-valued continuous vector, and the decoder converts these continuous vectors back to discrete molecular representations. The predictor estimates chemical properties from the latent continuous vector representation of the molecule. Continuous representations allow us to automatically generate novel chemical structures by performing simple operations in the latent space, such as decoding random vectors, perturbing known chemical structures, or interpolating between molecules. Continuous representations also allow the use of powerful gradient-based optimization to efficiently guide the search for optimized functional compounds. We demonstrate our method in the domain of drug-like molecules and also in the set of molecules with fewer that nine heavy atoms.
Introduction
Chemical design is constrained by the enormous, costly search over chemical space and by discrete methods that lack scalable gradient guidance. The paper introduces a continuous, data-driven molecular representation that supports automatic generation and gradient-based optimization of candidate compounds.
- Motivation: Only about 10^8 substances have been synthesized, versus an estimated 10^23–10^60 potential drug-like molecules.This disparity makes exhaustive experimental exploration infeasible.
- Limitations of existing methods: Existing approaches rely on fixed-library search or discrete methods such as genetic algorithms, which require costly exploration, hand-crafted rules, and cannot guide search with gradients.Fixed libraries are monolithic, while genetic generation requires manually specified mutation and crossover heuristics.
- Proposed approach: The proposed representation encodes molecules as continuous vectors that can be modified and decoded into new compounds, enabling larger gradient-guided steps through chemical space.A differentiable property model maps latent representations to desirable properties for optimization.
- Model: A deep autoencoder converts SMILES strings into fixed-dimensional vectors and reconstructs them, while an added multilayer perceptron predicts properties from the latent representation.The autoencoder and property predictor are trained jointly using reconstruction and regression errors.
Results and discussion
The VAE produces a continuous latent space that captures molecular structure, supports smooth interpolation, and generates molecules matching training-set property distributions while remaining novel. Joint property prediction organizes latent representations by molecular properties and enables Gaussian-process optimization toward higher objective values, with predictive limitations affecting search behavior.
- Representation of molecules in latent space: Encoded molecules follow approximately normal distributions in each latent dimension, as enforced by the variational regularizer.This behavior is observed for 5,000 randomly selected ZINC molecules outside the training set.
- Representation of molecules in latent space: The most frequent stochastic decoding is typically closest to the original latent point, indicating that the representation captures molecule-relevant features.Decoding produces one prominent molecule alongside lower-frequency structural variations.
- Representation of molecules in latent space: As latent distance from ibuprofen increases, generated structures become less similar and eventually resemble random training-set molecules.Spherical interpolation between random drug molecules produces smooth transitions while avoiding low-probability regions associated with linear interpolation.
- Molecule generation: The VAE generates realistic molecules whose chemical-property distributions follow the training data, while drug-like outputs are generally new and hand-designed mutations are less novel and less drug-like.For QM9, smaller combinatorial coverage leads generated molecules to reproduce essentially the training population statistics.
- Property prediction of molecules: Joint property prediction organizes latent molecules along property gradients, whereas models without property prediction show no discernible property-related pattern.For some electronic properties, prediction performance is similar to graph convolutions, though further hyperparameter tuning could improve accuracy.
- Optimization of molecules via properties: Gaussian-process optimization in latent space consistently yields higher-percentile values for the objective 5× QED −SAS than two baseline search methods.With smaller training sets, weaker GP predictive power can produce several local minima rather than a global optimum; localized optimization may increase molecular diversity.
Conclusion
The proposed continuous molecular representations enable library-free exploration and directed gradient-based search, with high reconstruction fidelity, predictive power, and model-based optimization. Key limitations include invalid or undesirable generated structures, motivating graph-based, grammar-based, and actively learned extensions.
- Contributions: Continuous molecular representations eliminate hand-built compound libraries and enable directed gradient-based search through chemical space.The approach supports a new family of methods for exploring chemical space.
- Results: The method achieved high reconstruction fidelity, captured characteristic training-set features, showed good predictive power, and supported model-based molecular optimization.Predictive performance arose from jointly training the autoencoder and predictor.
- Limitations and extensions: The text-based encoding makes learning unnecessarily difficult because the decoder must learn which SMILES strings are valid.A graph-based autoencoder could directly address graph isomorphism and strings that do not correspond to valid molecular graphs.
- Limitations and extensions: Generating arbitrary molecular graphs with a neural network remains an open problem, while explicit SMILES grammars, active validity learning, and adversarial networks are under development.These extensions aim to avoid forcing the model to learn the SMILES grammar implicitly.
- Limitations and extensions: The autoencoder sometimes generated formally valid graphs containing moieties undesirable for stability or synthesis, so objectives should encode structural constraints and balance desirable traits.Examples include acid chlorides, anhydrides, aziridines, enamines, hemiaminals, and related unstable or synthetically constrained groups.
- Implications: Carefully composed objective functions give the reported results potential to create new avenues for molecular design.The objective should capture and balance as many desirable traits as possible so optimization focuses on genuinely desirable compounds.
Methods
The model used convolutional encoders and recurrent decoders to map SMILES strings into continuous representations, with stochastic character-level decoding. Teacher forcing improved generated-string validity but complicated training, while separate fully connected networks predicted dataset-specific molecular properties from latent vectors.
- Autoencoder architecture: Convolutional networks encoded SMILES strings, with improved performance attributed to repetitive, translationally invariant substrings corresponding to chemical substructures.The cited substructures include cycles and functional groups.
- Autoencoder architecture: The model trained on canonicalized SMILES, using 35 characters and maximum length 120 for ZINC, versus 22 characters and length 34 for QM9.Shorter strings were padded to the dataset-specific maximum length.
- Autoencoder architecture: The decoder sampled characters from position-wise probability distributions, so the same latent point could produce different SMILES strings under different random seeds.The output GRU received the character sampled at the previous time step and was trained with teacher forcing.
- Autoencoder architecture: Teacher forcing increased generated-SMILES accuracy and validity outside the training data, but made training more difficult because the decoder could ignore the variational encoding.The variational loss was annealed after 29 epochs during 120 total epochs.
- Property prediction: Two fully connected layers of 1000 neurons predicted properties from latent representations, including logP, QED, SAS for ZINC and HOMO, LUMO, and R2 for QM9.The predictor used dropout rate 0.2, and property-prediction loss was annealed alongside variational loss.
Supplementary Materials
The supplementary materials document decoding performance across training and validation molecules, latent-space analyses, sampling and interpolation behavior, optimization results, and planned release of code and training data.
- Implementation: The authors state that the code and full training data sets will be made available through the chemical_vae GitHub repository.The repository path is aspuru-guzik-group/chemical_vae.
- Decoding performance: Both VAEs decode training molecules successfully and transfer well to validation molecules from the same molecular class, although the QM9 VAE underperforms against random validation molecules.Table 3 reports decoding percentages after 1000 attempts for 1000 training molecules and 1000-molecule validation sets from ZINC and eMolecules.
- Latent-space analysis: Latent-space analyses characterize sampled-coordinate distributions, pairwise molecular distances, and the effects of jointly predicting molecular properties.Figures 2, 3, and 5 examine sampling behavior, PCA structure, coordinate statistics, and distances in the latent space.
- Optimization and generation: Supplementary figures compare optimization strategies, property distributions generated by the VAE and genetic algorithms, and linear versus spherical interpolation between FDA-approved drugs.The optimization objective shown is 5× QED −SAS, while property comparisons cover LogP, SAS, and QED.