GraphEBM: Molecular Graph Generation with Energy-Based Models

• Molecular graph generation is a promising alternative to chemical-space search for finding molecules with desirable properties, but the underlying graphs are discrete, irregular, and permutation invariant to node order.Source: §1 · p. 1
• Many existing molecular graph generators fail to preserve permutation invariance, potentially assigning different likelihoods to permutations of the same graph.Source: §1 · p. 1
• Energy-based models define an unnormalized data distribution in which points near the data manifold receive lower energies; the normalization constant is usually intractable.Source: §1 · p. 1
• GraphEBM parameterizes its energy function to preserve permutation invariance in molecular graph generation.Source: §1 · pp. 1–2
• Experiments on random, goal-directed, and compositional generation tasks show that GraphEBM is effective.Source: §1 · p. 2
• Langevin dynamics trains the energy function by approximately maximizing likelihood and generates samples from the trained model.Source: §1 · p. 2
• For goal-directed generation, GraphEBM pushes molecule energies down by flexible amounts based on property values, pushing harder for higher desirable-property values.Source: §1 · p. 2
• GraphEBM explores compositional generation toward multiple molecular objectives.Source: §1 · p. 2

2.1 PARAMETERIZED ENERGY FUNCTION

• The energy network encodes padded atom-and-bond graphs, applies edge-type-specific message passing with summed aggregation, pools node states, and maps the graph representation to a scalar energy.Source: §2.1 · pp. 2–3, §2 · p. 2

2.2 PERMUTATION INVARIANCE

• Permutation-equivariant graph layers and an invariant readout make both the energy and likelihood unchanged by node-order permutations.Source: §2.2 · p. 3

2.3 TRAINING

• Langevin-refined hallucinated samples approximate model draws; training lowers real-graph energies and raises hallucinated-sample energies, while positive-sample dequantization is optional.Source: §2.3 · pp. 3–4

2.4 GENERATION

• Generation refines initial configurations toward low energies, then symmetrizes adjacency, discretizes atom and bond types, and corrects valency.Source: §2.4 · p. 5

2.5 GOAL-DIRECTED GENERATION

• Goal-directed training weights energy lowering by property value, pushing down energies harder for molecules with higher normalized property values.Source: §2.5 · p. 5

2.6 COMPOSITIONAL GENERATION

• Compositional generation sums energy functions trained toward separate property goals, then generates molecules using the combined energy.Source: §2.6 · p. 5

3 EXPERIMENTS

• On QM9 and ZINC250k, GraphEBM performs competitively with existing methods in random generation and produces non-trivial molecules.The random-generation results are reported in Tables 4 and 5; generated samples are shown in Figure 6.Source: §3 · p. 6
• In a visualized example, Langevin dynamics refines a randomly initialized sample toward a data point corresponding to a realistic molecule.Source: §3 · p. 6
• Goal-directed generation produces more molecules with high property values than random generation, while Figure 3 compares QED and plogp distributions across both goal-directed and compositional generation.The compositional-generation distributions are also compared with random generation in Figure 3(c).Source: §3 · p. 6, Figure 3 · p. 6
• GraphEBM finds more novel molecules with the best QED score (0.948) than baselines.Property optimization results are reported in Table 1.Source: §3 · p. 6, Table 1 · p. 6
• Examples of constrained optimization include an 18.03 plogp improvement after replacing carbon atoms with sulfur, and changes to long cycles that improve synthetic accessibility.Figure 4 marks the molecular modifications and reports similarity scores and improvements around the arrows.Source: §3 · p. 7, Figure 4 · p. 7
• GraphEBM obtains higher property improvements than JT-VAE, GCPN, and MoFlow in constrained property optimization, but its success rate is weaker than reinforcement-learning methods and comparable to JT-VAE.The reported comparison is in Table 2.Source: §3 · p. 6, Table 2 · p. 7

4 CONCLUSION AND OUTLOOK

• GraphEBM is presented as the first energy-based model for molecular graph generation that preserves permutation invariance.Source: §4 · p. 7
• The paper proposes flexibly lowering energies for goal-directed generation and explores compositional generation toward multiple objectives.Source: §4 · p. 7
• The authors report realistic generated molecules and describe the goal-directed and compositional generation proposals as effective and promising.Source: §4 · p. 7

A.1 MOLECULAR GRAPH GENERATION

• Molecular graph generators use VAEs, GANs, flow models, or RNNs, and generate graphs either sequentially or in one shot.Sequential methods add nodes and edges one by one, while one-shot methods generate them all at once.Source: §A.1 · p. 13
• Most reviewed molecular graph methods fail to guarantee permutation invariance, so different node orderings of the same graph may receive different likelihoods.Sequential methods choose a node order; several one-shot methods also depend on ordering or only approximate invariance.Source: §A.1 · p. 13
• GraphEBM develops energy-based modeling for molecular graph generation and is proposed for compositional generation toward multiple objectives.The authors report that the method can achieve permutation invariance.Source: §A.1 · p. 13

A.2 ENERGY-BASED MODELS

• EBMs assign configurations scalar energies; training lowers energies for observed configurations relative to unobserved ones, while inference seeks low-energy configurations.The energy function defines an unnormalized probability distribution over the data.Source: §A.2 · p. 14
• EBMs have been used as generative models for images, videos, 3D objects, and point sets.Source: §A.2 · p. 14
• Prior graph EBMs and related score-based models generated graph structures but not attributed graphs; GraphEBM is presented as the first EBM capable of generating attributed graphs.Source: §A.2 · p. 14

B GENERATION PROCESS OF GRAPHEBM

• Figure 5 illustrates the generation process of GraphEBM.Source: B GENERATION PROCESS OF GRAPHEBM · p. 14

C EXISTING GOAL-DIRECTED GENERATION STRATEGIES

• Prior goal-directed generation strategies include conditioning on the desired property, predicting properties from latent representations, and reinforcement learning.Source: C EXISTING GOAL-DIRECTED GENERATION… · p. 14
• These strategies are not straightforward to apply to GraphEBM, which generates molecules implicitly through Langevin dynamics and has no latent space.Source: C EXISTING GOAL-DIRECTED GENERATION… · p. 14
• Applying EBMs to generation conditioned on continuous properties is rarely studied, making goal-directed EBM generation challenging.Source: C EXISTING GOAL-DIRECTED GENERATION… · p. 14

Dataset

• QM9 contains 134k molecules with at most 9 atoms, while ZINC250k contains 250k drug-like molecules with at most 38 atoms.QM9 has 4 atom types and 3 bond types; ZINC250k has 9 atom types and 3 edge types.Source: D EXPERIMENTAL SETUP · p. 15

Implementation details

• Training uses a three-layer, 64-dimensional energy network, up to 20 epochs, spectral normalization, and clipped Langevin gradients.The implementation also uses Swish, α = 1, Gaussian noise with σ = 0.005, and tunes the noise scale, Langevin steps, and step size.Source: D EXPERIMENTAL SETUP · p. 15

Random generation

• Random-generation evaluation measures validity, uniqueness, and novelty over 10,000 generated molecules, averaging results across five runs.Validity is the fraction chemically valid; uniqueness is the fraction unique among valid molecules; novelty is the fraction of valid generated molecules absent from training.Source: D EXPERIMENTAL SETUP · p. 15

Goal-directed generation

• Goal-directed generation on ZINC250k compares generated and random molecules' QED and penalized logP score distributions.Penalized logP accounts for long cycles and synthetic accessibility.Source: D EXPERIMENTAL SETUP · p. 15

Molecule optimization

• Molecule optimization evaluates novel high-QED generation and constrained improvement under a Tanimoto Morgan-fingerprint similarity threshold.Baselines use different initializations: JT-VAE and GCPN select low-penalized-logP test molecules, while GraphAF and MoFlow select training molecules; both settings are compared.Source: D EXPERIMENTAL SETUP · p. 15

Compositional generation

• Compositional generation combines the two energy functions from the goal-directed experiments and generates molecules using the resulting energy.Source: D EXPERIMENTAL SETUP · p. 15

Generation performance

• Generation performance on QM9 and ZINC250k is reported in Tables 4 and 5, respectively.Source: E EXPERIMENTAL RESULTS · p. 15, Table 4 · p. 16, Table 5 · p. 16

Generated molecules

• Figure 6 visualizes molecules generated by GraphEBM.Source: E EXPERIMENTAL RESULTS · p. 15, Figure 6 · p. 16

Implicit generation process

• Figure 7 depicts the implicit generation process, with atom matrices in the first row and four bond-channel adjacency tensors in the remaining rows.The atom matrices and adjacency tensors are normalized by their maximum values for visualization.Source: E EXPERIMENTAL RESULTS · p. 15, Figure 7 · p. 16