MolDiff: Addressing the Atom-Bond Inconsistency Problem in 3D Molecule Diffusion Generation

1. Introduction

• Post-processing bond assignment can create extra-large rings, violate valency constraints, and reject correct aromatic bonds after small positional errors.These issues arise when bonds are assigned from a canonical bond-length lookup table after atom types and positions are generated.Source: §1 · p. 1
• Bond lengths do not uniquely determine bond type: C–C single bonds vary with aromaticity, and N–O single and aromatic bonds can share a length.Post-processing bond prediction also made more errors on GEOM-Drug than QM9, which contains molecules with up to nine heavy atoms.Source: §1 · p. 1
• MolDiff jointly samples atoms and bonds with a diffusion model to address atom-bond inconsistency.Its design also perturbs atoms and bonds at different noise levels, uses bond-predictor guidance for atom generation, and updates atom and bond representations in an E(3)-equivariant network.Source: §1 · pp. 1–2
• The authors propose metrics covering drug-likeness, geometric properties, and molecular rationality to assess generated molecules.They argue that commonly used metrics inherited from 2D molecule generation do not fully measure generated-molecule quality.Source: §1 · p. 2

2. Related work

• The passage identifies 1D SMILES as the most common molecular representation and notes that many sequence-based generative models can be applied to it (Weininger, 1988; G´omez-Bombarelli et al., 2018; Kusner et al., 2017; Segler et al., 2018).Source: §2 · p. 2
• Non-autoregressive 3D models using density grids or point clouds and VAE, flow, or diffusion methods add bonds after determining atoms.The passage contrasts them with autoregressive models, which generate atoms sequentially and either add bonds afterward or predict them after each atom.Source: §2 · p. 2
• Diffusion models for molecule-related tasks have treated 3D molecules as point clouds, with bond information provided as context or added post hoc.The cited tasks include conformation generation, de novo design, and linker design.Source: §2 · p. 2
• MolDiff is presented as the first E(3)-equivariant diffusion model for high-quality 3D molecular graph generation that tackles atom-bond inconsistency.The paper notes a concurrent study on molecular graphs with 3D conformations, Vignac et al. (2023), which focused on distinct aspects.Source: §2 · p. 2

3.1.1. FORWARD PROCESS

• The forward process adds Gaussian noise to positions and categorical noise to atom and bond types, with separate noise schedules available for each.The categorical process gradually moves types toward absorbing types; bond types use none-type as their absorbing type.Source: §3.1.1 · p. 3

3.1.2. REVERSE PROCESS

• The reverse process trains neural networks to approximate the true posterior, then generates molecules by repeatedly denoising samples from the prior distributions.The prior uses Gaussian noise for positions and absorbing types for atom and bond categories.Source: §3.1.2 · pp. 3–4

3.2. Equivariant Graph Neural Networks

• The E(3)-equivariant network updates atom and edge representations on a complete graph, using node and edge features to predict atom and bond types.Bond predictions use both directed edge representations to preserve symmetry.Source: §3.2 · p. 4

3.3. The Atom and Bond Noise Schedule

• Bond types reach the absorbing none-type prior before atom types and positions are substantially noised, keeping their relationship more like real molecules during diffusion.The first stage lightly perturbs atoms while bonds diffuse; the second continues noising atoms after most real bonds are removed.Source: §3.3 · pp. 4–5

3.4. Guidance of Bond Predictor

• A separately trained bond predictor scores bonds from noisy atom types and positions, and its confidence gradient guides reverse updates to improve atom positions.The method links higher bond-prediction confidence to more accurate bond lengths and atom positions.Source: §3.4 · p. 5

4.1. Generation Ability

• MolDiff exceeded EDM and its variants in diversity and similarity to the validation set, while its novelty and uniqueness values were near one but not the best.These comparisons were among valid molecules.Source: §4.1 · p. 6
• MolDiff outperformed EDM on validity, connectivity, and success rate; excluding bonds during diffusion harmed performance, while discrete atom and bond types outperformed continuous embeddings.The comparison also found that adding a bond-length loss did not improve performance.Source: §4.1 · pp. 6–7, Table 1 · p. 7

4.2. Quality of 3D Molecules

• MolDiff outperformed EDM on drug-likeness measures, including QED, synthetic accessibility, and Lipinski-rule adherence.The authors interpret this as producing better drug candidates.Source: §4.2 · p. 7, Table 2 · p. 7
• MolDiff had lower RMSDs and lower bond-angle and dihedral-angle JS divergence than EDM, but worse bond-length JS divergence; overall, its generated 3D structures were closer to ground truth.EDM’s lookup-table bond assignment kept bond lengths closer to canonical lengths.Source: §4.2 · p. 7, Table 2 · p. 7
• MolDiff had lower JS divergence than EDM across bond-count and bond-type distributions, indicating more realistic bond counts and more balanced bond-type ratios.The analysis covered counts per atom and distributions of basic and frequent bond types, pairs, and triplets.Source: §4.2 · pp. 7–8, Table 2 · p. 7

4.3. Analyses of the Noise Schedule and the Guidance

• Ablations showed that bond-predictor guidance and the new noise schedule improved 3D structures, bond counts, and ring distributions, but did not affect the balance of bond types.The authors also report that these strategies did not harm basic generation ability or drug-likeness.Source: Table 3 · p. 8, §4.3 · p. 8

4.2. Quality of 3D Molecules

• MolDiff produced better ring-count and ring-size distributions than EDM, and its frequent ring types were more realistic and more like the validation set.The figure compares the ten most frequent rings in the validation set with those generated by EDM and MolDiff.Source: Figure 3 · p. 7, Table 2 · p. 7, §4.2 · p. 8

4.4. Model Analysis

• The schedule produced fewer out-of-range bonds during diffusion, while confidence-guidance gradients pushed perturbed atom positions toward the right positions.The authors conclude that both mechanisms support more accurate 3D structures and more realistic bond and ring distributions.Source: Figure 4 · p. 8, §4.4 · pp. 8–9

5. Conclusion

• MolDiff simultaneously addresses the generation of molecular atoms and bonds.The paper reports that experiments show improved generation ability and generated-molecule quality.Source: §5 · p. 9
• Further work can explore more sophisticated noise schedules and more advanced guidance strategies.Source: §5 · p. 9
• The model’s potential for downstream real tasks remains an area for analysis.Source: §5 · p. 9

A.1. Hyper-parameters of the Neural Networks and Training

• Training uses 256-dimensional node and 64-dimensional edge embeddings, six graph-network layers, and AdamW for 110,000 iterations.The learning rate is 1 × 10^-4, batch size 256, and atom and bond loss weights are both 100.Source: §A.1 · p. 12

A.2. Details about the Diffusion Process

• The appendix derives a diffusion posterior from Eqs. 1 and 2 and defines element-wise vector multiplication.Source: §A.2 · p. 12

A.3. Details about the Model Architectures

• The networks process edge features and distances, pass vertex- and edge-based messages, and use concatenation and sigmoid operations.The appendix describes architectures φd, φv, φe, and φr; φd applies RBF kernels to edge lengths before combining them with edge features.Source: §A.3 · p. 12

A.4. Details about the Noise Schedule

• The sigmoid noise schedule uses s1 and sT to set its range, while w controls its medium-step slope and its change rate at the beginning and end.For bond types, the implementation uses a two-stage schedule: steps 1–600 use sT = 0.001 and w = 3; steps 600–1000 use s1 = 0.001, sT = 0.0001, and w = 2.Source: §A.4 · p. 13, Figure 5 · p. 13

A.5. Details about the Bond Predictor

• The bond predictor takes no position updates and receives endpoint atom types, rather than bond types, as initial edge features.It predicts step-0 bond types from noised positions, atom types, and the diffusion step, using cross-entropy training.Source: §A.5 · pp. 13–14

B. Preprocessing of the Geom-Drug Dataset

• Filtering GEOM-Drug and removing hydrogen atoms yielded 231,523 training, 28,941 validation, and 28,940 testing molecules.Kept molecules were RDKit-parseable and unbroken, had 8–60 heavy atoms, used only H, C, N, O, F, P, S, and Cl, and had single, double, triple, or aromatic bonds.Source: B. Preprocessing of the Geom-Drug… · p. 14

C.1. Details about the Experiment Settings and Baselines

• Experiments generated molecules until 1,000 valid and complete examples were obtained, repeated this three times, and reported averages in the main text.Standard deviations are reported in Appendix C.4.Source: §C.1 · p. 14
• Variants omit bond types during diffusion and add bonds afterward by lookup table, neural-network prediction, or Open Babel; other variants diffuse continuous type embeddings.A further variant adds an MSE bond-length loss; the variants do not use the proposed noise schedule or guidance.Source: §C.1 · p. 14

C.2. Details about the Metrics

• Structural analyses specify SMARTS patterns for selected bond types, bond pairs, and bond triplets.Source: §C.2 · p. 14

C.3. Details and More Results of the Ablation Study

• The ablation study found that the proposed bond-noise schedule and bond-predictor guidance did not damage basic generation abilities or drug-likeness.Table 4 reports additional ablation metrics.Source: §C.3 · p. 14, Table 4 · p. 15

C.4. Standard Deviations of the Values in the Main Text.

• Tables 5–7 provide standard deviations for generation abilities, molecule qualities, and ablation-study values, respectively.These deviations facilitate comparison of metric values reported in the main text and ablation study.Source: §C.4 · p. 15, Table 5 · p. 15, Table 6 · p. 15

C.5. More Examples of Generated Molecules

• The appendix presents several MolDiff-generated molecules as 3D conformations and their corresponding 2D graphs.Figures 7 and 8 show the conformations and molecular graphs, respectively.Source: §C.5 · p. 15

• Figures 7 and 8 provide examples of MolDiff-generated molecules and their molecular graphs.Source: Figure 7 · p. 16, Figure 8 · p. 16
• Table 7 reports standard deviations for values in the ablation study.Source: Table 7 · p. 17
• Three additional settings were evaluated: GEOM-Drug with hydrogens retained, GEOM-Drug with all elements, and QM9.The main-text GEOM-Drug evaluation used major elements and removed hydrogens.Source: D. Additional evaluations on more… · pp. 15, 17
• MolDiff sampling in these additional experiments did not use bond-predictor guidance.Source: D. Additional evaluations on more… · p. 17

D.1. Evaluation on Molecules with Hydrogens Reserved

• Hydrogens were omitted in the main evaluation because they can be inferred from heavy atoms and are often omitted in practical representations.Including hydrogens creates larger graphs and a more rigid benchmark.Source: §D.1 · p. 17
• With hydrogens retained, MolDiff’s success rate was 0.739, above EDM’s 0.371 without hydrogens, while generated-molecule quality was not obviously affected.The average molecule size rose from around 24 to around 45 atoms, and connectivity and success rate worsened.Source: §D.1 · p. 17, Table 8 · p. 17

D.2. Comparison between MolDiff and EDM on Molecules with All Elements

• The complete GEOM-Drug experiment restored minor elements previously excluded because they were uncommon, difficult to learn from sparse samples, and potentially challenging for biochemical experiments.Minor elements accounted for less than 0.2% of the dataset, including only 12 Si atoms.Source: §D.2 · p. 18
• MolDiff reached 0.860 success versus EDM’s 0.014 on complete GEOM-Drug, while retaining its quality advantage over EDM.Both models performed worse than on the filtered dataset.Source: §D.2 · p. 18, Table 9 · p. 18

• On QM9, MolDiff achieved the highest validity, connectivity, and success rate among the compared methods.Table 10 presents the QM9 comparison.Source: §D.3 · p. 18, Table 10 · p. 19
• MolDiff performed best on RMSD and dihedral-angle JS divergence, and second-best on bond-length and bond-angle JS divergence.Differences among methods for bond-length and bond-angle divergences were less significant.Source: §D.3 · p. 18, Table 10 · p. 19
• Novelty, diversity, uniqueness, and similarity to the validation set were similar across methods; MolDiff’s advantage was smaller on simpler, less drug-like QM9 than on GEOM-Drug.Source: §D.3 · pp. 18–19