PaperlayerView exact revision on arXiv

Source-linked AI summary

WEECFP-SuRGE: Wide Embedded Extended Connectivity Fingerprint with Substructure Rotary Graph-distance Encoding

Robert Epps

arXiv:2609.04672v1cs.LG

TL;DR

Molecular property prediction needs representations that retain substructure information while remaining effective without external pretraining. The paper introduces WEECFP and graph-distance rotary attention in WEECFP-SuRGE, whose 7-model blend ranks first among non-pretrained methods on TDC ADMET and achieves the lowest average regression rank. The authors also report near-lossless token reconstruction and scalable graph-distance encoding, while identifying dataset-size, descriptor, stereochemistry, and distance-model boundaries.

  • Problem

    Existing binary fingerprints have limited positional representation and can be difficult to use in larger models, while learned molecular representations often rely on large-scale pretraining.

  • Method

    The paper combines a parameter-free 1024-dimensional WEECFP fingerprint with a transformer whose SuRGE attention rotates substructure tokens according to shortest-path graph distance.

  • Results

    The 7-model WEECFP-SuRGE Blend is #1 overall among methods without external pretraining on TDC ADMET at average rank 5.1 and has the lowest regression rank at 3.9.

  • Takeaways & Limitations

    WEECFP-SuRGE provides a competitive from-scratch molecular-property approach across broad ADMET evaluation, with near-lossless token reconstruction and O(S) graph-distance positional memory.

  • Takeaways & Limitations

    The method inherits ECFP’s stereochemistry-insensitive Morgan enumeration unless stereochemistry is explicit, uses unit-weight bond distances, and truncates molecules beyond 300 tokens.

Abstract

from arXiv · show

We introduce WEECFP, a parameter-free 1024-dimensional continuous molecular fingerprint that scatters each Morgan substructure across roughly thirty-two signed positions of a single vector, and WEECFP-SuRGE, a transformer architecture whose self-attention applies SuRGE (Substructure Rotary Graph-distance Encoding) -- a RoPE-like rotation parameterized by molecular shortest-path graph distance -- to WEECFP substructure tokens. A 7-model blend of this architecture (the WEECFP-SuRGE Blend) achieves the lowest average regression rank on the TDC ADMET leaderboard; is #2 overall on the TDC ADMET leaderboard (behind only pretrained MapLight+GNN), and is #1 overall among methods that use no external pretraining; takes leaderboard #1 finishes on Pgp, Lipophilicity, CYP2D6 Substrate, Clearance Microsome, and LD50 (with the WEECFP-NoSuRGE Blend separately reaching #1 on HIA) across the full 22-benchmark suite -- without any external pretraining. On MoleculeNet, WEECFP-SuRGE beats every classical-fingerprint baseline on 3 of 4 regression tasks (ESOL, Lipophilicity, QM9). We further show that WEECFP tokenization is near-lossless: a greedy overlap reconstruction recovers the exact canonical SMILES of 99.9% of in-distribution molecules across 9 MoleculeNet datasets and 98.93% of molecules in a cross-dataset holdout (HIV->Lipophilicity), and that a three-reference farthest-first encoding of graph distance correlates at Pearson r = 0.901 with the true pairwise distance, enabling O(S) positional memory at matching accuracy.

1 INTRODUCTION

The paper introduces WEECFP, a parameter-free continuous fingerprint, and WEECFP-SuRGE, a transformer using graph-distance rotary encoding for molecular substructure tokens. It evaluates these methods broadly against classical and learned baselines.

  • WEECFP is a deterministic, parameter-free 1024-dimensional continuous fingerprint that writes each Morgan substructure to approximately 32 signed positions.Position and sign are correlated through the hash byte.
  • SuRGE applies a RoPE-like rotation based on shortest-path graph distance, with ref3 preserving Pearson r = 0.901 using O(S) memory.The architecture applies SuRGE either before the transformer or within every attention layer.
  • The resulting WEECFP-SuRGE system includes a 3-layer transformer and a 7-model blend for TDC evaluation.The evaluation covers 9 MoleculeNet tasks and all 22 TDC ADMET benchmarks.
  • The study compares the proposed models with full 12-fingerprint × 2-model baselines across the benchmark suite.

2 RELATED WORK

Related work spans hashed circular fingerprints, graph and transformer molecular representations, rotary position encoding, and standardized molecular-property benchmarks. WEECFP-SuRGE combines deterministic fingerprint tokens with graph-distance rotary encoding rather than learned structural embeddings or attention bias.

  • Circular and hashed fingerprints: Morgan and ECFP established circular substructure enumeration and hashed single-bit fingerprints, alongside standard alternatives such as MACCS, Avalon, and atom-pair fingerprints.The paper uses ECFP+Avalon+ErG with physicochemical descriptors as an augmented-feature control.
  • Graph neural networks and molecular transformers: Graph neural networks and molecular transformers commonly use self-supervised pretraining, while Graphormer and related models inject structural information through attention mechanisms.These approaches provide the learned-representation context for the proposed deterministic tokenization.
  • Rotary Position Encoding: RoPE is adapted from sequence position to scalar or vector-valued shortest-path graph distance in the proposed SuRGE encoding.
  • Benchmarks: MoleculeNet and the TDC ADMET Group provide standardized evaluation protocols spanning classical, pretrained, and fingerprint-combination methods.The paper reports pretrained and non-pretrained competitors separately.

3 METHOD

The method combines deterministic WEECFP substructure tokens with a transformer whose attention encodes molecular graph distance through SuRGE. It preserves substructure identity in a continuous fingerprint, represents topology with scalable positional encodings, and uses a fixed seven-model blend for evaluation.

  • WEECFP-S fingerprint: WEECFP is a deterministic, parameter-free 1024-dimensional continuous fingerprint that represents each Morgan substructure across roughly 32 signed positions.Per-substructure vectors are constructed before aggregation; atomic and bonded-neighbor information are written into additional blocks.
  • WEECFP-S fingerprint: Morgan enumeration identifies unique atom-radius substructures, which are hashed into position-and-sign writes before log-magnitude compression and summation.The same byte determines both target position and signed value, producing structured cancellation when writes collide.
  • WEECFP-S fingerprint: A greedy overlap reconstruction recovers exact canonical SMILES for 99.9% of in-distribution molecules and 98.93% of a HIV-to-Lipophilicity holdout.The holdout result covers 4,155 of 4,200 molecules, with all 45 failures attributable to stereochemistry only.
  • WEECFP-SuRGE: WEECFP-SuRGE tokenizes each substructure separately and applies a RoPE-like rotation whose phase depends on shortest-path graph distance rather than sequence position.A three-layer transformer attends over variable-length substructure sequences, using root-atom distances to encode topology.
  • WEECFP-SuRGE: The ref3 encoding uses three farthest-first reference atoms, preserves direction-specific distance information, and requires O(S) positional memory.Across 2,925 molecules it achieves Pearson r = 0.901 with true pairwise distance; at S = 1,000, it reduces memory by 42,667×.
  • Blend and training: The fixed TDC blend averages seven independently trained models spanning SuRGE transformers, an MLP, and dilated CNNs, with no external pretraining.The blend also includes tokenization and augmented-feature diversity while remaining fixed across all 22 TDC benchmarks.

4 EXPERIMENTS

The paper evaluates WEECFP-SuRGE across 9 MoleculeNet tasks and all 22 TDC ADMET benchmarks against classical fingerprints and public leaderboard methods. It combines benchmark comparisons with ablations of SuRGE, token scattering, position schemes, and model blending.

  • MoleculeNet: WEECFP-SuRGE beats every classical-fingerprint baseline on 3 of 4 MoleculeNet regression tasks: ESOL, Lipophilicity, and QM9.On FreeSolv, XGB+MACCS remains best.
  • TDC ADMET: 3.9 average regression rank across 9 TDC ADMET benchmarks is the lowest among listed methods, ahead of pretrained MapLight+GNN at 5.3.The Blend ranks 2nd overall at 5.1 across 22 benchmarks, behind pretrained MapLight+GNN at 4.3.
  • TDC ADMET: 5.1 average rank across all 22 TDC benchmarks makes the WEECFP-SuRGE Blend #1 among methods without external pretraining.It is also #1 among non-pretrained methods on regression at average rank 3.9 and is top-3 on 6 of 9 regression benchmarks.
  • Ablations: 0.030 RMSE improvement on QM9 and 0.013 RMSE improvement on Lipophilicity show gains for the best SuRGE variants over the no-r2 control.The atom-only no-r0 control is best on ESOL, where the dataset has 1,128 molecules.
  • Ablations: WEECFP outperforms WEECFP-Scatter on 7–9 of 9 MoleculeNet tasks, while Blocks 2 and 3 provide modest but reliable additional gains.The comparison indicates that Block 1’s signed multi-location scatter is load-bearing.
  • TDC ADMET: 5 leaderboard #1 finishes span Pgp, Lipophilicity, CYP2D6 Substrate, Clearance Microsome, and LD50.The regression #1 results are Lipophilicity, Clearance Microsome, and LD50; Pgp and CYP2D6 Substrate are classification wins.
  • Ablations: Mean TDC regression gain from blending is 0.01–0.05 on metric scale versus any single member.Blend members cover transformer, CNN, and MLP backbones plus two tokenization variants.

5 DISCUSSION

The discussion attributes WEECFP-SuRGE’s performance to continuous multi-location representations and graph-distance structure, especially for regression. It also identifies weaker classification performance, dataset-size sensitivity, and representation-scope limitations.

  • Representation: Binary ECFP supplies sparse bit features suited to trees but lacks magnitude information for neural-network gradient updates.WEECFP instead uses continuous signed scatter across multiple positions.
  • Inductive bias: SuRGE injects molecular topology into attention and contributes most where properties have geometric or topological interpretations.The discussion names Lipophilicity, QM9, Solubility, and LD50 as examples.
  • Pretraining: A 3-layer transformer trained from scratch on a few thousand molecules per benchmark rivals pretrained state-of-the-art on regression.Whether WEECFP pretraining closes the classification gap remains open.
  • Failure cases: The Blend ranks 21 on TDC Bioavailability and records AUPRC 0.770 versus 0.859 for MapLight+GNN on CYP2C9 Inhibition.The discussion associates these weaknesses with label variance, missing domain descriptors, and training-from-scratch limitations.
  • Limitations: WEECFP is stereochemistry-insensitive unless stereochemistry is explicit in SMILES, while SuRGE currently uses unit-weight bond graph distances.The 300-token cap truncates fewer than 1% of TDC-training molecules but should be revisited for macrocycles.

6 CONCLUSION

The paper concludes that parameter-free WEECFP tokens and graph-distance SuRGE provide a strong from-scratch molecular modeling pipeline. Its Blend achieves leading TDC ADMET regression and non-pretrained performance while outperforming classical fingerprints on most evaluated MoleculeNet regression tasks.

  • TDC ADMET: 3.9 average regression rank across 9 TDC benchmarks is the lowest among listed methods, while the Blend ranks 2nd overall across 22 benchmarks.It is #1 overall among methods without external pretraining at average rank 5.1.
  • MoleculeNet: WEECFP-SuRGE beats every classical-fingerprint baseline on 3 of 4 MoleculeNet regression tasks.The winning tasks are ESOL, Lipophilicity, and QM9.
  • TDC ADMET: Five leaderboard #1 finishes occur across Pgp, Lipophilicity, CYP2D6 Substrate, Clearance Microsome, and LD50.The WEECFP-NoSuRGE Blend separately reaches #1 on HIA.
  • Method: WEECFP-SuRGE combines a parameter-free 1024-dimensional continuous fingerprint with graph-distance rotary encoding applied at the transformer input or attention layers.The Blend contains seven models.

SUPPORTING INFORMATION

The supporting information provides complete fingerprint and downstream-model baseline grids for MoleculeNet regression and classification, alongside WEECFP-family transformer comparisons.

  • Regression: The regression baseline grid evaluates each classical fingerprint with XGBoost across four MoleculeNet tasks.
  • Regression: The regression baseline grid also evaluates each classical fingerprint with an MLP and reports final grounding references.
  • Classification: The classification baseline grid evaluates each classical fingerprint with XGBoost across five MoleculeNet tasks.
  • Classification: A corresponding MLP grid reports classification ROC-AUC results for the same five MoleculeNet tasks.
  • Classification: The supporting information also reports WEECFP-SuRGE transformer variants on the MoleculeNet classification tasks.

B.3 TDC ADMET CLASSIFICATION PER-BENCHMARK DETAIL

This section details TDC ADMET classification benchmarks and places WEECFP-SuRGE results alongside baseline, transformer, and diagnostic comparisons.

  • Benchmark detail: The TDC ADMET classification table covers 13 tasks and reports blend scores, leaderboard ranks, and a leaky-oracle diagnostic.
  • Benchmark detail: The WEECFP-SuRGE Blend reaches leaderboard #1 on Pgp with a score of 0.945.
  • Benchmark detail: The WEECFP-SuRGE Blend reaches joint leaderboard #1 on CYP2D6 Substrate at 0.738, alongside the NoSuRGE Blend at 0.744.
  • Baseline comparison: For classification, positive delta means a transformer variant exceeds the strongest classical baseline, while the observed pattern is noisier than regression.
  • Baseline comparison: Tables S9 and S10 compare transformer variants with the strongest classical-fingerprint baseline using task-specific deltas.

D PER-CONFIGURATION NOTES ON ALL WEECFP TRANSFORMER VARIANTS

The configuration notes distinguish six WEECFP transformer variants by graph-distance encoding, tokenization, computational cost, and task-specific performance.

  • SuRGE variants: WEECFP-SuRGE (Input, md) is strongest on ESOL at 0.861 and FreeSolv at 1.852 while applying mean-distance rotation only at input.
  • SuRGE variants: WEECFP-SuRGE (Input, ref3) adds directional information at input and contributes ensemble diversity, but is rarely the single best MoleculeNet regression variant.
  • SuRGE variants: WEECFP-SuRGE (md) is best on Lipophilicity at 0.706 among tested MoleculeNet regression methods and applies SuRGE at every layer.
  • SuRGE variants: WEECFP-SuRGE (ref3) is best on QM9 at 0.842 and is the best transformer on ClinTox, but is weaker than (md) on Lipophilicity.
  • Controls: The no-r2 control trails the best SuRGE variant by 0.01–0.03 RMSE on Lipophilicity and QM9, isolating the contribution of graph-distance rotation.
  • Controls: The atom-only no-r0 control is best on ESOL at 0.793 but performs substantially worse than SuRGE variants on Lipophilicity and QM9.

E.1 WEECFP-SURGE BLEND MEMBER ARCHITECTURES

The blend combines diverse WEECFP and Morgan architectures, augmented descriptors, and non-SuRGE controls, while reconstruction experiments assess whether signed-scatter tokens preserve molecular identity.

  • Blend composition: The seven-member blend uses four SuRGE transformers, one MLP, and two dilated CNNs without SuRGE.
  • Blend composition: Transformer members include WEECFP and Morgan tokens, with optional molecule-level Avalon+ErG augmentation.
  • Transformer members: The WEECFP-SuRGE Input-md transformer applies mean-distance rotation once at input to 1024-dimensional substructure tokens.
  • Non-transformer members: The MLP uses a flat WEECFP+Avalon+ErG vector, while CNN members use dilated convolutions without SuRGE.
  • Ablation: Removing SuRGE from the four transformer members changes average regression rank from 3.9 to 4.1, isolating the strategy’s reported advantage.
  • Reconstruction: Greedy overlap reconstruction using WEECFP-Scatter recovers exact canonical SMILES for 99.9% of in-distribution molecules across nine MoleculeNet datasets.

F.3 CROSS-DATASET HOLDOUT: HIV →LIPOPHILICITY

The HIV→Lipophilicity holdout tests whether WEECFP reconstruction depends on library overlap. WEECFP-Scatter retains near-lossless recovery despite the chemistry shift, while ref3 preserves strong graph-distance correlation across molecule sizes.

  • 4,200 Lipophilicity molecules were reconstructed from a library built on 41,127 HIV molecules, with folded 2048-d Morgan-bit tokens as a matched-parameter control.
  • 98.93% of molecules were recovered exactly in the cross-dataset holdout, and all 45 failures differed only in stereochemical annotation.
  • WEECFP-Scatter matched the Morgan-bit control in exact-match rate and exceeded it in cosine similarity at one-eighth the bit budget, 256 versus 2048.
  • The distance study evaluated mean-distance, 2-ref RSS, and 3-ref RSS against full shortest-path matrices across 2,925 molecules.
  • Pearson correlation for reduced distance embeddings rose from r = 0.816 at 3–10 atoms to r = 0.942 at 31–50 atoms, remaining r = 0.904 at 51–200 atoms.
  • The ref3 estimate achieved Pearson r = 0.951 and MAE = 1.14 bonds across 109,543 unordered pairs, with residuals slightly negative at larger distances.

G.2 MEMORY SCALING: PAIRWISE SURGE VS. REF3 SURGE

Pairwise SuRGE requires quadratic positional state because each key rotation depends on the querying token, whereas ref3 SuRGE restores factorization with three reference distances and O(S) memory. The resulting memory ratio grows linearly with sequence length.

  • Pairwise SuRGE materializes a positional tensor of shape B × H × S × S × dhead/2, yielding O(S2) memory per layer.At B = 32, H = 8, dhead = 32, and S = 100, this requires approximately 78 MB per layer; at S = 1,000, approximately 7.8 GB.
  • Ref3 SuRGE assigns each token three distances to farthest-first reference atoms and maps attention dimension pairs to references round-robin.
  • The ref3 factorization works because attention depends on reference-indexed position differences without storing a per-pair tensor.
  • Ref3 positional state has memory Mref3 = B · H · S · dhead/2 + B · S · 3 = O(S) per layer.
  • The pairwise-to-ref3 memory ratio grows linearly in S, with empirical deviations caused by the additive B · S · 3 raw-position term.
  • SuRGE is an orthogonal linear transform for a fixed rotation schedule and known bit-to-atom mapping, allowing analytical inversion of the encoded vector.
Loading 2609.04672v1…