PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-View Molecular Representation Learning with Hierarchical Graphs and Contextualized Fingerprints

Gwang-Hyeon Yun, Jong-Hoon Park, Bing Hu, Helen Chen, Anita Layton, Young-Rae Cho

arXiv:2609.15611v1cs.LGcs.AI

TL;DR

Molecular property prediction must generalize from scarce labels to structurally novel compounds, while existing methods provide limited fragment supervision or use fingerprints as fixed features. HiFi-Mol separately pretrains hierarchical graph and contextualized fingerprint encoders before integrating them for prediction. On MoleculeNet, the dual-view model achieves the best average performance across classification tasks, while maintaining competitive regression performance.

  • Problem

    Molecular property prediction requires representations that generalize from scarce labeled data to structurally novel compounds, but graph methods provide limited fragment-level supervision and fingerprints are typically fixed auxiliary features.

    Source: §1 · pp. 1–2

  • Method

    HiFi-Mol separately pretrains a hierarchical graph encoder with fragment-aware, multi-resolution supervision and a contextualized fingerprint encoder, then integrates their representations downstream.

    Source: §1 · p. 2, §4.1 · p. 3

  • Results

    The dual-view integration achieves the best average performance across classification tasks, with strong MoleculeNet classification results and competitive regression performance.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    Dataset-dependent strengths of the graph and fingerprint variants indicate that the two views provide complementary predictive signals.

    Source: §1 · p. 2, §5.2 · p. 6

    3D geometry is used only as pretraining supervision.

    Source: §6 · p. 8

Abstract

from arXiv · show

Molecular property prediction requires representations that generalize from limited labeled data to structurally novel compounds. Existing molecular pretraining methods often rely on a single view: graph-based approaches model atom-bond topology but provide limited fragment-level supervision, whereas fingerprint descriptors encode chemical patterns but are typically used as fixed auxiliary features. We propose HiFi-Mol, a multi-view framework that separately pretrains a hierarchical graph encoder and a contextualized fingerprint encoder before downstream integration. The graph branch uses fragment-aware masking with multi-resolution supervision to capture substructure-aware representations, while the fingerprint branch tokenizes active entries from seven fingerprint families and applies masked language modeling to learn contextualized embeddings. During fine-tuning, HiFi-Mol combines projected multi-resolution graph features with fingerprint embeddings for downstream prediction. Evaluated on MoleculeNet benchmarks under the scaffold split, HiFi-Mol achieves a 2.77% improvement in average ROC-AUC over the best baseline across eight classification tasks while maintaining competitive performance on three regression tasks. Further analyses reveal that fragment-aware masking improves graph representation quality, and classification results demonstrate dataset-dependent strengths of the individual graph and fingerprint variants, confirming that the two views provide complementary predictive signals.

1 Introduction

Molecular property prediction must learn from scarce labels and generalize to structurally novel compounds, while representing chemical patterns beyond atom-level topology. HiFi-Mol addresses gaps in graph substructure modeling and static fingerprint use with separately pretrained, integrated graph and fingerprint views.

1 Introduction

  • Limited labeled data and structurally novel test compounds make molecular representation learning challenging, requiring models to capture higher-order structures and functional patterns.These patterns are relevant to molecular activity.Source: §1 · p. 1
  • Atom-level graph methods have difficulty explicitly modeling meaningful substructures, while existing fragment methods lack integrated multi-resolution supervision or disconnect fragments from their atom-bond patterns.Relevant structures include functional groups, ring systems, and pharmacophoric scaffolds.Source: §1 · p. 1
  • Fingerprints encode complementary chemical patterns, but are typically static auxiliary vectors shallowly concatenated with graph embeddings rather than learnable contextualized views.Source: §1 · pp. 1–2
  • HiFi-Mol separately pretrains hierarchical graph and contextualized fingerprint encoders, then integrates their representations for downstream prediction.Graph pretraining uses fragment-aware masking and multi-resolution supervision; fingerprint pretraining applies masked language modeling to joint SMILES–fingerprint tokens.Source: §1 · p. 2
  • MoleculeNet results show strong classification and competitive regression, with dataset-dependent strengths suggesting complementary predictive signals from graph and fingerprint views.Source: §1 · p. 2

2 Related Work

Prior molecular pretraining uses generative, contrastive, and hybrid graph objectives, with geometric and fragment-level approaches extending graph representations. Fingerprints offer a complementary view, but are generally fused as static descriptors rather than contextualized representations.

2 Related Work

  • Graph pretraining includes generative reconstruction and contrastive agreement objectives, while hybrid methods combine these signals.Examples of generative methods include AttrMask and ContextPred [16], and contrastive methods include GraphCL, JOAOv2, and MolCLR.Source: §2 · p. 2
  • Figure 1 depicts a hierarchical graph with atom, fragment, and global nodes, and fragment-aware masking that propagates to atoms and incident bonds to target coherent substructures.Source: Figure 1 · p. 2
  • Geometric approaches align 2D graphs with 3D signals, but explicit supervision of chemically meaningful substructures remains relatively underexplored.Source: §2 · p. 2
  • Fragment-level methods model motifs or align molecular and fragment representations, yet often decouple fragment identities from the atom-bond patterns that define them.MGSSL predicts motif labels and topology; HiMol builds hierarchical graphs; GraphFP uses fragment-based contrastive learning.Source: §2 · p. 2
  • Molecular fingerprints encode predefined substructural patterns and can complement graph representations, but are typically static auxiliary descriptors fused shallowly.Fingerprint language modeling such as DELBERT tokenizes sparse vectors and uses masked language modeling to learn contextualized descriptor representations.Source: §2 · p. 2

3 Preliminaries

HiFi-Mol represents each molecule with a hierarchical graph and a joint SMILES–fingerprint token sequence. Its prediction function combines the graph and fingerprint encoders' outputs by feature concatenation.

3 Preliminaries

  • A hierarchical molecular graph contains atom, fragment, and global nodes, with chemical bonds and directed edges linking the hierarchy levels.Source: §3 · p. 3
  • The fingerprint representation tokenizes active entries across fingerprint families, encoding each entry's fingerprint type, bit index, and count value.Source: §3 · p. 3
  • The joint token sequence concatenates SMILES and fingerprint tokens and serves as input to the fingerprint encoder.Source: §3 · p. 3
  • For prediction, the graph and fingerprint encoders' outputs are concatenated as input to the task-specific prediction function.The graph encoder is ΦGNN, the fingerprint encoder is ΦFP, and their concatenation produces the predicted molecular property.Source: §3 · p. 3

4 Proposed Method

HiFi-Mol separately pretrains hierarchical graph and contextualized fingerprint encoders with view-specific objectives, then integrates their representations during downstream adaptation.

4.1 Hierarchical Graph Pretraining

  • Fragment-aware masking selects chemically coherent fragments and masks constituent atoms and incident bonds; five objectives supervise global, fragment, atom, bond, and distance signals.The masked nodes and bonds serve as reconstruction targets for the graph encoder.Source: §4.1 · p. 3, Figure 2 · p. 4

4.2 Contextualized Fingerprint Pretraining

  • The fingerprint encoder processes tokens from seven complementary fingerprint families and SMILES with masked language modeling to learn contextualized descriptor representations.During pretraining, 15% of joint tokens are randomly replaced by a mask token and reconstructed.Source: §4.2 · pp. 4–5

4.3 Multi-View Downstream Integration

  • HiFi-Mol keeps graph and fingerprint encoders separate during view-specific pretraining, preserving distinct structural and descriptor-level representations.The framework combines their representations during downstream adaptation.Source: §4.3 · p. 5
  • For prediction, atom-, fragment-, and global-level graph summaries and a pooled fingerprint representation are projected, concatenated, and passed to a task-specific head.Graph summaries use mean pooling for atom representations, max pooling for fragment representations, and the global representation directly.Source: §4.3 · p. 5

5 Experiments

In scaffold-split MoleculeNet experiments, HiFi-Mol leads the classification comparison on average and remains competitive on regression tasks. Ablations and retrieval analyses examine its pretraining components and the structural organization learned by its graph encoder.

5.2 Downstream Performance in MoleculeNet

  • 76.15 average ROC-AUC, a 2.77% relative improvement over MolCLR, gives HiFi-Mol the highest average across eight classification benchmarks.Source: Table 2 · p. 6, §5.2 · p. 6
  • HiFi-Mol is second-best on ESOL and FreeSolv, while HiFi-MolGraph is best on ESOL and Lipophilicity; multi-view gains are comparatively weaker on regression.Source: Table 3 · p. 6, §5.2 · p. 6
  • At least one HiFi-Mol variant reaches state-of-the-art on seven of eight classification benchmarks, with graph-only stronger on BBBP, Tox21, ToxCast, and BACE and fingerprint-only on HIV, MUV, ClinTox, and SIDER.Source: §5.2 · p. 6

5.3 Ablation Study

  • On eight classification benchmarks, replacing fragment-aware masking with random masking reduces average ROC-AUC by 0.91; removing any individual graph objective also degrades performance.Source: §5.3 · p. 6, Table 4 · p. 7
  • GIN is the strongest tested backbone, while all four GNN variants remain competitive across the representative small and large datasets.Source: Table 5 · p. 7, §5.3 · p. 7
  • Removing global contrastive consistency causes the largest average ROC-AUC decline (ΔAvg. = −1.23), followed by fragment identity prediction (−0.81), distance prediction (−0.70), and atom-bond recovery (−0.29).Source: §5.3 · p. 7

5.4 Retrieval Analysis of Pretrained Graph Representations

  • With fragment-aware masking, the pretrained graph encoder retrieves scaffold-similar neighbors; without it, retrieved molecules are less chemically coherent despite high embedding-space cosine similarity.Source: Figure 3 · p. 7, §5.4 · p. 7
  • Pretraining with FAM improves retrieval on Tox21 and ToxCast: Mean Tanimoto@5 rises +0.023 on both datasets, while Scaffold Match@5 improves +0.034 and +0.030, respectively.Source: §5.4 · p. 7

6 Conclusions and Future Work

HiFi-Mol separately pretrains hierarchical graph and contextualized fingerprint encoders, then integrates them for molecular property prediction. It achieves the best average performance across classification tasks, while dual-view integration does not always improve regression results.

6 Conclusions and Future Work

  • HiFi-Mol separately pretrains hierarchical graph and contextualized fingerprint encoders before integrating them during downstream adaptation.Fragment-aware masking with multi-resolution supervision targets substructure-aware graph representations, while masked language modeling over joint SMILES–fingerprint sequences learns contextual descriptor relationships.Source: §6 · p. 8
  • The dual-view integration achieves the best average performance across classification tasks, but does not always yield gains over single-view variants on regression tasks.The paper reports that future work will investigate scalable cross-view interaction mechanisms and extensions to equivariant 3D molecular modeling.Source: §6 · p. 8
  • 3D geometry is used only as pretraining supervision, not as a downstream input.The authors identify this as a scope boundary and propose equivariant 3D molecular modeling as a future extension.Source: §6 · p. 8

A Graph Initialization Details

Hierarchical graph initialization uses type-specific node features and edge features that encode both bond attributes and cross-resolution relations.

A Graph Initialization Details

  • Atom nodes sum embeddings for ten chemical attributes, with atomic number 119 reserved as a mask token.The attributes are atomic number, chirality, formal charge, hybridization, hydrogen count, implicit valence, degree, aromaticity, ring membership, and radical-electron count.Source: A Graph Initialization Details · p. 8, Table 6 · p. 8
  • Fragment nodes use fragment dictionary identities, global nodes average initial atom feature embeddings, and masked atom or fragment nodes receive a learnable mask embedding.A learnable type embedding distinguishes atom, fragment, and global nodes.Source: A Graph Initialization Details · p. 8
  • Each edge sums four feature embeddings for bond type, direction, stereo configuration, and conjugation, with bond types extended to encode hierarchical relations.The listed directed relations connect atoms to fragments, fragments to atoms or other fragments, and fragments to the global node.Source: A Graph Initialization Details · p. 8

GenAI Usage Disclosure

The disclosure limits LLM use to grammar checking and language polishing; the authors state that the scientific work is theirs.

GenAI Usage Disclosure

  • LLM use was limited to grammar checking and language polishing of the manuscript.The disclosure assigns no other manuscript role to the LLM.Source: GenAI Usage Disclosure · p. 8
  • The authors state that they performed the scientific content, experimental design, analysis, and conclusions.The disclosure attributes these activities to the authors.Source: GenAI Usage Disclosure · p. 8
  • The disclosure distinguishes manuscript language polishing from the authors’ scientific work.It specifically identifies grammar checking and language polishing as the LLM’s sole use.Source: GenAI Usage Disclosure · p. 8

Related papers

Loading 2609.15611v1…