PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning from Protein Structure with Geometric Vector Perceptrons

Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael J. L. Townshend, Ron Dror

arXiv:2009.01411v3q-bio.BMcs.LGstat.ML

TL;DR

Learning from macromolecular structure requires jointly handling geometric and relational information, but existing architectures generally emphasize one aspect. The paper introduces GVPs within GNNs to process scalar and Euclidean vector features, and reports superior performance on protein design and model quality assessment. The approach is presented as a general architecture for structure-based learning, with further applications proposed for other biomolecular settings.

  • Problem

    Existing structure-learning methods typically leverage either geometric or relational information, leaving a unifying architecture for both aspects unavailable.

  • Method

    GVP-GNN augments graph neural networks with geometric vector perceptrons that operate directly on scalar and Euclidean vector features.

  • Results

    GVP-GNN empirically outperforms existing architectures on protein model quality assessment and computational protein design.

  • Takeaways & Limitations

    The architecture provides a unified approach for learning from dual geometric and relational representations of 3D macromolecular structure.

  • Takeaways & Limitations

    The demonstrated applications are limited to protein model quality assessment and computational protein design, with other areas left for future work.

Abstract

from arXiv · show

Learning on 3D structures of large biomolecules is emerging as a distinct area in machine learning, but there has yet to emerge a unifying network architecture that simultaneously leverages the graph-structured and geometric aspects of the problem domain. To address this gap, we introduce geometric vector perceptrons, which extend standard dense layers to operate on collections of Euclidean vectors. Graph neural networks equipped with such layers are able to perform both geometric and relational reasoning on efficient and natural representations of macromolecular structure. We demonstrate our approach on two important problems in learning from protein structure: model quality assessment and computational protein design. Our approach improves over existing classes of architectures, including state-of-the-art graph-based and voxel-based methods. We release our code at https://github.com/drorlab/gvp.

1 INTRODUCTION

Learning from macromolecular structure combines 3D geometric information with relational structure, but existing methods typically emphasize one aspect. The paper introduces GVP-GNN to bridge these aspects and demonstrates it on protein design and model quality assessment.

  • Motivation: Macromolecular structures are represented by atom or group positions in 3D Euclidean space, supporting predictions about structural properties.Examples include assessing model quality and predicting whether molecules bind in a particular geometry.
  • Motivation: Protein learning requires both spatial arrangement and orientation, which govern molecular dynamics and function, and relational structure from sequences and residue interactions.
  • Motivation: Recent methods commonly use GNNs for relational reasoning or CNNs for direct geometric processing, rather than jointly leveraging both aspects.
  • Contribution: GVPs replace standard MLPs in GNN aggregation and feed-forward layers while operating directly on scalar and geometric vector features.This preserves geometric information that might be lost by reducing vector features to scalars.
  • Experiments: GVP-GNN is demonstrated on computational protein design and model quality assessment, and outperforms existing methods on both tasks.

2 RELATED WORK

Prior protein-structure learning methods use sequential, voxelized, or graph-structured representations. These approaches emphasize handcrafted sequence features, geometric convolutions, or relational reasoning, respectively.

  • Representation families: Protein-structure learning methods largely operate on sequential, voxelized, or graph-structured representations.The paper reviews these categories before evaluating its architecture.
  • Sequential representations: Sequential methods encode each amino acid with handcrafted features describing contacts, orientations, energy terms, or protein topology.
  • Voxelized representations: Voxelized methods apply 3D CNNs directly to atom positions represented as occupancy maps, enabling detection of structural motifs and pockets.
  • Graph-structured representations: Graph representations model proteins as proximity graphs, allowing GNNs to reason about amino-acid relationships and flexible connectivity patterns.

3 METHODS

The method represents protein backbones as proximity graphs with scalar and vector features, then propagates these features using GVP-based message passing. GVPs preserve geometric transformation properties while supporting expressive invariant scalar functions.

  • 3 METHODS: The architecture directly represents 3D geometric features as vectors in R3 throughout graph propagation instead of converting them to rotation-invariant scalars.This uses a shared global coordinate system and avoids transforming features between local coordinate systems.
  • 3.1 GEOMETRIC VECTOR PERCEPTRONS: A GVP maps scalar features s ∈ Rn and vector features V ∈ Rν×3 to new scalar and vector features s′ ∈ Rm and V′ ∈ Rµ×3.It is designed to learn vector-valued and scalar-valued functions over geometric features.
  • 3.1 GEOMETRIC VECTOR PERCEPTRONS: GVPs use linear transformations, nonlinearities, vector norms, and vector-channel scaling to extract rotation-invariant information while controlling output dimensionality.
  • 3.1 GEOMETRIC VECTOR PERCEPTRONS: Under arbitrary rotations and reflections, GVP scalar outputs are invariant and vector outputs are equivariant.These transformation properties are stated as provable architectural guarantees.
  • 3.1 GEOMETRIC VECTOR PERCEPTRONS: A GVP that maps vectors to scalars can approximate any continuous rotation- and reflection-invariant scalar function under mild assumptions.The result is formalized for inputs containing at least three linearly independent vectors and bounded nonzero norms.
  • Experiments: On a synthetic dataset, GVP-GNN matches CNN performance on a geometric task and standard GNN performance on a relational task, while performing significantly better when both are combined.
  • 3.2 REPRESENTATIONS OF PROTEINS: Protein backbones are encoded as proximity graphs whose nodes represent amino acids and whose edges connect each residue to its 30 nearest Cα neighbors.Node features include dihedral-angle encodings, orientation vectors, imputed Cβ directions, and amino-acid identity when available.
  • 3.3 NETWORK ARCHITECTURE: GVP-GNN performs message passing in which neighboring node and edge embeddings update node representations through graph propagation steps.The inputs include scalar and vector features describing the protein backbone.

4 EVALUATION METRICS AND DATASETS

The evaluation covers computational protein design and model quality assessment using held-out structural datasets and task-specific metrics. CPD uses native sequence recovery and perplexity, while MQA evaluates global and per-target correlations.

  • Protein design: CPD infers amino acid sequences for given structures, but native sequence recovery is used as a practical proxy because direct benchmarking is difficult.Perplexity on held-out native sequences is also reported, with both metrics assuming native sequences are optimized for their structures.
  • Protein design: The CATH 4.2 CPD split contains 18,204 training, 608 validation, and 1,120 test structures partitioned by CATH classification to reduce structural similarity across splits.
  • Protein design: TS50 provides an additional 50-structure CPD test set and enables comparison with Rosetta’s computationally expensive fixbb protocol.
  • Model quality assessment: MQA selects the best model from candidate structures and should assess both absolute quality and relative quality within each target’s candidate pool.
  • Reported results: On CATH 4.2, GVP-GNN achieves state-of-the-art protein-design performance in per-residue perplexity and sequence recovery against Structured Transformer.Perplexity is lower-is-better, while recovery is higher-is-better and summarizes correctly recovered residues across sampled sequences.
  • Model quality assessment: MQA is evaluated using global correlation pooled across targets and average per-target correlation among candidates for each target.
  • Model quality assessment: The MQA training and validation data contain 79,200 candidate structures for 528 CASP 5–10 targets, followed by testing on later CASP datasets.

5 EXPERIMENTS

Experiments show that GVP-GNN performs strongly on computational protein design and model quality assessment, while ablations indicate that direct access to both scalar and vector features is important.

  • Protein design: GVP-GNN achieves state-of-the-art protein-design performance on CATH 4.2, substantially improving perplexity and sequence recovery over Structured Transformer.The comparison uses the same training and validation sets; performance also improves over a retrained Structured GNN variant.
  • Protein design: 44.9% recovery on TS50 compares with Rosetta’s 30%, while GVP-GNN outperforms methods based on all three structural-representation classes.The authors place it second out of nine methods, but caution that training datasets were not canonical across methods.
  • Model quality assessment: GVP-GNN outperforms all other structural methods on CASP 11–12 in both global and mean per-target correlation, and exceeds ProQ3D on all but one benchmark.The table reports global and mean per-target Pearson correlation coefficients, with higher values better; ProQ3D additionally uses sequence profiles.
  • Model quality assessment: On CASP 13, GVP-GNN outperforms all methods in global correlation and all structure-only methods in per-target correlation.The comparison covers 23 single-structure methods with complete predictions on 20 evaluation targets; Table 3 identifies GVP-GNN as top overall in global correlation.
  • Interpretability: Visualizing intermediate vector features is possible because the MQA network updates vector features alongside scalar features at each node embedding.The paper discusses interpretability of these learned vector features in Appendix F.
  • Ablation studies: Removing direct geometric access through vanilla MLP layers or scalar-only propagation causes considerable performance decreases on CPD and MQA.Vector-only propagation performs even worse because it removes scalar inputs and the GVP component with approximation guarantees.

6 CONCLUSION

GVP-GNN is designed for learning from macromolecular structures that combine geometric and relational information. It offers theoretical equivariance and empirically outperforms existing architectures on protein quality scores and sequence designs.

  • GVP-GNN augments graph neural networks with simple layers that perform geometric reasoning over Euclidean vector features.
  • The authors identify protein complexes, RNA structure, and protein-ligand interactions as future application areas.
  • GVP outputs are invariant for scalar features and equivariant for vector features under arbitrary 3D rotations and reflections.

A.2 APPROXIMATION OF ROTATION-INVARIANT FUNCTIONS

The GVP architecture has an analogue of the universal approximation property for continuous scalar functions invariant to 3D rotations and reflections. The construction uses vector transformations, norms, and a dense network to recover invariant coordinates and approximate the target function.

  • GVPs can approximate arbitrary continuous scalar-valued functions invariant under rotations and reflections, given bounded inputs with three linearly independent vector entries.
  • A vector-to-scalar GVP followed by a dense network with one hidden layer provides the approximation machinery.
  • The proof factors the target as F = ˜F ◦ω, where ω canonically orients the input and extracts 3ν − 3 coefficients.
  • The resulting invariant coordinates use norms and inner products, which remain unchanged under rotations and reflections.
  • Constructing Wh from vectors and selected pairwise differences lets row-wise L2 norms compute the orientation representation needed by the approximation.

B SYNTHETIC TASKS

Synthetic experiments separate geometric and relational reasoning using controlled 3D point-structure tasks. GVP-GNN combines the strengths of CNNs and standard GNNs, especially when both objectives are trained together.

  • Dataset: The synthetic dataset contains n = 100 random points in R3 within a ball of radius r = 10, separated by at least distance d = 2, with associated unit vectors.
  • Representations: The voxelized representation preserves coordinates through trilinear interpolation and represents sidechains in a separate channel.
  • Representations: The graph representation uses a k = 10 nearest-neighbor proximity graph with node types, edge lengths, and orientation features.
  • Tasks: The off-center task measures geometric reasoning, while the perimeter task measures relational reasoning about the triangle formed by three special points.
  • Models: All models use intermediate dimensionality 32 and the same training procedure, without hyperparameter tuning or architecture search.
  • Results: GVP-GNN matches or outperforms CNNs on the geometric task, maintains GNN performance on the relational task, and significantly outperforms both on the combined task.

C MQA DATASETS: FURTHER DETAILS

The MQA datasets use CASP targets and candidate structures to evaluate protein model quality assessment. Training and validation include native structures, whereas testing excludes native structures to mimic CASP and real-world use.

  • Datasets: The MQA training and validation data contain 528 targets from CASP 5–10, with 150 candidate structures per target and a 480/48 train-validation split.
  • Datasets: Native structures are included for training and validation to span the greatest range of GDT-TS scores but excluded from testing.
  • CASP assessments: CASP stage 1 contains 20 candidate structures per target, whereas stage 2 contains 150 candidates per target from structure prediction servers.
  • Test sets: The CASP 11 and 12 test sets follow the list established by Karasikov et al., while the CASP 13 set contains 1472 stage 2 candidates from 20 targets.
  • Dataset table: Table 6 is identified as the table presenting the MQA datasets.

D TRAINING AND HYPERPARAMETERS

Training uses task-specific losses and shared GVP-GNN configurations, while comparisons evaluate scalability and performance against established MQA architectures.

  • Training objectives: MQA training combines an absolute loss with a pairwise loss over candidate-structure pairs for the same target.The pairwise term also improves global correlation, likely because the larger number of possible pairs makes overfitting more difficult.
  • Training objectives: CPD training uses cross-entropy or negative log likelihood loss for classification or discrete generative modelling.
  • Shared configuration: Both models use 16 vector and 100 scalar node and hidden channels, plus 1 vector and 32 scalar edge channels.Inputs are transformed by GVPs before graph propagation.
  • Hyperparameters: The search varies graph propagation layers from 3 to 6 and the relative weight of the MQA pairwise loss from 0 to 2.
  • Training resources: Training takes around two days for both models on a single Titan X GPU, with GPU memory rather than compute power identified as the bottleneck.The authors anticipate that runtime can be further optimized.
  • MQA comparisons: On CASP 11-12, DimeNet does not perform comparably to either GVP-GNN or GraphQA, whereas GVP-GNN is best-performing by global correlation on CASP 13.For CASP 13 per-target correlation, GVP-GNN outperforms other structure-only methods and most methods using sequence profiles.

E.3 CPD: RESULTS ON TS50

On the TS50 computational protein design benchmark, GVP-GNN outperforms the compared methods except DenseCPD, which remains best-performing.

  • Results: GVP-GNN outperforms all other compared TS50 methods except DenseCPD.The comparison includes CNN, distance-map, sequential-representation, and ProteinSolver methods.
  • Compared methods: The TS50 comparison includes ProDCoNN, DenseCPD, SBROF, Wang’s model, SPIN2, and ProteinSolver.
  • Evaluation setup: ProteinSolver is evaluated by sampling 100 sequences at temperature 1 for each structure using its public web server.
  • Evaluation setup: Because TS50 has no canonical training and validation sets, the authors remove sequences with more than 30% similarity from CATH 4.2 training and validation sets before retraining.
  • Results: GVP-GNN is the second-best-performing method on TS50 sequence recovery, behind the CNN method DenseCPD.For GVP-GNN and ProteinSolver, recovery follows the definition used in Table 1; other methods use classification accuracy.

F VISUALIZATION AND INTERPRETATION OF LEARNED FEATURES

The MQA model learns intermediate vector channels that can display visually interpretable geometric patterns across protein structures.

  • Learned features: Among 64 intermediate vector channels learned by the MQA model, several appear visually interpretable.The GVP updates vector features at node embeddings during graph propagation, so intermediate vectors are learned rather than merely input directions.
  • Learned features: Four visualized channels point toward compacting motion, along an alpha-helix axis, outward from the structure, or inward into the structure.Vectors are rooted at the amino-acid node positions on the backbone.
Loading 2009.01411v3…