Towards Generalizing Pretrained All-Atom Representations to Unified Binder Design

• Existing approaches often use all-atom representations for modality-specific design, while geometric latent diffusion represents different molecular modalities in a shared space.UniMoMo and AnewOmni support binder design across molecular classes.Source: §1 · p. 2
• PairFuse uses lightweight layerwise adapters to fuse pretrained single and pair representations into invariant block features and native geometric attention.It keeps both pretrained models frozen and retains the original latent space, sampler, and all-atom decoder.Source: §1 · pp. 2–3
• The augmented denoiser preserves rotational equivariance, and pairwise attention fusion adds expressivity beyond scalar-feature-only fusion under a stated rank condition.The comparison is against fixed-width native query–key projections.Source: §1 · pp. 2–3
• PairFuse improves structural accuracy and interface quality on antibody and peptide benchmarks, with favorable results on small-molecule design.The reported evaluation spans antibody, peptide, and small-molecule binder design.Source: §1 · pp. 2–3

• Unified geometric latent diffusion represents different molecular classes in a shared space, with each molecular block encoded by an invariant feature and a three-dimensional coordinate.Blocks can represent peptide amino-acid residues or small-molecule fragments.Source: §2 · p. 3
• A pretrained all-atom decoder reconstructs the binder from generated latent blocks and the design condition.Source: §2 · p. 3
• The EPT denoiser predicts noise on generated blocks while context blocks remain fixed during the forward noising process.The reverse-diffusion sampler also keeps context blocks fixed.Source: §2 · pp. 3–4
• EPT’s scalar features and attention logits remain invariant under global rotations, while its vector features transform equivariantly.These transformation properties ensure that the denoiser satisfies the stated rotational-equivariance condition.Source: §2 · p. 4
• PairFormer-based all-atom models learn single and pair representations that capture token-level structural information and pairwise interactions.The overview identifies BoltzGen PairFormer as the source of pretrained representations used alongside the geometric latent diffusion model.Source: §2 · pp. 4–5, Figure 1 · p. 5

3 Pretrained All-Atom Representation Fusion

• PairFuse adapts AnewOmni with frozen BoltzGen PairFormer representations, training only layerwise fusion adapters.It integrates representations into evolving invariant scalar features and native geometric attention logits.Source: §3 · p. 5

3.1 Layerwise All-Atom Representation Fusion

• Scalar-feature fusion enriches EPT features, while pairwise attention fusion adds a bias to native geometric attention logits using pretrained single and pair representations.The augmented attention aggregates scalar and vector features through the original EPT operations without modifying its equivariant vector operations.Source: §3.1 · pp. 5–7

3.2 Theory

• Under rotation-invariant conditioning features, transferred representations, and correspondence masks, the augmented denoiser preserves rotational equivariance for any fusion parameters.Scalar outputs remain invariant and coordinate outputs transform equivariantly under rotation.Source: §3.2 · p. 7
• When the stated rank condition holds, direct pairwise attention fusion can express attention patterns that scalar-feature-only adapters cannot reproduce through fixed-width native query–key projections.The condition is rank(T) > datt.Source: §3.2 · p. 8

3.3 Training and Inference

• Training freezes the BoltzGen trunk and AnewOmni parameters and optimizes only fusion parameters; inference reuses cached PairFormer representations with the native sampler and decoder unchanged.No additional structural supervision is introduced.Source: §3.3 · p. 9

4.1 Experimental Setup

• The evaluation compares PairFuse with AnewOmni, LLM4GEN, EnzyAdapter, and IP-Adapter across fixed antibody, peptide, and small-molecule benchmark sets, using matched trainable parameter counts for trained fusion methods.The fixed sets contain 60 RAbD targets, 93 LNR targets, and 100 CrossDocked2020 targets; results for trained methods average three seeds.Source: §4.1 · pp. 9–10, §4.2 · p. 10

4.2 Main Results

• On RAbD, PairFuse reduces complex RMSD from 2.5563 to 2.3748 Å, raises DockQ from 0.6853 to 0.7019, and raises native-contact recall from 50.9592% to 52.4309% versus AnewOmni.The fraction of designs with receptor-aligned HCDR3 Cα RMSD ≤2 Å increases from 34.00% to 41.61%; LLM4GEN has the highest amino-acid recovery.Source: Table 1 · p. 10, §4.2 · p. 11
• On LNR, PairFuse reduces complex RMSD from 8.5619 to 8.2821 Å and raises DockQ from 0.4280 to 0.4383 versus AnewOmni, while amino-acid recovery decreases slightly from 6.4306% to 6.3436%.PairFuse also has the highest mean scores on all three interface-quality metrics; IP-Adapter has the highest sequence recovery at 6.5084%.Source: §4.2 · p. 11, Table 2 · p. 11
• On CrossDocked2020, PairFuse lowers Vina score from -5.1279 to -5.1517 kcal/mol and Atom-type MAE from 0.4188 to 0.3775 versus AnewOmni, while QED is 0.5721.Vina min decreases from -5.9715 to -5.9841 kcal/mol and High Affinity increases from 44.8174% to 46.8298%.Source: §4.2 · p. 11, Table 3 · p. 11
• Across antibody and peptide design, PairFuse has the lowest mean complex RMSD and highest mean scores on all three interface metrics; small-molecule docking results are favorable, with other chemical-quality metrics broadly comparable.The comparisons cover RAbD, LNR, and CrossDocked2020.Source: Table 1 · p. 10, Table 2 · p. 11, Table 3 · p. 11

4.3 Ablation Studies and Implementation Details

• In seed-42 ablations, Full PairFuse outperforms Scalar Fusion and Scalar Fusion + Cross-Attention Bias on all three interface metrics and lowers complex RMSD for both peptide and antibody design.All six pooled paired improvements in the interface metrics have pointwise 95% target-bootstrap confidence intervals above zero, supporting an additional benefit from direct pair-representation transfer into native attention.Source: §4.3 · p. 12, Table 4 · p. 12

5 Discussion and Conclusion

• PairFuse integrates frozen PairFormer single and pair representations into a geometric latent diffusion model, while preserving rotational equivariance and adding attention expressivity under a stated rank condition.The reported benchmark outcomes are strongest on antibody and peptide structural and interface metrics; small-molecule results vary across chemical-quality metrics.Source: §5 · p. 13
• Unknown generated structures, particularly small-molecule ligands, may lack direct PairFormer token correspondences, potentially limiting representation-transfer effectiveness.The paper identifies better accommodation of unknown structures and additional pretrained all-atom models as future directions.Source: §5 · p. 13

A Related Work

• Unified models such as UniMoMo and AnewOmni represent different molecular classes in shared block-level latent states, while PairFuse brings pretrained all-atom representations into AnewOmni's shared latent space [Kong et al., 2025, 2026].Other related directions include atom-level interaction modeling and broader all-atom biomolecular design.Source: A Related Work · p. 15
• All-atom models maintain token-level and pairwise representations, and PairFuse extracts these from frozen BoltzGen PairFormer to represent molecular blocks and their interactions.The related-work discussion also covers structure prediction and generation across proteins, nucleic acids, ligands, and other molecular entities.Source: A Related Work · p. 15
• PairFuse extends lightweight adaptation to unified biomolecular design by transferring PairFormer representations through layerwise scalar residuals and attention biases while keeping both pretrained backbones frozen.This connects adaptation approaches such as IP-Adapter with geometric latent diffusion [Ye et al., 2023].Source: A Related Work · p. 16

B Proof of Theorem 3.1

• Under the stated assumptions, the augmented denoiser's coordinate prediction transforms equivariantly under rotation for arbitrary fusion parameters.The proof compares denoiser evaluations at the original and jointly rotated generated and context coordinates.Source: B Proof of Theorem 3.1 · pp. 17, 19
• The induction begins with invariant native inputs and geometric initialization whose scalar features remain invariant while vector features rotate consistently with coordinates.The proof uses unchanged pairwise distances and the assumed equivariance of the native initialization.Source: B Proof of Theorem 3.1 · p. 17
• At each layer, scalar-feature and pairwise-attention fusion preserve invariant attention weights, so scalar and vector attention outputs retain their respective transformation laws.The argument assumes invariant transferred representations and masks, as well as native operations and EPT layers that preserve the required properties.Source: B Proof of Theorem 3.1 · pp. 17–18

C Proof of Theorem 3.2

• The comparison fixes the geometric bias G while allowing scalar-feature-only adapters access to both S and P, but they add no direct bias to native attention logits.The bound follows from the native query–key dimension, not restrictions on the scalar-feature adapter.Source: C Proof of Theorem 3.2 · pp. 20–21
• If rank(T) > datt, no scalar-feature-only adapter reproduces PairFuse attention at the fixed head and state.The proof compares attention weights using the same native projections and geometric bias.Source: C Proof of Theorem 3.2 · p. 20
• If rank(T) > datt + db, even a comparison retaining single-conditioned bias cannot reproduce PairFuse attention without the explicit pair-conditioned term.This is the stronger expressivity condition established for the explicit pair-conditioned attention bias.Source: C Proof of Theorem 3.2 · pp. 20, 22

D.1 Training, Validation, and Test Data

• Test evaluation uses 60 RAbD antibody, 93 LNR peptide, and 100 CrossDocked2020 small-molecule targets, with 30 generation attempts per target.Validation selects one shared checkpoint for all three modalities; the benchmark collections are separate from source training and validation partitions.Source: §D.1 · p. 23

D.2 Construction of Pretrained Donor Representations

• Frozen BoltzGen PairFormer single and pair representations encode token-level structural information and pairwise relationships without requiring a sampled BoltzGen structure.They are aligned with AnewOmni blocks and reused during training and denoising. Generated ligand blocks lack directly aligned single representations and pair representations requiring both endpoints to be covered.Source: §D.2 · pp. 23–24

D.3 Training and Model Selection

• The shared checkpoint is selected by the lowest combined native validation total loss on 614 records, while both pretrained backbones remain frozen.The loss combines latent-feature and coordinate denoising losses, weighting modality-level means by generated-block counts.Source: §D.3 · p. 25

D.4 Hyperparameters and Optimization

• PairFuse uses six fusion layers, one at each EPT layer, and AdamW with learning rate 3 × 10−4 and weight decay 0.01.The learning rate uses a 100-update warmup and cosine schedule; training runs use three seeds.Source: §D.4 · p. 25

D.5 Inference and Evaluation

• 60 RAbD, 93 LNR, and 100 CrossDocked2020 targets receive 30 generation attempts each, using 100 diffusion steps and batch size 10.The benchmark totals 253 targets.Source: §D.5 · p. 25
• Antibody and peptide evaluation reports structural, sequence, steric, energy, and interface metrics, while small-molecule evaluation reports docking, drug-likeness, synthetic-accessibility, and rule-compliance metrics.Interface metrics average candidates within each target and weight targets equally; small-molecule candidate scores are pooled.Source: §D.5 · p. 26
• DockQ combines native-contact recovery, interface RMSD, and receptor-aligned binder RMSD using the standard DockQ normalization.Native-contact recall counts reference interfacial residue pairs below 5 Å recovered in the generated complex; interface RMSD uses interface residues defined by a 10 Å interchain threshold.Source: §D.5 · p. 26

E Ablations and Computational Analysis

• The ablations examine single- and pair-representation contributions and quantify layerwise-fusion inference runtime and memory overhead against the pretrained baseline.Source: E Ablations and Computational Analysis · p. 26

E.1 Single and Pair Representations

• Joint fusion reduces antibody complex RMSD from 2.4263 to 2.3769 Å and raises DockQ from 0.6980 to 0.7025; for peptides, it reduces complex RMSD from 8.4471 to 8.2497 Å and raises DockQ from 0.4324 to 0.4393.The comparison covers single-only, pair-only, and joint fusion; Table 6 reports antibody results and Table 7 reports peptide results.Source: §E.1 · p. 27, Table 6 · p. 27, Table 7 · p. 27
• Adding single representations to pair-only fusion improves all five displayed antibody and peptide metrics, although pair-only fusion gives the lower peptide intra-region clash rate (0.8165% versus 0.8982%).For antibodies, Rosetta dG decreases from −2.2303 to −8.1333 REU; for peptides, interface RMSD decreases from 3.0615 to 2.9919 Å and contact recall increases from 33.4552% to 34.1400%.Source: §E.1 · pp. 27–28
• Pair-only fusion yields the highest small-molecule QED (0.5801), exceeding joint fusion (0.5755).Single representations complement pair representations more consistently for antibody and peptide design, with different effects across ligand properties.Source: Table 8 · p. 27, §E.1 · p. 28
• Full PairFuse outperforms an equally sized matched adapter without pretrained donor representations on antibody and peptide complex RMSD, DockQ, contact recall, and backbone interface lDDT.The matched adapter uses identical data and compute budgets; its small-molecule Vina score, Vina min, and Vina dock are −5.1011, −6.0286, and −7.0355 kcal/mol, respectively, with QED 0.5766.Source: §E.1 · p. 28

E.2 Computational Cost

• PairFuse adds 12.6%, 27.0%, and 39.2% runtime overhead for antibody, peptide, and small-molecule batches, respectively, with peak allocated memory increasing by approximately 0.19, 0.12, and 0.17 GB.Measurements use one NVIDIA RTX A6000, batch size ten, and 100 diffusion steps; timing includes sampling and all-atom decoding but excludes donor-feature extraction and projection-cache preparation.Source: §E.2 · p. 28, Table 9 · p. 28

F.1 Native EPT Architecture

• Native EPT initializes graph features and then applies EPT layers with equivariant self-attention and feed-forward operations.The algorithm excerpts also identify graph initialization from invariant features and coordinates.Source: F Algorithms · p. 28, §F.1 · p. 29

F.2 Architecture of the PairFormer-Augmented EPT

• The PairFormer-augmented EPT retains native graph initialization, feed-forward layers, and conditioning, adding scalar-feature fusion and an attention-logit bias.The added mechanisms are highlighted as modifications to the native EPT computation.Source: §F.2 · pp. 29–30

F.3 PairFormer Fusion Modules

• Scalar-feature fusion adds a representation-derived update to each native scalar feature, using aligned block representations and a coverage mask.The algorithm defines the update as ΔU and adds it to U at each layer.Source: §F.2 · p. 30, §F.3 · p. 30
• Pairwise attention bias incorporates pair representations into attention logits, with pair-dependent terms active only when both blocks have available representations.The masked softmax attends only to keys whose coverage mask is active.Source: §F.3 · p. 30, §F.2 · p. 30
• Fusion projections are layer-specific, while the single- and pair-representation encoders are shared across layers.Source: §F.3 · p. 30

G Complete Benchmark Metrics

• Best and second-best means are marked only for metrics with a specified preference direction.The tables follow the model and aggregation settings in Sections 4.2, D.4, and D.5.Source: G Complete Benchmark Metrics · p. 31

G.1 Antibody design on RAbD

• The complete antibody metrics are reported for RAbD.Source: Table 10 · p. 31

G.2 Peptide design on LNR

• The complete peptide metrics are reported for LNR.Source: Table 11 · p. 32

G.3 Small-molecule design on CrossDocked2020

• The table continuation covers small-molecule design on CrossDocked2020.Source: Table 12 · p. 32, §G.3 · p. 33
• The listed model columns are AnewOmni, LLM4GEN, EnzyAdapter, IP-Adapter, and PairFuse.Source: §G.3 · p. 33
• The continuation includes functional-group entries for c1ccc2ncccc2c1, c1ccc2[nH]cnc2c1, and c1ccc2ccccc2c1.Source: §G.3 · p. 33