PaperlayerView exact revision on arXiv

Source-linked AI summary

BERT4DTI : BERT-based Model for Predicting Drug-Protein Interactions

Thanina Hamitouch, Khadidja Henni, Abdelkrim Arie, Amina Selma Haichour, Neila Mezghani, Lina Abou-Abbas

arXiv:2609.33254v1cs.LGcs.AIcs.CL

TL;DR

Drug–protein interaction prediction can help prioritize pairs before costly experiments, but sparse, biased interaction labels and the limits of independent sequence encoding complicate the task. BERT4DTI combines partially fine-tuned pretrained encoders with bidirectional mutual attention and convolutional classification; it is competitive across three benchmarks, especially in sensitivity, though trade-offs remain.

  • Problem

    Drug–protein interaction prediction can prioritize pairs before costly experiments, but known interactions are sparse and biased, unknown pairs can be uncertain negatives, and independently encoded sequences miss pair-specific dependencies.

    Source: §1 · p. 1

  • Method

    BERT4DTI combines partially fine-tuned ChemBERTa and ProtBERT encoders with bidirectional mutual attention and a convolutional classification head for binary DTI prediction.

    Source: §1 · p. 2, §2.1 · p. 2

  • Results

    BERT4DTI has the highest sensitivity across BIOSNAP, DAVIS, and BindingDB, and achieves the best BIOSNAP ROC-AUC (0.920) and PR-AUC (0.924).

    Source: §3.2 · p. 3, §3.4 · p. 3

  • Takeaways & Limitations

    BERT4DTI is a competitive screening model with reduced trainable size relative to full BERT fine-tuning, not a uniformly superior predictor.

    Source: §4 · p. 4

    Runtime, peak memory, inference latency, calibration, and leakage-aware external validation remain unevaluated or future-work needs.

    Source: §3.4 · p. 4

Abstract

from arXiv · show

Understanding how drugs interact with protein targets is fundamental to drug discovery, drug repurposing and the early identification of promising therapeutic candidates before costly experimental testing. Sequence-based DTI models face three practical limitations: labelled interactions are scarce and unevenly distributed, large pretrained chemical and protein encoders are expensive to fine-tune end-to-end, and independently encoded sequences do not capture pair-specific dependencies. We present BERT4DTI, which encodes SMILES strings with ChemBERTa and amino-acid sequences with ProtBERT, applies bidirectional mutual attention between token-level representations, and classifies the resulting interaction features using convolutional layers and a multilayer perceptron. To reduce trainable size, ProtBERT is truncated to 18 retained layers and only the last two layers of each encoder are fine-tuned. On BIOSNAP, DAVIS and BindingDB, BERT4DTI is competitive, achieving the best ROC-AUC and PR-AUC on BIOSNAP and the highest sensitivity on all three benchmarks. An ablation on DAVIS shows that mutual attention improves PR-AUC and specificity. With 125M trainable parameters compared with 353M for full BERT fine-tuning, BERT4DTI provides a favourable performance-parameter trade-off for sequence-based DTI screening, while leaving runtime profiling, calibration and leakage-audited validation for future work.

1 Introduction

Drug–protein interactions matter for drug discovery, off-target analysis, and repurposing, while computational prediction can help prioritize pairs before costly laboratory validation. BERT4DTI addresses challenges in sequence-based prediction by combining restricted encoder fine-tuning with explicit cross-modal interaction learning.

1 Introduction

  • In silico DTI prediction can prioritize plausible compound–protein pairs before costly, time-consuming experimental screening.Drug–protein interactions are central to drug discovery, off-target analysis, and drug repurposing.Source: §1 · p. 1
  • Known interactions cover a small, biased part of chemical–protein space, and treating unknown pairs as negatives introduces label uncertainty.Drugs and proteins also differ in vocabulary, sequence length, and biological meaning.Source: §1 · p. 1
  • Full fine-tuning of large pretrained encoders requires many trainable parameters, while frozen encoders may adapt insufficiently to DTI prediction.Late concatenation of independently encoded sequences also does not explicitly model which tokens matter in a particular pair.Source: §1 · p. 2
  • BERT4DTI combines partially fine-tuned ChemBERTa and ProtBERT with bidirectional mutual attention and a convolutional classification head.The paper evaluates it on BIOSNAP, DAVIS, and BindingDB, with a component ablation and trainable-parameter analysis.Source: §1 · p. 2

2 The BERT4DTI Method

BERT4DTI frames DTI prediction as binary classification of drug–protein sequence pairs. It partially fine-tunes pretrained chemical and protein encoders, uses bidirectional mutual attention to form pair-conditioned representations, and classifies them with convolutional branches and an MLP.

2.1 Problem Definition and Inputs

  • Each drug–protein sequence pair is classified by an estimated interaction probability, with a validation-selected threshold determining positive predictions.The architecture projects contextual token representations to a common dimension, applies bidirectional mutual attention, and uses independent 1D CNN blocks followed by an MLP.Source: §2.1 · p. 2

2.4 Convolutional Classification and Objective

  • Independent 1D CNN branches process the fused drug and protein features, which are concatenated and passed to an MLP with a sigmoid output.The reported configuration uses SmoothL1Loss; performance is interpreted with ROC-AUC, PR-AUC, sensitivity, and specificity rather than loss alone.Source: §2.4 · pp. 2–3

3 Experiments

Across BIOSNAP, DAVIS and BindingDB, BERT4DTI has a strong positive-retrieval profile but is not uniformly best across metrics. Ablations support mutual attention’s contribution on DAVIS, while parameter and validation caveats constrain efficiency and generalization claims.

3.1 Datasets, Protocol and Metrics

  • The fixed benchmark partitions include balanced BIOSNAP pairs and strongly imbalanced DAVIS and BindingDB test sets, for which PR-AUC and specificity provide essential context.Sequences are truncated or padded to 100 drug tokens and 545 protein tokens; results are averaged over five runs with validation ROC-AUC model selection.Source: §3.1 · p. 3

3.4 Discussion

  • On BIOSNAP, BERT4DTI’s specificity is lower than MolTrans and fine-tuned BERT, reflecting a positive-retrieval profile rather than uniform superiority.On DAVIS and BindingDB, other models lead on PR-AUC or specificity despite BERT4DTI’s sensitivity advantage.Source: §3.4 · p. 3

3.3 Ablation and Parameter Efficiency

  • On DAVIS, removing mutual attention lowers ROC-AUC from 0.92 to 0.89, PR-AUC from 0.37 to 0.29 and specificity from 0.83 to 0.75.Removing both modules recovers PR-AUC close to the full model but increases trainable size from 125M to 183M.Source: §3.4 · p. 3, §3.3 · p. 3, Table 2 · p. 4

3.4 Discussion

  • The ablation indicates that mutual attention contributes information beyond independent pretrained encodings and late fusion; removing the CNN branch has a smaller ROC-AUC and PR-AUC effect but reduces specificity.The latter result is consistent with local convolutional aggregation remaining useful after mutual attention.Source: §3.4 · pp. 3–4
  • BERT4DTI has 125M trainable parameters, versus 353M for full BERT fine-tuning, but more than MolTrans (62.8M) and DLM-DTI (86.7M).The reported efficiency comparison concerns trainable size, not measured runtime.Source: §3.3 · p. 3, §3.4 · p. 4, Table 3 · p. 4
  • Runtime, peak memory and inference latency were not measured, and depend on sequence length, frozen-layer computation, batch size and hardware.The study identifies a need for dedicated deployment profiling.Source: §3.4 · p. 4
  • Additional held-out, scarce-data, protein-family and CYP analyses are descriptive and do not establish general external generalization.They do not replace similarity-clustered splits, duplicate-pair auditing, calibration or an external benchmark with positive and negative evidence.Source: §3.4 · p. 4

4 Conclusion

BERT4DTI combines pretrained chemical and protein encoders with partial fine-tuning, mutual attention, and convolutional classification. It is a competitive screening model with a reduced trainable size, but is not universally dominant.

4 Conclusion

  • BERT4DTI is a competitive screening model, not a universally dominant predictor, with competing methods sometimes achieving better PR-AUC or specificity on imbalanced datasets.The conclusion also highlights strong results across three benchmarks, especially for positive-pair retrieval.Source: §4 · p. 4
  • BERT4DTI combines ChemBERTa and ProtBERT with partial fine-tuning, bidirectional mutual attention, and convolutional classification for sequence-based DTI prediction.The conclusion says the ablation study supports mutual attention's value for cross-sequence interaction modelling.Source: §4 · p. 4
  • Future work will integrate biomedical knowledge graphs and retrieval-augmented language models to support explainable, evidence-based interaction prioritization.Source: §4 · p. 4

GenAI Usage Disclosure

Generative AI tools were used for language polishing and formatting assistance only. The authors take responsibility for the scientific content and checked it against the original experimental report.

GenAI Usage Disclosure

Related papers

Loading 2609.33254v1…