PaperlayerView exact revision on arXiv

Source-linked AI summary

QProtFuse: Query-Guided Layer Fusion for Retrieval-Augmented Protein Function Prediction

Mianzhi Pan, Peishuo Liu, Yawen Ouyang, Botian Wang, Yiming Rong, Hao Zhou, Jianbing Zhang

arXiv:2610.30180v1cs.IRq-bio.QM

TL;DR

Protein RAG methods emphasize external retrieval signals while underexploring the rich knowledge encoded within a single PLM. QProtFuse adaptively fuses multi-layer protein features using the query and jointly trains protein-to-protein and protein-to-annotation alignment. On Prot-Inst-OOD, QProtFuse outperforms all baselines in protein and annotation retrieval, reaching Recall@1 of 16.4% and Recall@10 of 25.9% with our reranking configuration, and improves downstream function descriptions.

  • Problem

    Protein RAG methods emphasize external retrieval signals while underexploring the rich knowledge encoded within a single PLM.

    Source: §1 · p. 2

  • Method

    QProtFuse adaptively fuses multi-layer protein features using the query and jointly trains protein-to-protein and protein-to-annotation alignment.

    Source: §1 · p. 2

  • Results

    On Prot-Inst-OOD, QProtFuse outperforms all baselines in protein and annotation retrieval, reaching Recall@1 of 16.4% and Recall@10 of 25.9% with our reranking configuration, and improves downstream function descriptions.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Adaptive use of a single PLM’s internal knowledge is an effective approach to retrieval-augmented protein function prediction.

    Source: §5 · p. 9

    Evaluation focuses on Prot-Inst-OOD, and generated responses are not subsequently refined or verified against retrieved evidence.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Predicting protein functions from amino acid sequences is essential to understanding biological processes. Large language models (LLMs) offer flexible function descriptions to natural-language queries, yet their limited protein understanding motivates retrieval-augmented generation (RAG). Existing protein RAG methods primarily expand external retrieval signals while overlooking the rich knowledge encoded within a single protein language model (PLM). We show that retrieval performance varies across PLM layers and biological tasks, making conventional final-layer selection suboptimal. To this end, we propose QProtFuse, which uses queries to adaptively fuse multi-layer protein features and jointly learns protein and annotation retrieval through intra-modal and cross-modal contrastive objectives. On Prot-Inst-OOD, QProtFuse achieves the best protein and annotation retrieval scores, reaching 16.4\% Recall@1 with annotation reranking. These gains improve downstream RAG-based function descriptions, yielding the highest LLM-as-a-judge scores for both Qwen3-8B (6.22/10) and DeepSeek-V4-Pro (6.68/10). Ablations and weight analyses demonstrate the value of query-guided fusion and complementary training objectives for exploiting a PLM's internal knowledge.

1 Introduction

Protein RAG methods use external retrieval signals while leaving the knowledge within a single protein language model insufficiently explored. QProtFuse adaptively fuses layer features by query and achieves strong retrieval and downstream function-description results on Prot-Inst-OOD.

1 Introduction

  • Protein RAG methods emphasize external retrieval signals, while the distinct knowledge encoded across a single PLM’s layers remains underexplored.The paper frames this as a question of whether layers have different retrieval strengths and how to use that internal knowledge.Source: §1 · p. 2
  • Across four Prot-Inst-OOD subtasks, different ESM-2 layers achieve the best retrieval; uniform layer averaging beats MMseqs2- or ProstT5-enhanced features but generally trails task-best layers.These findings motivate a more adaptive fusion strategy.Source: §1 · p. 2
  • QProtFuse uses query-guided layer fusion and jointly aligns proteins with proteins and annotations to exploit a PLM’s internal knowledge.The two retrieval objectives support both protein and annotation retrieval.Source: §1 · p. 2
  • QProtFuse achieves superior protein and annotation retrieval on Prot-Inst-OOD, reaching 16.4% Recall@1 and 25.9% Recall@10 with annotation reranking, and the highest judge scores among evaluated methods.The judge scores are 6.22/10 for Qwen3-8B and 6.68/10 for DeepSeek-V4-Pro.Source: §1 · p. 2

2 Related Work

Prior protein-understanding methods add external retrieval signals, while layerwise probing finds useful biological information beyond final-layer features. Work on multi-layer fusion in multimodal LLMs motivates adapting layer fusion to protein retrieval.

2.1 LLMs for Protein Understanding

  • Protein-oriented LLMs use direct sequence tokenization or encoder-based alignment, while protein RAG methods add external similarity signals; QProtFuse instead combines a single PLM’s layers according to the query.Earlier work includes MMseqs2 and ESM dual-index retrieval and query-conditioned ESM–ProstT5 feature ensembling.Source: §2.1 · pp. 2–3

2.2 Layerwise Probing in PLMs

  • Prior probing finds structural and functional information across PLM depth, including intermediate ESM-2 layers outperforming the final layer for kinase function prediction (Kumar and Jha, 2025).Beshkov and Malthe-Sorenssen (2026) report structural fidelity highest near, but before, the final layer.Source: §2.2 · p. 3

2.3 Multi-Layer Feature Fusion

  • Multimodal LLM work combines encoder layers and uses textual queries to adapt their weights; QProtFuse extends layer fusion to PLMs with joint protein-to-protein and protein-to-description contrastive objectives.Prior visual fusion work includes Dense Connector and query-adaptive weighting.Source: §2.3 · pp. 3–4

3 Method

QProtFuse treats retrieval effectiveness as a proxy for selecting evidence for query-conditioned protein-function generation. It groups and fuses protein-language-model layers according to the query, trains protein and annotation alignment objectives, and supports three retrieval modes.

3 Method

  • QProtFuse trains a retriever to select reference evidence for a protein sequence and natural-language query, using retrieval effectiveness as a proxy for evidence quality.Directly optimizing evidence selection would require searching a prohibitively large space and costly LLM inference.Source: §3 · p. 4

3.1 Query-Guided Layer Fusion

  • Neighboring ESM layers are similar, so QProtFuse groups consecutive layers and uses the biological query to weight their contributions to a query-conditioned protein feature.The method partitions layers into seven groups; the same sequence can receive different layer mixtures under different queries.Source: §3.1 · pp. 4–5, Figure 3 · p. 4

3.2 Joint Contrastive Objectives

  • Joint training combines protein-to-protein alignment, which separates functionally related and unrelated proteins under a query, with protein-to-annotation alignment.Positive pairs share a target description; eligible negatives have similar instructions but dissimilar descriptions. Both objectives update the fusion network while the ESM and text encoders remain frozen.Source: §3.2 · p. 5

3.3 Inference

  • The same checkpoint supports protein retrieval, annotation retrieval, and protein retrieval followed by annotation reranking.Protein retrieval rankings can vary with the biological perspective in the query, while reranking uses annotation relevance to refine a protein shortlist.Source: §3.3 · pp. 5–6

4 Experiments

On Prot-Inst-OOD, QProtFuse leads PP and PA retrieval and improves downstream function description. Ablations and weight analyses support query-guided fusion and complementary training objectives, with task- and query-specific layer mixtures.

4.1 Experimental Setup

  • Experiments use Prot-Inst-OOD’s out-of-distribution split and retrieval setup, assessing retrieval and generated descriptions against representative RAG, retrieval, and language-model baselines.The benchmark covers four tasks; retrieval uses Recall@K and MRR, while generation evaluation includes ROUGE-L, Entity-BLEU, and LLM-as-a-judge scores.Source: §4 · p. 6, §4.1 · p. 6

4.2.1 Retrieval Results

  • QProtFuse scores highest on all three PP and PA retrieval metrics and improves over ESM all-layer averaging in MRR (14.0 to 14.7) and R@10 (24.6 to 25.9).QProtFuse also outperforms reranked ProtQueSt on all three metrics and has the highest MRR and R@1 among Hybrid methods; PA scores are not directly comparable with PP and Hybrid because PA ranks fewer candidates.Source: Table 1 · p. 6, §4.2.1 · p. 7

4.2.2 RAG Generation Results

  • With Qwen3-8B, QProtFuse outperforms RAPM and ProtQueSt on all three generation metrics; with DeepSeek-V4-Pro, it has the highest Judge-F, while ProtQueSt retains the highest Entity-BLEU.QProtFuse improves judge precision over ProtQueSt across all four tasks with both LLMs.Source: Table 2 · p. 7, §4.2.2 · p. 7

4.3 Ablation and Analysis

  • Query-guided fusion outperforms fixed-mean and global-weight strategies in PA and Hybrid retrieval, while joint training improves both retrieval modes over single-objective training, especially PA.PP retrieval remains comparable across fusion strategies; global weights offer little improvement over fixed mean.Source: Figure 4 · p. 7, §4.3 · pp. 7–8
  • Joint training yields task-specific mixtures of early, middle, and late layers, while single-objective training often collapses toward one group; query examples show distinct layer preferences.Catalytic queries combine early and late layers, general-function queries favor the final group, and domain/motif queries draw across depths; localization emphasizes early-to-middle layers, unlike general-property descriptions.Source: Figure 5 · p. 8, §4.3 · p. 8, Figure 6 · p. 8

5 Conclusion

The conclusion presents QProtFuse as a way to exploit multi-layer knowledge within a single PLM for protein retrieval and function description. Its results and analyses support adaptive fusion and complementary retrieval objectives on Prot-Inst-OOD.

5 Conclusion

  • QProtFuse achieves the strongest protein and annotation retrieval performance on Prot-Inst-OOD and improves downstream function description.Ablations support query guidance and complementary training objectives, while weight analysis reveals distinct fusion patterns for biological queries.Source: §5 · pp. 8–9
  • The best-performing PLM layers vary across biological tasks, making final-layer selection and naive all-layer averaging suboptimal.The paper concludes that adaptive use of a single PLM’s internal knowledge is effective for retrieval-augmented protein function prediction.Source: §5 · pp. 8–9

Limitations

The authors identify limited evaluation scale and decoupled retrieval-generation as current constraints on QProtFuse.

Limitations

  • Evaluation focuses on Prot-Inst-OOD, and the frozen-LLM design neither refines generated responses nor verifies them against retrieved evidence.The authors identify larger protein resources and evidence-aware generation, response verification, or joint retrieval-generation optimization as directions for further work.Source: Limitations · p. 9

Ethical Considerations

The section flags possible misuse of query-guided fusion in protein research and discloses AI assistance in drafting. It also reports that task-specific single-layer choices generally outperform uniform averaging.

Ethical Considerations

  • Malicious queries could direct the system to identify proteins relevant to harmful biological applications; the authors recommend query screening, access controls, and human review.The recommendation specifically concerns potentially sensitive requests.Source: Ethical Considerations · p. 9
  • The authors used OpenAI Codex for initial manuscript drafts from author-provided outlines, then revised and polished the drafts.The authors take responsibility for the final manuscript, implementation, and reported results.Source: Ethical Considerations · p. 9

A Single-Layer Features versus Uniform Averaging

B Prot-Inst-OOD Dataset

Prot-Inst-OOD repartitions Mol-Instructions into four protein-function tasks and uses sequence-cluster splitting with repeated filtering to reduce annotation leakage.

B Prot-Inst-OOD Dataset

  • Prot-Inst-OOD repartitions Mol-Instructions into four tasks covering molecular roles, biological processes and localization; catalytic activities and reactions; domains and motifs; and general functional descriptions.The dataset statistics table reports sample counts and distinct instruction types, but the supplied passage gives no values.Source: B Prot-Inst-OOD Dataset · p. 12, Table 4 · p. 12
  • Protein sequences are clustered with MMseqs2, then clusters are randomly assigned to training and test sets at an initial 80:20 ratio.Source: B Prot-Inst-OOD Dataset · p. 12
  • To reduce annotation leakage, each test protein is matched to its nearest training protein, and test samples sharing that match’s label are moved to training; filtering is repeated twice.Source: B Prot-Inst-OOD Dataset · p. 12

C Evaluation Metrics

The paper evaluates retrieval with annotation-based ranking metrics and generated descriptions with lexical, biological-entity, and LLM-judge metrics. Generation uses a shared prompt setup, with specified sampling and inference settings.

C Evaluation Metrics

  • Retrieval hits require an exact match to the query protein’s ground-truth bio-entity annotation; Recall@1 and Recall@10 measure hit fractions, while MRR@10 measures the first hit’s rank.Source: C Evaluation Metrics · p. 12
  • All three retrieval metrics are computed per task and macro-averaged across the four tasks.Source: C Evaluation Metrics · p. 12
  • ROUGE-L measures longest-common-subsequence overlap between prediction and reference, and the paper reports its token-level F1 score.The implementation uses rouge-score 0.1.2 without stemming.Source: C Evaluation Metrics · p. 12
  • Entity-BLEU measures overlap in extracted biological entities using standard BLEU with unigram and bigram precision, reported as percentages.Entities are extracted using RAPM’s vocabulary derived from InterPro, EC-GO, and Mol-Instructions metadata.Source: C Evaluation Metrics · p. 12

D Generation Details

  • The generation template uses the same two randomly selected examples per task, truncates example sequences longer than 80 residues, and fills a retrieved-records placeholder with ten samples.RAPM and ProtQueSt use their respective published prompts; Figure 7 marks text included only for RAG in blue.Source: Figure 7 · p. 13, D Generation Details · p. 13

C Evaluation Metrics

  • The LLM judge assesses biological recall, precision, specificity, and plausibility; scores range from 0 to 10, and the evaluation uses 500 randomly sampled predictions per task.The study uses ProtQueSt’s prompt and rubric with Qwen3.8-27B as judge, for 2,000 evaluated samples per method.Source: C Evaluation Metrics · p. 13

D Generation Details

  • Qwen3-8B runs locally with vLLM and DeepSeek-V4-Pro is accessed through its online API; both use temperature 1.0, a 512-token output limit, and disabled thinking.Source: D Generation Details · p. 13

E Training Configuration

QProtFuse training uses cached ESM features and a three-seed, 30-epoch configuration. Baseline training settings differ, and ProtQueSt was independently implemented because its code was unavailable.

E Training Configuration

  • QProtFuse has 5.13M trainable parameters and is trained for 30 epochs with three random seeds; each run takes about 0.36 GPU hours excluding ESM-feature precomputation.All reported results use the last checkpoint.Source: E Training Configuration · p. 13
  • QProtFuse training uses mixed bfloat16 on four NVIDIA H200 GPUs, with a global batch size of 1,024 and AdamW weight decay of 10^-4.The router learning rate is 10^-4 and the remaining modules use 10^-3.Source: E Training Configuration · p. 13
  • For Prot2Text-V2, only the projection MLP and LLM LoRA adapters are trained, using rank 32, scaling factor 64, and dropout 0.1.Training uses up to 24 epochs on 16 NVIDIA H200 GPUs, with an effective batch size of 256.Source: E Training Configuration · p. 13
  • Because ProtQueSt’s code was unavailable for these experiments, the authors independently implemented and trained it following the configuration disclosed in its paper.Source: E Training Configuration · p. 13

F Detailed Generation Results

On Prot-Inst-OOD, QProtFuse has higher judge precision than ProtQueSt in every task with both generators; accompanying tables report per-task generation metrics and judge dimensions.

F Detailed Generation Results

  • QProtFuse achieves higher judge precision than ProtQueSt in every task with both generators.The per-task judge table reports scores on a 0–10 scale and defines precision among its dimensions.Source: F Detailed Generation Results · p. 13, Table 6 · p. 14
  • Table 5 reports per-task generation results for protein function, general function, domain/motif, and catalytic activity.Its metrics are E-BLEU, RG-L, and Judge-F; the caption defines E-BLEU as Entity-BLEU-2 and RG-L as ROUGEL (%).Source: Table 5 · p. 14, F Detailed Generation Results · p. 14
  • Table 6 breaks down per-task judge scores by recall, precision, specificity, and plausibility.Scores use a 0–10 scale.Source: Table 6 · p. 14
  • Figure 8 presents additional examples of query-specific group weights alongside their corresponding queries.Source: Figure 8 · p. 15
Loading 2610.30180v1…