SSE-Bio: A Structured Self-Evolving Agent with Agentic Retrieval Policy for Multi-Hop Biomedical Reasoning

• Biomedical multi-hop QA requires linking entities through specialized relations, where unresolved bridge entities can derail reasoning or leave multi-answer results incomplete.The task supports evidence-grounded biomedical discovery, including disease understanding and drug repurposing (Xiang et al., 2025).Source: §1 · p. 1
• Existing agentic systems may use static retrieval workflows and coarse-grained prompt rewriting, motivating explicit retrieval control and fine-grained memory evolution.Without verification and feedback, retrieval-augmented systems may leave bridge entities unresolved and return incomplete answers in multi-answer settings.Source: §1 · p. 1
• The Proxy is initialized with supervised retrieval labels, then refined using GRPO, decision-contrastive branches, and rewards for answer correctness and evidence-supported reasoning.Alternative retrieval branches are compared under the same structured reasoning state.Source: §1 · p. 2
• SSE-Bio consistently outperforms strong baselines on BioHopR, MedHop, and Humanity’s Last Exam: Biomedicine.Source: §1 · p. 2
• SSE-Bio unifies short-term structured state tracking, long-term template memory, and explicit retrieval management in one agentic framework.The Proxy decides whether to retrieve knowledge triplets and/or prior templates at each reasoning step.Source: §1 · p. 2
• SSE-Bio evolves memory through local edits to templates from successful trajectories rather than rewriting whole prompts, supporting more auditable updates.This is designed to mitigate hallucination and instruction-drift risks from unconstrained template rewriting.Source: §1 · p. 2

Biomedical RAG Models

• Biomedical RAG draws on external medical evidence and graph-based or agentic retrieval, but cited approaches lack explicit verification and feedback over evolving reasoning paths.Examples include i-MedRAG (Xiong et al., 2024), MedGraphRAG, KRAGEN, and AMG-RAG (Rezaei et al., 2025); the broader direction is described by Tran et al. (2025) and Fang et al. (2025).Source: §2 · p. 2

Biomedical Multi-agent Systems

• Biomedical multi-agent systems use specialized collaboration, knowledge-graph agents, and broader tool use to support complex reasoning.Examples are MDAgents (Kim et al., 2024), KGARevion (Su et al., 2024), and Biomni (Huang et al., 2025a).Source: §2 · p. 3
• STELLA (Jin et al., 2025) adapts through evolving templates and tools, while SSE-Bio adds state-aware retrieval and bounded, fine-grained memory editing.STELLA offers limited control over when and what evidence is retrieved, and rewrites templates from successful cases; SSE-Bio couples retrieval control with structured long-term memory updated by local edits.Source: §2 · p. 3

3.1 Agentic Formulation

• A Manager, Proxy, Dev agent, and Critic iteratively plan and execute reasoning, assess evidence-grounded answers, and refine the state or long-term memory.The Proxy selects retrieval actions from the current state; the Critic evaluates trajectories without access to the gold answer.Source: §3.1 · p. 3

3.2 Structured Memory

• Structured memory tracks each question’s task type, information gap, feedback, and retrieved evidence, while reusable templates guide plans and are locally edited or newly distilled.Templates encode reasoning flow, verification criteria, tool-use policy, and failure warnings; editing only fields needing refinement constrains broad changes that can cause instruction drift.Source: §3.2 · p. 4

3.3 Agentic Retrieval Policy

• The Proxy chooses among four retrieval actions—neither source, knowledge triplets, templates, or both—to provide step-specific facts and reusable reasoning priors.Triplets are queried using the state’s information gap; templates are queried using its task type, with BiCA retrieval and filtering applied to both sources.Source: §3.3 · p. 4

3.4 Training of the Proxy Policy

• The Proxy is SSE-Bio’s only trainable component, initialized with SFT pseudo-labels from the retrieval action whose rollout earns the highest reward (Wei et al., 2022; Huang et al., 2025b).The Manager, Dev, Critic, retrievers, and reasoning environment remain fixed.Source: §3.4 · p. 4
• SSE-Bio expands alternative retrieval actions from the same reasoning state into decision-contrastive rollout groups; early branches violating schema, leaving gaps unresolved, or producing unsupported bridge entities are pruned, and top-K remain by intermediate answer-grounded reward.GRPO is applied at the action level because the Proxy outputs discrete retrieval actions, comparing the current policy with the old sampling policy.Source: §3.4 · pp. 4–5

Datasets

• BioHopR contains 7.63K paired instances split 7:3 into approximately 5.34K training and 2.29K test pairs, with linked 1-hop and 2-hop questions and single- or multi-answer evaluation.SSE-Bio is trained on BioHopR and transferred directly to MedHop and HLE for zero-shot evaluation.Source: §4 · pp. 5–6

Baselines

• Baselines cover general-purpose and medical LLMs, retrieval-augmented generation, agent-based systems, and self-evolving agents; agentic baselines share SSE-Bio’s backbone configuration.Gemini-2.5-Pro is used for Manager/Critic roles and Claude-4.5-Sonnet for execution.Source: §4 · p. 5

Evaluation

• BioHopR evaluation uses PrecH1, PrecH2, Bothcor, and Bothwr with embedding-based precision and cosine-similarity matching; higher is better for the first three metrics, while lower is better for Bothwr.Source: §4 · pp. 5–6
• MedHop and HLE are evaluated using zero-shot test accuracy after direct transfer from BioHopR.Source: §4 · p. 6

Implementation details

• Gemini-2.5-Pro is fixed for Manager and Critic roles and Claude-4.5-Sonnet for the Dev agent; evaluation freezes template memory and excludes test and transfer-benchmark instances from template creation or editing.Source: §4 · p. 6

5.1 RQ1: How does SSE-Bio compare with baselines?

• SSE-Bio significantly beats STELLA on all four single-answer BioHopR metrics and ranks first across all four metrics in the multi-answer setting.Single-answer PrecH1 rises from 38.94 to 47.21, PrecH2 from 14.89 to 21.38, and Bothcor from 9.96 to 16.52, while Bothwr falls from 56.13 to 47.93. In multi-answer, it reaches 38.36 on PrecH1, 16.24 on PrecH2, and 11.73 on Bothcor.Source: §5.1 · pp. 6–7

5.2 RQ2: How does the Proxy affect SSE-Bio?

• 16.52 to 10.62 single-answer Bothcor without the Proxy shows retrieval control matters; the GRPO-trained Proxy also improves over SFT.With the Qwen-72B Proxy, GRPO raises Bothcor from 15.08 to 16.52 and lowers Bothwr from 49.48 to 47.93; the learned Proxy also beats fixed retrieval schedules under the same budget.Source: §5.2 · p. 7, Table 2 · p. 7

5.3 RQ3: What are the effects of structured self-evolution?

• 16.52 to 11.72 Bothcor when replacing structured state and fine-grained editing with free-form state and coarse rewriting shows the full design performs best.Structured state alone and fine-grained editing alone recover part of the gap, with Bothcor of 13.86 and 14.52, respectively. Fine-grained editing also changes fewer unrelated fields and introduces fewer unsupported or contradictory instructions than coarse rewriting.Source: §5.3 · pp. 7–8, Table 2 · p. 7

5.5 RQ5: How does SSE-Bio perform across different relation types?

• 22.83 to 34.26 single-answer PrecH2 for Disease:Drug:Phenotype shows stronger gains on a relation requiring second-hop reasoning.For that relation, multi-answer PrecH2 rises from 10.58 to 16.82; Drug:Disease:Protein has smaller increases, from 11.94 to 13.18 single-answer and 3.89 to 4.52 multi-answer PrecH2.Source: Table 3 · p. 8, §5.5 · p. 8

5.4 RQ4: How much headroom remains under stronger retrieval?

• 18.06 single-answer Bothcor with oracle triplets and templates, versus 16.52 for SSE-Bio, shows headroom under stronger retrieval.Removing triplets or prior templates lowers Bothcor to 13.64 or 14.26, respectively. In multi-answer, combined oracle retrieval raises Bothcor from 11.73 to 13.05 and lowers Bothwr from 57.13 to 55.78; Appendix A.12 evaluates robustness to injected noisy triplets.Source: Table 2 · p. 7, §5.4 · p. 8

6 Conclusions

• SSE-Bio combines state-aware retrieval through a trainable proxy with structured templates and fine-grained editing, avoiding fixed retrieval pipelines and unconstrained self-rewriting.Source: §6 · p. 9
• 6.56 points on single-answer Bothcor over STELLA on BioHopR; across three benchmarks SSE-Bio beats strong baselines and leads multi-answer settings.Source: §6 · p. 9
• Ablation and relation-type analyses indicate that the gains are supported by effective retrieval control and structured self-evolution.Source: §6 · p. 9

Limitations

• Stronger biomedical evidence retrieval and template matching could improve downstream performance.The work identifies more advanced retrieval and matching strategies as future work.Source: Limitations · p. 9
• Long-term template memory may accumulate redundant or outdated templates despite fine-grained editing.The authors propose memory consolidation, pruning, and provenance tracking as future directions.Source: Limitations · p. 9
• How far larger proxy backbones can improve retrieval control remains an open question.The proxy analysis reports that stronger proxy backbones consistently improve retrieval control.Source: Limitations · p. 9
• Training only the Proxy alongside fixed commercial agents may affect reproduction and leaves cross-backbone policy transfer untested.SFT labels and GRPO rewards come from rollouts within the same fixed agent environment.Source: Limitations · p. 9

A.1 Computing Hardware

• Experiments used a compute node equipped with six NVIDIA GH200 Grace Hopper superchips.The hardware provided sufficient memory and computational capacity for the training and inference settings used.Source: §A.1 · p. 12

A.2 Examples of Structured Memory

• Table 4 defines fields for short-term structured state and long-term template memory, which support retrieval control and replanning and store reusable reasoning priors for future questions.Figures 3 and 4 illustrate a structured state and a fine-grained template update.Source: Table 4 · p. 12, §A.2 · p. 12
• Figure 3’s short-term state organizes task type, information gap, Critic feedback, and retrieval state to guide retrieval control and next-step planning.Source: Figure 3 · p. 12, §A.2 · p. 12
• Figure 4’s long-term template update keeps task type fixed while locally editing reasoning flow, verification criteria, tool-use policy, and failure warning after a successful trajectory.Other template fields remain unchanged; the figure marks the original implicated field in grey and its update in blue.Source: Figure 4 · p. 12, §A.2 · p. 12

A.3 Prompt and API Sensitivity

• The prompt analysis compares Empty, Basic, and role-specific system prompts for the Manager, Dev, and Critic agents.User prompts, schemas, retrieval action space, template format, stopping criterion, answer format, model APIs, and trained Proxy remain fixed.Source: §A.3 · pp. 12–13
• Prompt changes cause only small drops, while alternative Manager/Critic APIs cause moderate but not catastrophic degradation.Together, the analyses indicate reasonable stability under both variations.Source: §A.3 · p. 13
• The API analysis replaces the Manager and Critic backbones while keeping the Dev agent, trained Proxy, retriever, template memory, prompts, and evaluation protocol fixed.Source: §A.3 · p. 13

A.5 Implementation Details

• SSE-Bio uses a four-agent architecture with a trainable Proxy and fixed commercial Manager, Dev, and Critic agents, and is intended for research rather than clinical use.The Manager and Critic use Gemini-2.5-Pro, the Dev agent uses Claude-4.5-Sonnet, and the Proxy uses open-weight backbones; the appendix warns against clinical diagnosis, treatment recommendation, or patient-facing deployment.Source: §A.5 · p. 13

A.4 Efficiency Details

• 16.52 Bothcor for SSE-Bio exceeds Biomni’s 10.45 and STELLA’s 9.96 on BioHopR, while its 28.3M tokens exceed their 23.6M and 21.2M, respectively.Per-example inference adds 3.1K tokens, 1.8 calls, 5.4s, and approximately $0.012 relative to STELLA; Proxy training costs 138 GPU-hours once and is not incurred at inference.Source: §A.4 · p. 13, Table 7 · p. 14, Table 8 · p. 14

A.6 Agentic Retrieval

• For retrieval, SSE-Bio ranks biomedical triplets using the current information gap and retrieves up to five, while selecting one compatible template using task type and schema checks.The template threshold is 0.75; successful trajectories are aligned only when task type matches and similarity meets the threshold, otherwise a new template is distilled, with edits limited to Critic-identified fields. Training-rollout analyses found five triplets and one template performed best.Source: §A.6 · pp. 14–15

A.7 Proxy Policy Training

• The Proxy’s GRPO reward prioritizes final answer correctness and uses evidence support as a tie-breaker, while SFT pseudo-labels select the best retrieval action and GRPO retains all group branches.For BioHopR, outcome reward is one only when both linked answers meet the benchmark criterion; the behavioral term measures the fraction of required reasoning steps explicitly supported by selected triplets or an accepted template step.Source: §A.7 · p. 15

A.8 Datasets; A.9 Evaluation; A.10 Baselines

• Evaluation spans BioHopR, MedHop, and HLE: Biomedicine; BioHopR has 7.63K paired instances, and test and other benchmark data are excluded from Proxy tuning and template construction.BioHopR reports PrecH1, PrecH2, Bothcor, and Bothwr; MedHop and HLE: Biomedicine use accuracy. Baselines include general and medical LLMs, RAG systems, biomedical agents, and self-evolving agents, with Biomni and STELLA the strongest agent and self-evolving baselines, respectively.Source: §A.8 · p. 15, §A.9 · pp. 15–16, §A.10 · p. 16

A.11 Proxy Training and Memory Construction Statistics

• Memory construction begins with an empty template memory and uses validated successful BioHopR training trajectories, with accepted updates mostly refining aligned templates through local field-level edits.The final memory remains compact and covers BioHopR task-type variants; SFT labels span all four retrieval operations, while GRPO uses decision-contrastive trajectory groups rather than a fixed retrieval strategy.Source: Table 11 · p. 16, Table 13 · p. 17, §A.11 · p. 18

A.12 Robustness to Noisy Retrieval; A.13 Additional Analysis of Retrieval Control and Memory Evolution

• Adding two irrelevant triplets reduces single-/multi-answer Bothcor by 0.50/0.42 points; always retrieving both sources yields lower Bothcor than the learned Proxy, which selects sources by state.Single-answer Bothcor falls from 16.52 to 13.18 and multi-answer Bothcor from 11.73 to 9.02 under always-both retrieval. Fine-grained editing also changes fewer fields (1.46 vs. 3.28), preserves unchanged fields more often (88.0% vs. 64.0%), and lowers unsupported and contradictory instruction rates (4.0% vs. 12.0%; 2.0% vs. 7.0%).Source: §A.12 · p. 18, §A.13 · pp. 18–19

A.14 Error Analysis; A.15 Case Study: Failure Analysis against STELLA; Item STELLA SSE-Bio

• 31.2% of failed HLE cases stem from Dev reasoning errors and 24.6% from triplet retrieval failures; a BioHopR example contrasts STELLA’s unsupported AR bridge with SSE-Bio’s verified FOLH1 path.The example requires both the drug-to-FOLH1 and FOLH1-to-disease links; the appendix notes that better retrieval alone will not address remaining errors.Source: §A.14 · p. 19, Table 18 · p. 20, Item STELLA SSE-Bio · p. 20