BioMed-Agent-RL: A Meta Learning, All You Need for Biomedical Applications

Md Asaduzzaman Jabin; Zihao Wu; Tianming Liu

Source-linked AI summary

BioMed-Agent-RL: A Meta Learning, All You Need for Biomedical Applications

Md Asaduzzaman Jabin, Zihao Wu, Tianming Liu

arXiv:2608.21864v1Aug 22, 2026cs.LGcs.AIcs.CV

TL;DR

Clinical vision-language and agent systems face multimodal misalignment, hallucination, limited grounding, and static orchestration in complex biomedical reasoning. BioMed-Agent-RL combines adaptive model orchestration with DPO, CPO, and curriculum-based GRPO, and its evaluation reports improvements across benchmarks, architectures, datasets, query categories, and complexity levels. The approach reaches a maximum accuracy of 73.03% and a maximum gain of 10.5% over DPO, while remaining less accurate on long longitudinal multimodal cues.

Problem
Clinical vision-language and agent systems have limited multimodal alignment, grounding, adaptability, and generalization for complex biomedical reasoning.
Method
BioMed-Agent-RL uses adaptive language-model orchestration with reasoning and lesion segmentation modules, preference optimization, and curriculum-based GRPO reinforcement learning.
Results
73.03% maximum accuracy and 10.5% maximum gain over DPO were reported, with positive improvements across architectures, datasets, query categories, and complexity levels.
Takeaways & Limitations
The framework supports robust multimodal clinical reasoning and grounding across diverse biomedical tasks and evaluation settings.
Takeaways & Limitations
The agent often has low accuracy on long longitudinal multimodal cues and is planned for persistent memory and uncertainty-aware reward modeling.

Abstract

from arXiv · show

The current progress of Clinical Vision Large Language Models (C-VLLMs) has substantially improved digital diagnostics, still these frameworks often endure lesion noises, modality misalignment, hallucination, and missed contextual grounding in complex clinical cases. Moreover, prevailing agent systems usually depend on static and non-adaptable pipelines and lack the versatility necessary for complex medical reasoning. To resolve these difficulties, we present BioMed-Agent-RL, a unified medical agent that incorporates adaptive orchestration, policy, and reward-based reinforcement learning (RL) models for biomedical applications. To ensure reliability, it invokes clinical context-aware preference optimization (CPO), direct preference optimization (DPO), and group relative policy optimization (GRPO) with dynamic entropy regulation. This pipeline utilizes a multimodal meta-learning approach that operates as a field-specific expert and human judgment synthesizer. The agent adaptively utilizes a set of model-level expertise, such as clinical grounding and reasoner, lesion segmenter, and field-specific synthesizer, across various clinical modalities (e.g., X-ray) by utilizing an iterative and adaptive RL approach. The agent learns to seriously synthesize misleading, conflicting vision cues and trust in inherent reasoning, while specialist advice is faulty. An intensive ablation study is conducted across multiple benchmarks, and the agent significantly outperforms existing state of the art models, such as GPT-5, attaining up to ~73% accuracy (gain of ~5%) over contemporary baselines. As a result, the framework suggests a new standard for building factual, reliable, robust, and expert-like intelligent agent systems for independent clinical reasoning.

I. INTRODUCTION

Biomedical vision-language systems face reasoning, alignment, hallucination, and adaptability challenges across complex clinical tasks. BioMed-Agent-RL addresses these issues through orchestrated multimodal agents and curriculum-based reinforcement learning with preference optimization.

Motivation: Clinical VLLMs and generalist backbones struggle with multimodal misalignment, hallucination, and reasoning across diverse biomedical subfields.The paper motivates adaptive clinical agents for grounding, lesion reasoning, segmentation, and report generation across modalities such as MRI, CT, and X-rays.
Preference optimization: CPO selects preference cues using region-specific lesion noising, while agent pipelines weight cues to favor critical visual embeddings over hallucinated words.The paper also notes that lesion noising can disturb vital visual reasoning and introduce hallucinations.
Proposed framework: BioMed-Agent-RL integrates reasoning, lesion segmentation, weighted DPO, clinical-aware preference optimization, and curriculum-based GRPO.Its design combines model orchestration with reinforcement learning to improve multimodal alignment and clinical reasoning.
Agent collaboration: The agent is designed to combine contradictory recommendations and rely on intrinsic reasoning when specialist advice is faulty.The backbone paradigm and clinical grounding across metrics and benchmarks are described as central to this behavior.

A. Clinical AI Agent

Biomedical clinical agents span many specialties but commonly remain unimodal, text-oriented, unstable, computationally intensive, and insufficiently adaptive. These limitations are associated with multimodal misalignment, factual errors, weak contextual reasoning, and inefficient lesion grounding and segmentation.

Clinical AI Agent: Medical AI agents cover clinical, radiology, pathology, radiotherapy, dermatology, dental, genomics, EHR, and surgical applications.The paper distinguishes unimodal or text-based agents from multimodal systems in these biomedical settings.
Clinical AI Agent: Many existing agents use a static sole backbone without self-evolution, adaptation, or re-correction mechanisms.The paper identifies these architectural properties as recurring limitations of current clinical agents.
Clinical AI Agent: These systems are prone to multimodal misalignment, factual misalignment, false contextual reasoning, and inefficient lesion grounding and segmentation.The passage presents these as consequences of the broader limitations in current medical agent systems.

B. Reinforcement Learning Based Agent

Reinforcement learning supports adaptive and iterative agent behavior, but existing approaches can remain unstable, poorly aligned with human feedback, and rigid in tool orchestration. The paper situates its comparison across multiple reinforcement-learning approaches and five clinical benchmarks.

Reinforcement Learning Based Agent: Reinforcement learning is used in agentic AI for self-adaptation, iteration, and self-evolving behavior.The paper lists DPO, PPO, CPO, and GRPO among RL concepts applied to medical tasks such as report generation and self-reasoning.
Reinforcement Learning Based Agent: Existing RL-based medical agents can suffer inaccurate human-feedback alignment, instability, and mismatched cooperation between expert tools.They may also rely on a generalist backbone and in-adaptive orchestration of language-model tools.
Reinforcement Learning Based Agent: The paper compares reinforcement-learning approaches across five clinical benchmarks.Table I is identified as the performance comparison of RL approaches across those benchmarks.

III. METHODS

BioMed-Agent-RL combines multimodal clinical agents with preference optimization, group-relative reinforcement learning, curriculum design, and entropy-regulated adaptive aggregation. The pipeline routes queries to domain experts, incorporates specialist and human judgments, and produces a final clinical output.

Multimodal Clinical Processing: Multimodal clinical inputs are processed by agents that generate clinical reasoning and grounding responses from visual and textual cues.The VLLM component iteratively predicts token distributions for multimodal input.
Adaptive Agent Collaboration: The system routes each task to a domain expert, obtains specialist outputs, and passes them to an aggregator for final output selection.Task captioning ranks task categories, selects an expert, and forwards the resulting response to the aggregation agent.
Preference Optimization: CPO and weighted DPO assign diagnostic-relevance weights to preference data while normalizing and trimming clinical relevance scores.The weighted loss uses normalized scores with lower and upper bounds α and β.
Group-Relative Reinforcement Learning: GRPO updates the policy using within-group relative rewards rather than a critic function and applies a clipped objective with KL regularization.Each multimodal query produces multiple responses whose rewards determine normalized advantage values.
Curriculum and Entropy Regulation: C-GRPO divides examples into easy, medium, and hard curriculum levels according to expert accuracy and regulates entropy to balance exploitation with exploration.Hard cues receive the strongest entropy coefficient, encouraging exploration when expert suggestions are incorrect.
Reward Design: The framework combines template and accuracy rewards to promote precise specialty assignment and systematic semantic reasoning.Format rewards range from 0.0 to 0.50, while accuracy rewards range from 0.0 to 1.0.

A. Datasets and Baselines

The evaluation covers two agent architectures and five medical datasets spanning visual question answering and report generation. Comparisons include LLaVA-Med, DeepSeek-VL, and GPT-5 across diverse multimodal query types.

Agent Architectures: BioMed-Agent-RL is evaluated with a single backbone and a domain-specific collection of VLLM experts.The study compares both architectural setups.
Datasets: The benchmark uses RAD-VQA and SLAKE for VQA, plus Path-VQA, MIMIC-CXR, and IU-Xray for report generation.These five datasets form the two evaluation categories.
Baselines and Query Types: Baselines include LLaVA-Med, DeepSeek-VL, and GPT-5, with queries covering closed-ended, open-ended, grounding-based, and attribution-based tasks.The passage also identifies summarization-oriented queries within the evaluation design.

B. Implementations

The implementation uses MMedAgent with a structured prompt format and trains the curriculum-based GRPO agent under specified rollout, optimization, hardware, and entropy settings. The resulting training process requires 10–12 days on eight Nvidia A5000 GPUs.

Backbone and Prompting: MMedAgent serves as the backbone, while prompts separate reasoning, lesion grounding, and answers using <thinking> and <gpt_answer> tags.The prompt blueprint is adapted to VQA and report-generation query categories.
Training Algorithm: C-GRPO is implemented to improve robustness, grounding accuracy, and resistance to hallucinated results.The algorithm is integrated into the agentic system's training workflow.
Hyperparameters: Training uses batch size 256, eight rollouts, temperature 9.0, learning rate 1x10^-5, and three specialists.KL curriculum coefficients are 3x10^-3, 5x10^-5, and 7x10^-7 across curriculum settings.
Compute: 10–12 days of training and inference are performed on Nvidia 8xA5000 hardware with 192 GB VRAM.Dynamic entropy coefficients are 0.03, 0.003, and 0.0003 for hard, medium, and easy cues, respectively.

C. Ablation Studies

The ablation studies evaluate C-GRPO across backbones, benchmarks, query categories, complexity levels, and qualitative clinical outputs. Results show consistent aggregate gains, while performance declines as query difficulty increases.

Backbone and benchmark ablations: C-GRPO achieves higher average F1 than CPO and DPO across four backbones and five clinical benchmarks.Mean F1 ranges from 49.95% for LLaVA-Med to 73.08% for C-GRPO using MMedAgent checkpoints, with 95% confidence intervals across benchmarks.
Backbone and benchmark ablations: 1% to 10.5% ∆F1 gains are reported for C-GRPO across all five biomedical benchmarks compared with CPO and DPO.RAD-VQA improves from 2.9% to approximately 4% during iterations, while later benchmarks show gains of approximately 5%, 4.8%, and 6.5%.
Query-category analysis: 68% to 73.3% F1 is maintained across most query categories, while open-ended queries dip to 59%.The study uses 6,000 total data points and 1,200 multimodal instances per category.
Query-complexity analysis: 73.08% accuracy on easy cues declines to approximately 58% on hard cues as clinical grounding and reasoning difficulty increases.The complexity analysis considers 2,000 clinical instances per complexity level.
Qualitative comparison: BioMed-Agent-RL produces more specific clinical keywords and rectangular lesion grounding than DPO and CPO in qualitative VQA comparisons.The example includes terms such as “patchy opacity,” “right lower lobe,” “pleural effusion,” and “acute pneumonia.”

V. FUTURE DIRECTION

Future work targets longitudinal multimodal reasoning and reward reliability. The authors propose persistent memory with adaptive retrieval and uncertainty-aware reward modeling for GRPO.

Longitudinal multimodal cues: Low accuracy on long longitudinal multimodal cues motivates persistent memory augmentation with adaptive retrieval.The planned module is intended to address episodic cues.
Reward reliability: Uncertainty-aware reward modeling and policy functions are planned to improve agent reliability during GRPO curriculum learning.This is identified as a second planned direction for the agentic system.

VI. CONCLUSION

The paper presents C-GRPO-based BioMed-Agent-RL for complex biomedical VQA and report generation. Across diverse benchmarks and evaluation settings, it reports strong accuracy, generalization, and reasoning consistency, while related work frames the broader clinical-agent context.

Conclusion: C-GRPO-based BioMed-Agent-RL targets complex biomedical VQA and report generation across five semantic query categories, multiple benchmarks, and three complexity levels.The conclusion describes evaluation across diverse clinical tasks and cue complexities.
Conclusion: 73.03% maximum accuracy represents a maximum gain of 10.5% over the closest RL approach, DPO.The reported improvements are described as reliably surpassing DPO and CPO optimization approaches.
Conclusion: Performance gains are reported as architecture- and algorithm-agnostic, with steadily positive enhancements across diverse clinical tasks.The conclusion also reports generalization to uncommon and highly complex clinical cues.
Conclusion: C-GRPO is described as improving reasoning consistency under multi-stage and structural reasoning conditions and reducing error distribution through reward modeling.This conclusion is stated within the scope of the evaluated clinical AI system.