Source-linked AI summary
AgriGPT-VL: Agricultural Vision-Language Understanding Suite
Bo Yang, Yunkui Chen, Lanfei Feng, Yu Zhang, Xiao Xu, Jianyu Zhang, Nueraili Aierken, Runhe Huang, Hongjian Lin, Yibin Ying, Shijian Li
TL;DR
Agricultural applications lack domain-tailored multimodal models, curated corpora, and rigorous evaluation. AgriGPT-VL addresses this with a unified dataset, model, and benchmark suite, and reports strong results on its agricultural vision-language benchmark and an external evaluation.
Problem
Agricultural applications face a shortage of domain-tailored multimodal models and curated resources, while general-purpose MLLMs can struggle with specialized agricultural knowledge and reliable visual support.
Source: §1 · p. 2Method
The suite combines the Agri-3M-VL corpus, an agriculture-specialized model trained through a progressive curriculum, and the AgriBench-VL-4K benchmark.
Source: §1 · p. 2, §2.3 · p. 3, §3.1 · p. 3Results
AgriGPT-VL achieves the best results on all AgriBench-VL-4K metrics and 70.10% accuracy on external AgriBench, versus Gemini-2.5-Pro’s 56.94%.
Source: §4.1 · p. 7, §4.4 · p. 8Takeaways & Limitations
The suite is presented as a compact, reproducible blueprint for specialized multimodal systems in agriculture and beyond.
Source: §5 · pp. 8–9
Abstract
from arXiv · showhide
Despite rapid advances in multimodal large language models, agricultural applications remain constrained by the scarcity of domain-tailored models, curated vision-language corpora, and rigorous evaluation. To address these challenges, we present the AgriGPT-VL Suite, a unified multimodal framework for agriculture. Our contributions are threefold. First, we introduce Agri-3M-VL, the largest vision-language corpus for agriculture to our knowledge, curated by a scalable multi-agent data generator; it comprises 1M image-caption pairs, 2M image-grounded VQA pairs, 50K expert-level VQA instances, and 15K GRPO reinforcement learning samples. Second, we develop AgriGPT-VL, an agriculture-specialized vision-language model trained via a progressive curriculum of textual grounding, multimodal shallow/deep alignment, and GRPO refinement. This method achieves strong multimodal reasoning while preserving text-only capability. Third, we establish AgriBench-VL-4K, a compact yet challenging evaluation suite with open-ended and image-grounded questions, paired with multi-metric evaluation and an LLM-as-a-judge framework. Experiments show that AgriGPT-VL outperforms leading general-purpose VLMs on AgriBench-VL-4K, achieving higher pairwise win rates in the LLM-as-a-judge evaluation. Meanwhile, it remains competitive on the text-only AgriBench-13K with no noticeable degradation of language ability. Ablation studies further confirm consistent gains from our alignment and GRPO refinement stages. We will open source all of the resources to support reproducible research and deployment in low-resource agricultural settings.
1. Introduction
Agricultural applications lack domain-tailored multimodal models, curated corpora, and rigorous evaluation, while general-purpose MLLMs struggle with specialized agricultural knowledge and reliable visual reasoning. The paper introduces a unified dataset, model, and benchmark suite to address these gaps.
1. Introduction
- General-purpose MLLMs lack specialized agricultural knowledge and can produce factual errors and unreliable context-aware support, while existing agricultural models often lack visual reasoning.Prior agricultural efforts were frequently text-only or narrowly scoped.Source: §1 · p. 2
- Agri-3M-VL combines 1M image-caption pairs, 2M VQA pairs, 50K expert VQA instances, and 15K GRPO samples in a corpus described as the largest agricultural vision-language corpus to the authors’ knowledge.The suite also introduces the AgriGPT-VL model and AgriBench-VL-4K evaluation benchmark.Source: §1 · p. 2
- AgriBench-VL-4K includes 2,018 open-ended VQA questions and 1,858 image-grounded single-choice questions.The paper also says its resources will be open-sourced for reproducibility and deployment in low-resource agricultural settings.Source: §1 · pp. 2–3
2. Related Work
Agricultural language models advanced text-based understanding, and newer work introduced multimodal resources, but those efforts have often remained limited in scale or scope. The paper frames a unified dataset, model, and benchmark as a response to this fragmented landscape.
2.1. Text-Only Language Models in Agriculture
- Text-focused agricultural models advanced domain-specific language understanding but lacked visual reasoning, limiting them to tasks that do not require image interpretation.Examples include AgriBERT, AgriLLM, AgroLLM, and AgriGPT.Source: §2.1 · p. 3
2.2. Emergence of Multimodal Agricultural Systems
- Agricultural multimodal datasets and models expanded into VQA and captioning, but resources often remained small, narrowly task-focused, or short on rigorous quality control.Examples include Agri-LLaVA, AgriCLIP, VL-PAW, AgMMU, AgroBench, and AgriEval.Source: §2.2 · p. 3
2.3. The Need for a Unified Vision–Language Ecosystem
- Prior work often developed datasets, models, or benchmarks separately, leaving a gap for a unified ecosystem integrating all three at scale.The proposed suite pairs Agri-3M-VL, AgriGPT-VL, and AgriBench-VL-4K as a cohesive foundation.Source: §2.3 · p. 3
3. AgriGPT-VL
AgriGPT-VL combines a multi-agent data-generation pipeline, progressive text-to-vision-language training, and a held-out evaluation benchmark. The section describes how each component is constructed and how its data and evaluation are organized.
3.1. Agri-3M-VL Dataset
- The Data Generator turns multi-source agricultural images into 1M captions, 2M high-quality VQA pairs, 50K expert-level VQA instances, and 15K GRPO samples through four stages.The stages are caption generation, instruction synthesis, multi-agent refinement, and instruction filtering; the pipeline is presented as transferable beyond agriculture.Source: §3.1 · pp. 3, 5
- Instruction synthesis builds diverse factual and reasoning VQA from image-caption pairs, while multi-agent refinement uses feedback, evaluation, and revision to improve grounding and quality.The refinement protocol checks correctness, clarity, and completeness, with low scores triggering revision; it also includes repeated-answer self-consistency checks.Source: §3.1 · pp. 4–5
- Captioning supplies semantic grounding, instruction synthesis adds reasoning diversity, multi-agent refinement structures feedback, and filtering enforces factual reliability.Together, these stages support model training and are presented as a dataset-construction blueprint for other scientific domains.Source: §3.1 · p. 5
3.2. AgriGPT-VL Model Training
- Training progresses from agricultural text grounding to shallow caption alignment, deep image-QA alignment, and GRPO preference optimization.Shallow alignment freezes the vision encoder and language model while training connector and adapter layers; deep alignment gradually unfreezes them, and GRPO rewards image-text consistency, logic, and verifiable terminology.Source: §3.2 · pp. 5–6
- For GRPO optimization, the method builds 15K samples for reinforcement learning; GRPO rewards image-text consistency, internal logic, and verifiable terminology.Source: §3.2 · p. 6
3.3. AgriBench-VL-4K
- AgriBench-VL-4K contains 2,018 open-ended QA pairs and 1,858 single-choice questions regenerated from held-out images, with cross-question consistency and confusable distractors designed to make evaluation discriminating.Near-duplicate images and text are removed, and agriculture- and computer-science-trained annotators review remaining items; about 7.8% of initial items are rejected.Source: §3.3 · p. 6
4. Results
AgriGPT-VL leads across the reported agricultural multimodal metrics, with gains also reported in text-only evaluation and external and general-capability tests. Ablations distinguish the contributions of alignment and GRPO.
4.1. Comparative Experiment
- On AgriBench-13K, AgriGPT-VL leads across mainstream text metrics, with the paper reporting no sacrificed language ability and stronger standardized agricultural terminology and answer style.The text-only evaluation uses tasks without images.Source: §4.1 · p. 7, Table 1 · p. 7
- AgriGPT-VL achieves the best results on all AgriBench-VL-4K metrics, while blind pairwise comparisons show consistently higher win rates against most strong baselines.The metrics cover single-choice visual reasoning and open-ended answer quality; the pairwise evaluation swaps output positions to reduce order bias.Source: §4.1 · p. 7, Table 2 · p. 7
4.2. Ablation Study
- The ablation attributes lexical and descriptive consistency gains to Shallow Alignment, cross-modal reasoning gains primarily to Deep Alignment, and further factual-faithfulness and robustness gains to GRPO.GRPO's largest reported gain is on the stricter image-level cross-consistency metric, Acc+.Source: §4.2 · p. 8, Table 3 · p. 8
4.3. Generalization Evaluation
- On six public benchmarks, text performance is largely preserved on MMLU and OPENBOOKQA, declines modestly on ARC, and vision-language performance matches or exceeds the base model.The model shows parity on SEED-BENCH and MMBENCH and clear gains on MMMU; the paper reports broad competence across approximately 40K out-of-domain samples.Source: §4.3 · p. 8, Table 4 · p. 8
4.4. External Evaluation on AgriBench
- AgriGPT-VL reaches 70.10% accuracy on AgriBench, versus 56.94% for Gemini-2.5-Pro, on an external benchmark with different data sources and annotation protocols.The benchmark contains approximately 700 single-choice questions and is described as decoupled from the training data and data-generation pipeline.Source: §4.4 · p. 8
5. Conclusion
The paper presents AgriGPT-VL as an agricultural vision-language suite combining data generation, curriculum-based multimodal training, and benchmark evaluation. It describes the framework as a reproducible blueprint for specialized multimodal systems.
5. Conclusion
- AgriGPT-VL unifies large-scale data generation, curriculum-based multimodal training, and benchmark evaluation in an agricultural vision-language understanding suite.The conclusion characterizes the model as demonstrating agronomic reasoning and visual grounding without sacrificing general capabilities.Source: §5 · p. 8
- The paper presents a compact, reproducible framework that provides a practical blueprint for building specialized multimodal systems in agriculture and beyond.Source: §5 · p. 9