PaperlayerView exact revision on arXiv

Source-linked AI summary

PhenoIntel: A Lifecycle-Aligned Multi-Agent Web Application for Verified, Accessible Plant Phenotype Analysis

Narendren S, Soumyashree Kar

arXiv:2608.27999v1cs.AI

TL;DR

Plant-phenotyping platforms lack reliable verification, uncertainty reporting, assumption checking, and accessible workflows. PhenoIntel addresses these gaps with a lifecycle-aligned multi-agent web application; across ten checkpoints, held-out Macro F1 spans 0.78–0.996 and detectors reach mAP50 = 0.96.

  • Problem

    Existing plant-phenotyping pipelines lack verification-first orchestration, architecture-aware uncertainty quantification, and extensible, auditable coverage across crops, traits, and models.

  • Method

    PhenoIntel assigns nine specialised agents to separately checked lifecycle stages that share a typed record, with uncertainty matched to model behaviour and self-extending model support.

  • Results

    Across ten checkpoints, performance was generally strong, spanning held-out Macro F1 of 0.78–0.996, while detectors reached mAP50 = 0.96.

  • Takeaways & Limitations

    PhenoIntel provides browser-based, GPU-free phenotyping with independently auditable outputs, FAIR-compliant provenance, and bounded failure modes.

  • Takeaways & Limitations

    Evaluation is limited by single-run or unconfirmed-split results, and the 1,200-test suite does not provide a live-session system-level ablation.

Abstract

from arXiv · show

Existing conversational plant-phenotyping platforms are difficult for plant scientists to use and lack the reliability scientific research demands: failed analyses are reported as valid measurements rather than flagged as missing, statistical tests run without checking assumptions, predictions carry no uncertainty estimate, and specialised hardware limits accessibility. We present PhenoIntel, a lifecycle-aligned multi-agent web platform that turns the full machine-learning workflow into a reliable, user-friendly phenotyping system. Nine specialised agents divide the analysis into stages, from image collection through model selection, inference, and reporting, rather than handing the whole task to one AI manager. Independent checks separate these stages, and every agent reads from and writes to one shared, fixed-structure record, so an inconsistent output from one stage is caught before it reaches the next. Uncertainty is matched to each model family, conformal prediction, detection-confidence spread, or Monte Carlo Dropout, rather than applied uniformly, and quality thresholds adapt to crop and task instead of one global cutoff. When no suitable model exists, PhenoIntel can propose, validate, and integrate a new one on its own. The model repository spans ten trained models across five crops and four imaging modalities. Classification models reach Macro F1 of 0.78-0.996; object-detection models reach 0.96 mAP@50 with a 54% reduction in counting error over an unoptimised baseline; and a temporal model reaches held-out Macro F1 of 0.7050. PhenoIntel runs in a browser on standard hardware, requiring no GPU, and a 1,200-test automated suite confirms complete pipeline execution. Every result carries calibrated uncertainty, validated statistics, and FAIR-compliant provenance, a combination existing conversational phenotyping tools do not offer.

1 Introduction

The introduction frames reliable high-throughput plant phenotyping as an agricultural bottleneck requiring confidence intervals, plausibility checks, and traceable provenance. It diagnoses monolithic orchestration as the structural source of unchecked failures and motivates PhenoIntel’s lifecycle-aligned, independently verified multi-agent architecture.

  • Motivation: High-throughput plant phenotyping converts images into quantitative traits used in breeding, diagnosis, and yield forecasting.Examples include leaf shape, disease severity, organ counts, canopy structure, and growth rate.
  • Motivation: Confidence intervals, plausibility checks, and traceable provenance records are presented as core requirements for reliable phenotyping.
  • Related work: Language-model manager systems determine step by step which tool to invoke next, including PhenoAssistant’s GPT-4o-based phenotyping manager.This orchestration approach has also been adapted to domains such as chemistry and genomics.
  • Problem diagnosis: A recurring failure pattern is that one inference step performs detection, segmentation, and measurement without catching failures before reporting.Consequently, a missing detection can become indistinguishable from a genuine zero.
  • Contributions: PhenoIntel replaces a single monolithic orchestrator with nine specialised agents separated by independent verification checkpoints.The paper evaluates this architecture across ten trained models and six cross-domain case studies, alongside an audit of a prior monolithic phenotyping orchestrator.

2 Related Work

Prior plant-phenotyping tools and AI systems provide useful image analysis, modelling, uncertainty methods, and agent orchestration, but generally lack integrated verification across the full workflow. PhenoIntel addresses these gaps through lifecycle-aligned stages, model-aware uncertainty, adaptive quality control, extensible coverage, and auditable reporting.

  • Conventional and deep-learning phenotyping: Established tools use manually configured thresholding, filtering, and scripted image-analysis pipelines that work within specific crop, device, and plant-condition contexts.PlantCV, FIJI/ImageJ, Canopeo, and RootNav exemplify this non-agentic approach.
  • Conventional and deep-learning phenotyping: Deep-learning phenotyping applies specialised vision architectures, including DINOv2, YOLOv8, Mask2Former, and Segment Anything, without addressing end-to-end model orchestration.These systems cover agricultural classification, structural detection, and segmentation tasks.
  • Uncertainty quantification: Uncertainty quantification is established in related scientific domains but remains rarely integrated into plant-phenomics pipelines and is often focused on probabilistic semantic segmentation.Existing plant-phenotyping work uses Bayesian CNNs, Monte Carlo Dropout, and variational inference for crop-weed segmentation and uncertainty-aware decisions.
  • Conversational and multi-agent systems: Existing tool-use and multi-agent systems coordinate specialised capabilities, but published plant-health systems can rely on message exchange without a shared state object and expose communication-reliability concerns.Chat Demeter divides rice-disease diagnosis across four role-specific agents and reports over 99% classification accuracy on a rice-disease benchmark.
  • Research gaps and PhenoIntel objectives: The literature identifies three gaps: unchecked failure propagation, uncertainty methods not matched to model behaviour, and limited extensible, auditable coverage.These gaps motivate automatic checks between lifecycle stages, architecture-aware uncertainty, adaptive per-(crop, task, modality, metric) thresholds, and self-extending model coverage.

3 Data Description and Curation

PhenoIntel curates datasets through a uniform, leakage-aware pipeline spanning eight crop–task–modality combinations. Curation combines duplicate removal, stratified splitting, label review, and imbalance handling before multi-task evaluation.

  • Dataset Coverage: Eight crop–task–modality combinations comprise the model zoo, including classification and detection datasets across rice, wheat, maize, banana, and coffee.The listed datasets use RGB close-range or UAV aerial imagery and include approximately 800–17,000 images per classification dataset.
  • Curation Pipeline: Every dataset undergoes exact-checksum and perceptual near-duplicate removal before a stratified 70/15/15 train/validation/test split.A leakage check confirms that duplicate or near-duplicate images do not cross dataset splits.
  • Label and Class Quality: Label-quality screening flags feature-space nearest-neighbour disagreements for human review rather than automatically removing potentially genuine hard examples.The same curation process addresses class imbalance with weighted cross-entropy and targeted augmentation.
  • Evaluation Pipeline: The uniform evaluation pipeline runs from raw data sources through preprocessing, augmentation/TTA, architecture selection, training, calibration, uncertainty quantification, metrics, and packaged output.It is applied across classification, detection, satellite time-series, and trait-regression model families.

4 Methodology

PhenoIntel structures plant-phenotyping analysis as independently checked lifecycle stages coordinated through typed records, producing verified measurements with confidence intervals, plausibility flags, and provenance. Its model-selection and self-extension mechanisms provide fallbacks for unfamiliar crop, task, or imaging combinations while keeping acceptance decisions behind explicit validation gates.

  • Lifecycle-aligned pipeline: PhenoIntel separates the workflow into nine lifecycle-aligned agents with independent verification between stages and a shared fixed-structure record.The system returns a checked trait measurement, confidence interval, plausibility flag, and full production record.
  • Lifecycle-aligned pipeline: The Planning Agent converts a plain-language request into a typed session plan, while the other eight agents consume its fields instead of parsing text.The plan is checked against a fixed template at the first quality checkpoint; a rule-based keyword matcher is the fallback when no AI model is available.
  • Model selection: Model selection uses crop, task, and imaging type from a structured registry, applying exact-match, cross-crop, degraded-model, and zero-shot fallbacks.The same step performs a lightweight out-of-distribution check using the selected model’s training images, while term matching provides an offline vocabulary fallback.
  • Self-extension: Unknown coverage is handled by Task Ontology and Pipeline Adaptation agents that classify requests, apply deterministic overrides, or generate candidate pipelines subject to schema and consistency checks.Candidate vision models can come from the built-in zoo, timm, Hugging Face Hub, torchvision, or a fully custom architecture, all compiled and verified before registration.
  • Verification and quality control: Quality thresholds are set per crop, task, modality, and metric from domain facts, then cached and applied through plain arithmetic checks with safe defaults for malformed responses.Generated pipelines enter the system vocabulary only after validation across several independent sessions, and language-model proposals do not decide their own acceptance.

5 Experiments and Evaluation

PhenoIntel’s evaluation spans model-specific experiments and six cross-domain case studies, assessing not only predictive performance but also completion, statistical validity, reproducibility, provenance, and accessibility. The protocol defines task-specific metrics and uncertainty measures before reporting results.

  • Evaluation design: Six cross-domain case studies exercised prediction, zero-shot fallback, and new-model training entirely through the web application, without command-line tools or environment configuration.The cases covered modalities, crops, and tasks absent from the primary training corpus, with pass/fail criteria and outcomes reported together.
  • Evaluation criteria: Task completion required a final phenotype output and scientifically valid statistical report without crashes, timeouts, silent zero-value failures, or unhandled exceptions.Unhandled-exception and crash rate were tracked as the complementary failure mode.
  • Reproducibility and provenance: A 1,200-test automated suite independently checked internal consistency, while provenance evaluation counted twelve always-present FAIR-related files and session resumability.The suite complemented rather than substituted for predictive-accuracy evaluation.
  • Evaluation criteria: Statistical validity required Shapiro–Wilk testing before parametric tests, Kruskal–Wallis + Dunn fallback, effect sizes with p-values, and Holm–Bonferroni correction.These requirements were structural properties of every session’s APA-style statistical report rather than optional checks.
  • Accessibility: Accessibility was measured by cold-start time-to-first-result, setup-step count, and local-GPU requirement using the same protocol for PhenoIntel and the prior system.This enabled a like-for-like accessibility comparison.
  • Metrics and uncertainty: Classification used Macro F1 and calibration metrics, detection used mAP50 and counting MAE, regression used MAE, RMSE, and Monte Carlo Dropout coverage, and segmentation used leaf-count MAE and RMSE.Uncertainty evaluation was matched to task family: ECE and conformal coverage for classification, detection-confidence variance for detection and segmentation, and P10–P90 coverage for regression.

6 Results · 6.1 Case Study Outcomes and Evaluation Framework

Six provenance-logged, end-to-end case studies evaluate PhenoIntel as an integrated web application across exact-match, fallback, Build, recovery, and temporal workflows. The results show completed analyses, registered models, adaptive warnings, and successful architecture adaptation, while evaluation covers completion, accuracy, calibration, reproducibility, FAIR governance, and internal validation.

  • 6.1 Case Study Outcomes and Evaluation Framework: Six provenance-logged case studies run end-to-end through the web application across exact-match, zero-shot fallback, Build, recovery, and temporal workflows.Together they are presented as evidence that PhenoIntel operates as an integrated system rather than isolated components.
  • 6.1 Case Study Outcomes and Evaluation Framework: Evaluation spans task completion, factual accuracy, calibration, reproducibility, FAIR governance, and internal validation, with the prior system included where like-for-like comparison is possible.
  • 6.1 Case Study Outcomes and Evaluation Framework: mAP50 = 0.9418 and MAE = 1.10 were achieved in CS1 across 10 images, with mean uncertainty ±7 spikes and every checkpoint passing.Two minor non-blocking issues—missing calibration data and unavailable literature lookup—did not affect detection.
  • 6.1 Case Study Outcomes and Evaluation Framework: MAE = 3.00 followed a threshold fix in CS2’s zero-shot Arabidopsis fallback, which correctly tracked true leaf count on four of five images.Plausibility warnings were expected for thresholds calibrated to field crops and were downgraded rather than failing the session.
  • 6.1 Case Study Outcomes and Evaluation Framework: Macro F1 = 0.9555 was reached by CS3’s classification model across 375 images, while approximately 10% were flagged for possible mislabelling and routed to expert review.The run had no leakage or duplicates, and the model was registered to the zoo.
  • 6.1 Case Study Outcomes and Evaluation Framework: mAP50 = 0.689 and boll-counting error 0.667 were obtained on CS4’s internal split, while a separate full-pool check reached mAP50 = 0.9049.The results used non-nested sets of very different sizes and were therefore reported separately.
  • 6.1 Case Study Outcomes and Evaluation Framework: 82% held-out coverage against an 80% target was achieved after CS5 generated and validated a single-stream architecture following three failed inference attempts.Placeholder outputs reflected a missing output mapping, a registration gap separate from recovery failure.
  • 6.1 Case Study Outcomes and Evaluation Framework: Macro F1 = 0.973 was reached by CS6’s temporal model across 800 time-series patches with no leakage, duplicates, or anomalies.The case confirms temporal-backbone routing and clean training and verification rather than generalisation benchmarking.

6.2 Pipeline Completion Rate … 6.6 Statistical Output Validity

Across five consecutive evaluation sections, PhenoIntel completed every tested workflow, produced strong but reliability-tiered phenotype results, attached model-specific uncertainty, and structurally enforced scientifically valid statistical reporting.

  • 6.2 Pipeline Completion Rate: All six live case studies and ten model-zoo checkpoints completed without crashes, timeouts, or silent zero-value failures across four analysis pathways.Completion measures reliability rather than accuracy; CS2’s zero-shot fallback always finishes despite weak accuracy.
  • 6.3 Phenotype Accuracy: 0.9649 and 0.9963 are the held-out Macro F1 scores for the rice and maize classifiers, respectively, the model zoo’s strongest results.These results are measured against established public benchmark annotations.
  • 6.3 Phenotype Accuracy: 0.9605 mAP50 is achieved by the held-out GWHD wheat-spike detector, while rice panicle counting MAE falls to 3.54 from 7.66.The wheat detector scores 0.9418 on a separate CS1 subset, and the rice detector reaches mAP50 = 0.7476.
  • 6.3 Phenotype Accuracy: 0.10 and 0.22 are the temporal pipeline’s growth-rate and harvest-timing MAEs, with 86% and 83% interval coverage, while crop-type Macro F1 is 0.705.The crop-type ensemble head’s held-out Macro F1 is reported as 0.705.
  • 6.4 Uncertainty Calibration: 93.6% mean empirical coverage, mean ECE 0.068, and prediction-set size 1.14 characterize the 95%-target fallback intervals across five classifiers.The RAPS conformal pathway is calibrated and ready for live use, but its offline coverage guarantee remains unbenchmarked.
  • 6.4 Uncertainty Calibration: ±1.00 leaves and ±7.31 spikes are live-session confidence-interval widths for CS2 and CS1, while formal coverage is not reported without expert-annotated ground truth.These are per-session application figures, not offline model-zoo checkpoint benchmarks.
  • 6.4 Uncertainty Calibration: 86.2% and 83.5% P10–P90 coverage are achieved for growth rate and harvest timing against an 80% target using Monte Carlo Dropout (N=30).Both temporal uncertainty results meet or exceed the stated target.

6.7 Accessibility and Practical Tradeoffs

PhenoIntel delivers a first result in under a minute through a browser workflow on standard hardware, while preserving repeatable outputs and limiting practical overhead. Its accessibility comes with bounded verification and training costs, including modest checkpoint latency and laptop-scale fine-tuning limits.

  • Accessibility: Under a minute: PhenoIntel reaches its first result from a cold start after opening a browser, navigating to the URL, uploading images, and entering a prompt.The measured cold start averaged 15 seconds over five runs.
  • Accessibility: At least 8 expert steps and 10–30 minutes: the prior system requires environment installation, credentials, Jupyter launch, and a local GPU.Its documented setup includes Anaconda, CUDA-compatible PyTorch, 40+ Python dependencies, and five Azure OpenAI credentials.
  • Practical tradeoffs: At most 50 ms: each verification checkpoint adds limited latency relative to the 1–5 s required by an existing classification batch.The evaluated deployment used modest hardware without a GPU, while language-model calls used a free-tier API.
  • Practical tradeoffs: 70 minutes: the Build workflow caps fine-tuning at this wall-clock duration and updates only final layers by default on a laptop-class machine.The cited hardware is an Intel i3 with 8 GB RAM; full-network training would take longer but could close the accuracy gap.
  • Practical tradeoffs: Identical phenotype values: repeated runs on the same images remain unchanged across hardware tiers because AI decisions are constrained to three checked points.This constraint preserves deterministic outputs while retaining the system’s checked decision-making structure.

6.8 FAIR Principles and AI Governance

PhenoIntel applies FAIR principles at the pipeline level through reproducible provenance records and uses a contained statistical-code sandbox to support AI governance. Its provenance package includes 12 always-present, checksum-indexed files mapped to FAIR principles.

  • FAIR Principles: Every session produces a FAIR provenance record that other researchers can locate, open, understand, and use to reproduce results.The package applies Findable, Accessible, Interoperable, and Reusable principles at the pipeline level.
  • AI Governance: The statistical-code sandbox provides an audit trail, no network access, a read-only filesystem, hard memory and time limits, and execution-level reproducibility.These controls operationalise containment expectations associated with the EU AI Act and NIST AI RMF.
  • FAIR Principles: 12 always-present files are checksum-indexed in the provenance package manifest, with badges indicating the FAIR principle each file satisfies.The figure presents the package structure and its FAIR-principle mapping.

6.9 Internal Validation and Data Integrity

PhenoIntel’s internal validation combines a fully passing automated test suite with explicit controls for data leakage and evaluation design. These checks enforce pipeline consistency and ensure reported results are benchmarked, pathway-complete, and grounded in ground truth.

  • Automated validation: 1200/1200 automated tests passed, comprising 1,128 unit tests and 72 integration tests, with zero failures.The full local run took 240.5 s wall-clock and produced four non-blocking warnings.
  • Data leakage: Above 0.97 initial validation Macro F1 for the rice nitrogen classifier resulted from duplicate images crossing training and validation splits.Exact-hash deduplication followed by perceptual near-duplicate detection was used to address the leakage.
  • Evaluation design standards: Evaluation checks every inference pathway against ground truth and reports weak results rather than omitting them.Reported accuracies use external benchmarks, including CVPPP 2017 and GWHD, rather than benchmarks designed by the paper’s authors.

7 Discussion

PhenoIntel’s lifecycle-aligned multi-agent architecture addresses distinct agentic-AI failure modes through stage-specific checks and shared structured state, while extending verified plant-phenotyping capability. Its results support scientific rigour, agricultural applicability, and open-science reproducibility, but current evidence remains bounded by modelling, evaluation, and CPU-only hardware limitations.

  • Architecture: Stage-specific checks target failure propagation, tool-selection non-determinism, unstructured communication, unbounded recovery loops, and absent uncertainty quantification.The architecture uses typed shared state, a no-zero-from-failure validator, a structured registry, four-tier fallback, shared session state, a one-replan limit, and bounded adaptation.
  • Agricultural applicability: Macro F1 of 0.78–0.996 across five crops demonstrates expanded accessible crop-AI coverage despite smallholder-relevant low-data regimes.The classification models cover rice, wheat, maize, banana, and coffee; coffee uses N = 800 and banana N = 1,500.
  • Open-science reproducibility: 12/12 required provenance files across all sixteen sessions demonstrate FAIR provenance and full session resumability by construction.The provenance package includes each session’s manifest, calibration record, and verification log.
  • Limitations and future work: The weakest modelling component is the zero-shot segmentation fallback, because no fine-tuned structural-phenotyping checkpoint yet exists.A fine-tuned replacement, such as one trained on CVPPP 2017, is identified as the top priority.
  • Limitations and future work: Three case studies were single runs without bootstrap confidence intervals, while only one of two Pipeline Adaptation Agent triggers was exercised end-to-end.The tested trigger was inference failure in CS5; quality-gate rejection on a real dataset remained untested. All results were produced on a single CPU-only machine, constraining generalisation and scale.

8 Conclusion

PhenoIntel concludes by presenting a lifecycle-aligned, independently checked multi-agent web application with shared typed state, model-matched uncertainty, self-extension, browser-based CPU operation, automated validation, and FAIR provenance. The authors also acknowledge bounded evaluation conditions and identify segmentation, GPU infrastructure, and broader model-zoo expansion as priorities.

  • Conclusion: Nine agents divide plant phenotype analysis into separately checked stages, using one shared typed record with boundary checks V0–V7 and bounded recovery.This design addresses recurring failures associated with a prior monolithic orchestrator.
  • Conclusion: Model-matched uncertainty and self-extension allow PhenoIntel to support new task types and architectures under the same checks as other outputs.The self-extending layer adds new task types and architectures while preserving the platform’s validation framework.
  • Conclusion: 1,200 tests validate complete pipeline execution, while browser deployment requires no GPU and produces a FAIR-compliant provenance record per session.These properties target accessibility and reproducibility of phenotyping analyses.
  • Conclusion: Evaluation is bounded by a single unfunded, CPU-only machine and by results based on a single run or an unconfirmed split rather than reproduction.The authors identify a fine-tuned segmentation checkpoint, GPU-backed training infrastructure, and broader model-zoo expansion as priority future work.

CRediT Authorship Contribution Statement

The contribution statement assigns Narendren S V primary roles in implementation and manuscript preparation, while Soumyashree Kar contributed to study design, supervision, analysis, and project administration.

  • Narendren S V handled conceptualization, methodology, software, data curation, visualization, validation, and manuscript drafting and revision.
  • Soumyashree Kar contributed to conceptualization, methodology, supervision, investigation, validation, visualization, formal analysis, and manuscript revision.
  • Soumyashree Kar also led project administration.

Funding

The research received no specific grant funding from public, commercial, or not-for-profit funding agencies.

  • The research received no specific grant from public, commercial, or not-for-profit funding agencies.
Loading 2608.27999v1…