Source-linked AI summary
An Agentic Framework Using Rules and LLMs for Embedding and Annotating Descriptive Document Layouts: A Plant Science Use Case
Nicolas Turenne, Youcef Sklab, Eric Chenin, Jean-Daniel Zucker
TL;DR
Specialized botanical corpora need domain-adapted, explainable approaches for extracting structured plant traits. This paper combines agent-based OCR, indexing, rule-based parsing, and LLM vocabulary enrichment, increasing annotations by 59% for 75% of traits.
Problem
Existing retrieval and agentic approaches often lack domain-specific ontological adaptation for specialized scientific corpora, motivating automated botanical trait extraction.
Method
The modular pipeline combines OCR, genus-and-species segmentation and indexing, rule-based parsing, and LLM-based vocabulary enrichment for botanical trait extraction.
Results
59% average annotation increase followed vocabulary enrichment for 75% of traits across regional botanical datasets.
Takeaways & Limitations
The explainable pipeline supports construction of structured plant trait databases for taxonomy, ecology, and biodiversity research.
Takeaways & Limitations
Error propagation and reliance on human-guided rule refinement limit full automation.
Abstract
from arXiv · showhide
Background: Recent advances in information retrieval (IR) leverage both dense and sparse representations, large language models (LLMs), and specialized retrieval models to improve ranking accuracy, relevance, and cross-lingual performance. Complementary techniques such as passage indexing, document layout analysis, and semantic knowledge representation further enhance retrieval effectiveness by capturing fine-grained contextual and structural information. Emerging agentic LLM frameworks extend these capabilities by enabling planning, iterative reasoning, tool use, and multi-agent collaboration, thereby broadening applications across diverse domains. These frameworks also emphasize rigorous evaluation, ethical considerations, and trustworthiness, ensuring responsible deployment in real-world settings. We propose a modular, agent-based pipeline for botanical trait extraction. Optical character recognition (OCR) converts PDFs into machine-readable text, while segmentation and indexing organize content by genus and species. Rule-based parsers extract structured botanical traits, and ensembles of large language models (LLMs) expand trait vocabularies and resolve ambiguities. This approach ensures accurate species recognition, scalable annotation, and explainable integration of textual botanical descriptions, enabling robust and interpretable data extraction across large botanical corpora. Results: Using three regional botanical datasets, our system extracted 55,737 trait annotations across 4,961 species, averaging 9.1 traits per species. Integration of LLM-based enrichment improved coverage for 75% of traits, increasing total annotations by 59%. While the choice of OCR engine had a minor effect on species recognition, overall annotation counts remained stable, demonstrating the robustness, scalability, and reliability of the pipeline for large-scale botanical trait extraction.
1 Introduction
The paper presents a modular agentic framework for indexing and extracting botanical traits from descriptive documents. It combines OCR, segmentation, indexing, rules, and LLM reasoning to produce scalable, interpretable annotations across regional botanical datasets.
- Motivation: The approach embeds plant-trait domain knowledge into a searchable catalog for standardized retrieval and analysis of morphology, habitat, life cycle, and geographic distribution.These features are collectively referred to as plant traits by botanists and ecologists.
- Background: Recent IR advances combine dense and sparse representations, LLMs, retrieval models, passage indexing, layout analysis, and semantic knowledge representation to improve retrieval effectiveness.Related work also includes fine-grained proposition-level indexing and vision- or graph-based document layout analysis.
- Research gap: Existing agentic LLM methods support planning, reasoning, and multi-agent collaboration but often lack domain-specific ontological adaptation for specialized scientific corpora.The proposed framework addresses this gap through domain-focused botanical trait extraction.
- Method: The framework integrates OCR, document segmentation, passage indexing, and LLM-enhanced reasoning in a modular architecture for large-scale, interpretable botanical trait extraction.Its hybrid annotation combines deterministic linguistic rules with iterative vocabulary enrichment from multiple LLMs while maintaining transparency and explainability.
- Demonstration: Over 55,000 morphological trait annotations were achieved across 4,961 species from regional floras in New Caledonia, Senegal, and Cameroon.The demonstration applies the framework to thousands of species descriptions from regional botanical datasets.
2 State of the art
The state of the art combines dense and sparse information retrieval, passage and layout indexing, semantic representations, and agentic LLM frameworks for structured, context-aware information access. Against these approaches, the proposed framework specializes in botanical corpora through trait-focused parsing and a hybrid rule-, NER-, and ontology-based strategy.
- Information retrieval: IR systems span dense versus sparse representations and supervised versus unsupervised approaches, with vectorization evolving from weighted-frequency models to fixed-size embeddings.Dense and sparse retrieval can also be combined for cross-lingual document reranking.
- Indexing and layout analysis: Passage indexing encodes and indexes large passage collections for efficient retrieval, while document layout indexing organizes structural components for layout-based search and automated analysis.Layout-indexing methods generally use graph-based or vision-based approaches.
- Agentic LLM frameworks: Agentic LLM frameworks add planning, reasoning, tool invocation, and adaptation through layered or iterative calls for sustained, context-aware decision-making.Evaluation and governance remain central, including long-horizon reasoning benchmarks and trust, risk, security, fairness, transparency, robustness, safety, and oversight frameworks.
- Botanical specialization: The proposed method specializes in botanical corpora containing millions of species descriptions, herbarium labels, and Wikipedia entries, requiring trait-specific semantic parsing for morphological feature extraction.This botanical specialization is identified as a key difference from broad state-of-the-art information retrieval and knowledge-extraction approaches.
- Hybrid extraction approach: The framework combines rule-based gazetteers, fine-tuned NER, and ontology-driven standardization rather than relying primarily on dense and sparse embedding-based retrieval models.The hybrid strategy is intended to improve precision in structured plant-trait extraction and reduce noise from overlapping or ambiguous terminology.
3 Dataset and resources
The dataset comprises more than 550 multilingual botanical monographs spanning approximately 36 geographical regions, primarily digitized scans requiring OCR for downstream NLP. Documents were converted, organized by species, and linked with major botanical resources despite taxonomic disagreement and OCR-related segmentation challenges.
- Corpus and OCR: More than 550 monographs spanning approximately 36 geographical regions were included, with OCR requiring 127 hours of computation.Most documents were published between 1845 and 2000 and were digitized scans of printed botanical volumes.
- Corpus and OCR: Tesseract OCR version 5.5.0 converted scanned documents into machine-readable text for downstream NLP applications.The engine supports multiple languages and combines classical image processing with LSTM-based neural networks.
- Corpus and OCR: 32% increase in genus detection and 14% increase in species name recognition were observed when Tesseract was compared with classical PDF-to-text tools.Manual annotation found no significant improvement in overall accuracy, but Tesseract was adopted for its robustness and preliminary extraction performance.
- Corpus and OCR: The corpus contains documents in English, French, Spanish, Portuguese, Mandarin Chinese, and Russian.This multilingual composition supports text recognition across diverse botanical sources.
- Dataset structuring and resources: Species-specific text segments were organized into file-species units and connected to WCVP, TRY Database, and WFO resources.Accepted species-name totals differ between WFO (388,342) and WCVP (407,961), while OCR errors and textual inconsistencies complicate species-section boundary detection.
4 Methods
The methods comprise a five-stage agentic pipeline that converts scanned PDFs into structured botanical trait annotations through OCR, indexing, rule construction, LLM enrichment, and application of refined rules. Species-name disambiguation and genus- and species-level indexing support targeted, explainable extraction from taxonomic descriptions.
- Agentic workflow: The five-stage workflow performs OCR, indexes and embeds relevant passages, defines classification rules, enriches them with an LLM ensemble, and generates structured annotations.Each stage is implemented by agents whose outputs feed subsequent stages.
- Species-name matching: WordGen resolves minor species-name errors through exact matching, substitutions, insertions, deletions, and loose prefix matching against a curated epithet dictionary.The algorithm accepts a genus or genus plus epithet and returns possible matching names.
- Segmentation and indexing: Genus-specific segmentation reduces ambiguity from shared epithets, while indexing records each species passage’s first and last lines and its begin_line and end_line positions.A species may correspond to multiple passages within a genus file.
- Segmentation and indexing: Segmentation can misassign descriptions when a genus is missed or falsely detected, causing text from one genus to be appended to another genus file.The bias arises from copying lines into the current genus block until the next genus is detected.
- Rule-based extraction: Rule-based extraction identifies trait concepts and values from contextual markers and characteristic expressions, such as leaf-related terms paired with dimensional measurements for leaf size.Traits are defined as organism properties consisting of a concept and an associated value.
- LLM enrichment: LLM ensembles act as virtual domain experts that expand manually defined botanical rules with new terms, increasing annotation coverage.Term expansion is implemented iteratively with an ensemble of LLMs.
5 Evaluation and discussion
The evaluation measured trait extraction and species-level annotation quality across botanical datasets, while testing LLM trait detection and pipeline robustness. Rule-based extraction produced 55,737 annotations across 4,961 species, and LLM enrichment improved annotation coverage across most traits.
- LLM trait detection: 67% (6/9) accuracy was achieved for fruit detection, compared with 56% (5/9) for inflorescence detection when an LLM was directly prompted on species descriptions.The prompt required binary trait identification without additional text, and the results indicated inconsistency in specialized factual extraction.
- Dataset-scale evaluation: 55,737 trait annotations were extracted across three regional botanical datasets using rule-based extraction for 29 traits.The datasets covered New Caledonia, Senegal, and Cameroon collections.
- Dataset-scale evaluation: 9.1 annotations per species were obtained on average, with 4,961 species having at least five annotations.Annotation density was uneven, declining sharply after five traits and reaching no more than 21 of 29 possible annotations per species.
- Pipeline evaluation: A slight improvement in species-name identification was observed with Tesseract compared with pdf2txt, illustrating OCR-related error propagation across dependent agents.The architecture distinguishes slower, more accurate neural-network OCR methods from faster, less precise geometry-based methods.
- Pipeline evaluation: 59% average improvement in annotations across all traits resulted from LLM-based vocabulary enrichment, which affected 75% of traits.Parser agents were compared with and without enrichment in an interdependent pipeline where OCR and indexing categories could not be removed without collapsing the system.
6 Conclusion
The study presents a modular, agent-based architecture that combines rule-based botanical trait extraction with LLM-driven vocabulary enrichment. Evaluations show improved trait coverage and annotation counts, while error propagation and human-guided rule refinement limit full automation.
- Architecture: The architecture integrates OCR, segmentation, indexing, rule-based parsing, and LLM vocabulary enrichment for large-scale extraction of botanical morphological traits.Its extraction rules target plant morphological information, while enrichment is applied across a group of large language models.
- Results: 75% of traits were affected by LLM-based enrichment, increasing annotations by an average of 59%.The evaluation used regional botanical datasets and found rule-based methods provided reliable trait annotation.
- Limitations: Error propagation across agents and reliance on human-guided rule refinement currently limit full automation.Despite these limitations, the framework is described as transparent and explainable.
- Implications: Balancing rule-based precision with LLM-driven vocabulary expansion supports structured plant trait databases for taxonomy, ecology, and biodiversity research.The framework combines precise rule-based extraction with expanded trait vocabularies to facilitate database construction.