PaperlayerView exact revision on arXiv

Source-linked AI summary

FSVL: Taxonomic Fast-Slow Vision-to-Language Collaborative Framework for Interpretable Confusable Fish Species Identification and Knowledge Generation in the Middle Yangtze River

Meng Liang, Fucheng Wang, Lang Zhao, Jun Chen, Ziying Li, Yuxiang Zhao, Haozhuang Chi, Wenkai Fang, Qiong Zhou, Junlin Yang, Jinming Wu

arXiv:2610.31880v1cs.CVcs.AI

TL;DR

Fine-grained identification of closely related fish species remains challenging, while reliable biodiversity monitoring and fisheries management require species-level identification. FSVL combines fast-slow visual recognition with retrieval-grounded language generation; on the evaluated test set, the cascade achieved 100.00% accuracy, and joint adaptation improved knowledge-generation metrics.

  • Problem

    Closely related freshwater fish species have subtle morphological differences, making fine-grained identification challenging for biodiversity monitoring and fisheries management.

    Source: §1 · p. 2

  • Method

    FSVL combines a fast classifier with a taxonomically guided slow path for confusable species and retrieval-augmented, parameter-efficient language-model adaptation.

    Source: §1 · p. 4, §2.2 · p. 9

  • Results

    The cascade achieved 100.00% test accuracy after correcting two errors from the 99.67% fast path, while joint adaptation raised scientific name accuracy to 100.00%.

    Source: §3.2.2 · p. 18, §3.3 · p. 20

  • Takeaways & Limitations

    The framework supports fish identification and biological knowledge generation for biodiversity monitoring and fisheries management in the middle Yangtze context.

    Source: §1 · p. 2, §5 · p. 28

    Domain shifts across water bodies, lighting, and acquisition devices may degrade stability, while cascaded reasoning adds latency that constrains real-time edge deployment.

    Source: §4.4 · p. 27

Abstract

from arXiv · show

The Yangtze River Basin is a critical region for freshwater biodiversity conservation. The implementation of the Yangtze River Ten-Year Fishing Ban has increased the importance of effective aquatic biodiversity monitoring and species-level identification. However, fine-grained identification of closely related freshwater fish species remains challenging due to subtle morphological differences. Existing deep learning methods often struggle to balance classification accuracy and inference efficiency, while LLMs may generate unsupported information when producing specialized biological knowledge. To address these challenges, this study proposes FSVL, a fast-slow dual-pathway framework integrating visual recognition with LLM-based Retrieval-Augmented Generation (RAG). A lightweight ResNet50 fast path performs preliminary classification across 51 fish species in the Middle Yangtze River Basin and routes predefined groups of easily confused species to a slow path for refined identification. Guided by taxonomic criteria, the slow path uses YOLOv8n to detect key morphological regions, including the head, dorsal fin, and tail, leveraging morphometric and outline traits to improve discrimination among closely related species. The framework further integrates a retrieval-enhanced Qwen3-14B backend, using soft tokens and Low-Rank Adaptation (LoRA) for parameter-efficient domain knowledge generation. On the evaluated test set of 607 images, the fast classifier achieved 99.67% accuracy, whereas the cascade corrected the two errors, yielding 100.00% overall accuracy; further validation on independent specimens and external conditions remains needed. Across 255 knowledge generation samples, scientific name accuracy rises from 91.37% to 100.00%, genus accuracy from 90.20% to 98.82%, and ROUGE-L from 0.335 to 0.655, supporting intelligent biodiversity monitoring and fisheries management.

1. Introduction

The introduction motivates fish identification as a need for biodiversity monitoring in the Middle Yangtze, where closely related species can be difficult to distinguish. It proposes FSVL, combining fast-slow visual recognition with taxonomically guided local features and retrieval-grounded knowledge generation.

1. Introduction

  • Accurate fish identification supports assessment of fish-community changes and enforcement of conservation regulations in the middle Yangtze, where the fishing ban heightens monitoring needs.The basin hosts approximately 215 fish species, and subtle morphological traits make visual identification difficult.Source: §1 · p. 2
  • Closely related fish species remain difficult to identify because diagnostic differences are subtle and can be obscured by posture, illumination, or background variation.Overall-image classifiers may miss local traits, and attention mechanisms do not explicitly encode taxonomic criteria.Source: §1 · p. 3
  • Traditional identification requires specialist expertise, while visual classification alone does not provide taxonomic explanations or structured biological knowledge.LLM-generated descriptions can be factually inconsistent even when retrieval is used.Source: §1 · pp. 3–4
  • FSVL classifies 51 fish species with a ResNet50 fast path and routes predefined confusable groups to a YOLOv8n slow path that fuses local diagnostic regions with global image features.Its knowledge-generation backend combines a fish knowledge base and Qwen3-14B with soft-token and LoRA adaptation.Source: §1 · p. 4

2. Materials and Methods

FSVL combines a 51-species fast classifier with a routed slow path for confusable taxa, then uses the confirmed label to generate retrieval-grounded biological descriptions. Its methods also describe the regional dataset, taxonomic feature selection, global-local fusion, and module training settings.

2.1 Dataset Construction

  • The described dataset was assembled from fish collected at five Middle Yangtze River sections and photographed under controlled aquarium conditions.The collection includes live or freshly sampled specimens and uses DSLR cameras, macro lenses, and diffused flash.Source: §2.1 · p. 5
  • The dataset covers 51 Middle Yangtze River fish species, with individual specimens kept within a single split and test images left unaugmented.The described split includes 4,818 training images and an independent 607-image test set.Source: §2.1 · pp. 5, 7

2.2 Key site definition guided by expert taxonomy

  • Expert identification progresses from body color and patterns to head and mouth, fins and pelvic spines, then lateral-line and scale counts; the slow path uses the head, dorsal fin, and tail as local regions.This feature selection encodes the described taxonomic process in the model.Source: §2.2 · p. 9

2.3 Dual path Fast Slow Collaboration

  • The system standardizes fish images, classifies them with ResNet50, routes predefined confusable taxa to a YOLOv8n slow path, and uses the confirmed label to guide RAG-based description generation.The slow path combines global and local features from the head, dorsal fin, and tail.Source: §2.3 · p. 10

2.4 Global Local Feature Fusion

  • The slow path concatenates one 2,048-dimensional global feature with three 2,048-dimensional local features into an 8,192-dimensional vector for classification.The global representation describes overall shape, color, and outline; local regions are detected and cropped by YOLOv8n.Source: §2.4 · pp. 12–13

2.5 RAG Knowledge Enhancement Module

  • The RAG module retrieves structured morphology, ecology, and distribution evidence for the classified species and supplies it to Qwen3-14B for biological description generation.Soft tokens and LoRA adapt the model while its core parameters remain frozen; training masks prompts from the response loss.Source: §2.5 · pp. 13–14

2.6 Experimental Setup

  • The fast path uses a two-stage ResNet50 training schedule, while the detector, fusion classifier, and language-model adaptation each have separately specified training settings.The second fast-path stage uses five epochs, a 1e-5 learning rate, and Color Jitter; backend adaptation uses eight soft prompt vectors and LoRA rank 8.Source: §2.6 · pp. 14–15

3. Results

The fast-slow system corrected both fast-path errors on the 607-image test set, while its detection and fusion components refined identification of confusable species. The jointly adapted language model improved taxonomic-name accuracy and generated structured species profiles.

3.1 Fast Path Results

  • 99.67% standalone test accuracy followed two-stage training; Color Jitter fine-tuning raised training accuracy from 96.72% to 99.29%.The fast path classified 51 species using ResNet50; Table 5 reports its training-stage recognition accuracy.Source: §3.1 · p. 15, Table 5 · p. 15

3.2.1 Key Part Localization Results of the YOLO Detector

  • YOLOv8n reached mAP50 0.98 and mAP50 95 0.804 on 135 validation images, with 0.8 ms detection and approximately 2.0 ms total processing per image.The detector was trained to locate the head, dorsal fin, and tail; its model size is 6.2 MB.Source: §3.2.1 · p. 16, Table 6 · p. 16, Figure 7 · p. 17

3.2.2 Results of Slow Path Fusion Classifier

  • The cascade corrected both fast-path errors, raising accuracy from 99.67% to 100.00% on the 607-image test set.The slow path is triggered only for six predefined confusable species; its fusion classifier also reached 100.00% internal validation accuracy across those six species.Source: §3.2.2 · pp. 17–18, Table 7 · p. 18
  • The fast path had 23.61 M parameters and 876.34 ms single-image CPU inference time, versus 51.34 M parameters and 2,991.95 ms for the slow path.The slow-path encoders account for 16.52 G FLOPs; these measurements exclude YOLO, SAM3, and LLM generation overhead.Source: §3.2.2 · p. 19, Table 8 · p. 19

3.3 LLM Enhancement Results

  • Joint soft-token and LoRA tuning raised scientific name accuracy from 91.37% to 100.00% and genus accuracy from 90.20% to 98.82%, while ROUGE-L rose from 0.335 to 0.655.The backend uses retrieved taxonomic records to generate structured species profiles with diagnostic traits for post-recognition verification.Source: §3.3 · p. 20, Table 9 · p. 20, Figure 11 · p. 21

3.4 Ablation Experiment Results

  • Fixed routing achieved 100.00% test accuracy, exceeding confidence-threshold strategies that scored 99.67%, 99.84%, and 99.34%.The fixed strategy directs six predefined confusable species to the slow path and uses fast-path predictions for other classes.Source: Table 10 · p. 21, §3.4 · p. 22
  • LoRA alone reached 93.33% scientific name accuracy and 92.94% genus accuracy, while soft-token-only tuning reduced both measures to 60.39%.Combining soft tokens with LoRA produced the best comprehensive performance; the two methods target entity relationships and domain-specific phrasing, respectively.Source: §3.4 · pp. 22–23

4.1 Evaluation of Asymmetric Routing Efficiency and Computational Overhead

The fast path uses standardized fish images for preliminary classification across 51 species. Slow-path computation is reserved for predefined confusable taxa, though field trigger rates and system latency require further profiling.

4.1.1 Fast Path Accuracy

  • Background standardization and preprocessing support the ResNet50 fast path’s classification across 51 species, with the discussion attributing performance partly to removal of environmental backgrounds.The text emphasizes that field-deployable recognition depends on data quality and standardized visual inputs.Source: §4.1.1 · p. 23

4.1.2 Routing Arbiter and Computing Power Allocation

  • The routing arbiter sends predictions in predefined confusable taxa to the slow path without confidence thresholds, but field trigger proportions require empirical profiling.Average system latency depends on the ambiguity distribution; the passage frames the design as balancing screening throughput and fine-grained precision.Source: §4.1.2 · pp. 23–24

4.2 Discriminative Efficacy of Taxonomically Grounded Global-Local Fusion

Taxonomically selected local regions provide discriminative cues for closely related fish, while global features add outline, proportion, and color context. Their fusion is associated with strong validation and system-level identification results.

4.2.1 Local Feature Recognition Method

  • Head, dorsal-fin, and tail regions were selected using evolutionary and taxonomic criteria, and detector localization supplies structured inputs for fine-grained recognition.The discussion connects these regions to expert inspection logic and describes local traits as useful when posture, lighting, or occlusion complicate identification.Source: §4.2.1 · p. 24

4.2.2 Fusion of Local Features and Global Features

  • Global outline, body proportion, and color features complement local morphological cues in a fused vector, whose classifier achieved high validation accuracy and improved overall accuracy after slow-path intervention.The discussion characterizes the taxonomy-driven design as simpler to implement than comparable attention or dual-branch approaches.Source: §4.2.2 · p. 25

4.3 Efficacy of Parameter-Efficient Adaptation in Grounded Ecological Reasoning

The section reports that parameter-efficient adaptation improves retrieval-grounded scientific and genus generation, while the generated profiles provide taxonomic reference information for secondary verification.

4.3 Efficacy of Parameter-Efficient Adaptation in Grounded Ecological Reasoning

  • Joint parameter-efficient adaptation significantly outperforms retrieval alone in scientific and genus accuracy.The comparison covers retrieval-augmented generation alone and schemes that add parameter-efficient fine-tuning.Source: §4.3 · p. 25
  • Soft tokens adapt task-related prompt representations, while LoRA adjusts attention weights to constrain generation alongside retrieved factual grounding.The section describes their combined use as adaptation of both input representations and model parameters.Source: §4.3 · p. 26
  • Generated profiles summarize diagnostic traits, ecological habits, and conservation status for frontline personnel’s secondary verification.The backend conditions on predicted taxonomic labels and retrieved records rather than directly checking specimen-specific traits in image pixels or bounding boxes.Source: §4.3 · pp. 26–27

4.4 Current Methodological Bottlenecks and Future Research Trajectories

The section identifies constraints in data, field generalization, computational deployment, and knowledge evaluation, then proposes research directions addressing each area.

4.4 Current Methodological Bottlenecks and Future Research Trajectories

  • The slow-path specialist covers six representative confusable species, and generated profiles rely on predicted taxa and retrieved records rather than direct pixel-level trait verification.The section also notes that latency profiling and external validation in unconstrained underwater environments remain future work.Source: §4.4 · p. 27
  • Scarce, imbalanced training data and shifts in water bodies, lighting, and acquisition devices constrain generalization, while multi-branch fusion and cascaded reasoning add computational overhead and latency.These constraints impede real-time deployment on compute- and power-limited edge monitoring terminals.Source: §4.4 · p. 27
  • Proposed directions include few-shot learning and domain adaptation, compression and edge computing, dynamic multimodal retrieval, and standardized evaluation benchmarks.Suggested evaluation measures include field accuracy, intragenus discriminability, and hallucination penalties.Source: §4.4 · pp. 27–28

5. Conclusion

The conclusion presents FSVL as a fast-slow framework for fine-grained fish identification and grounded knowledge generation, combining taxonomic visual analysis with retrieval and language-model adaptation.

5. Conclusion

  • FSVL uses dynamic routing to screen species on a lightweight fast path and activate a slow path for confusable species.The slow path uses expert taxonomic knowledge, key local morphological regions, and global-local feature fusion.Source: §5 · p. 28
  • A RAG backend with joint soft-token and LoRA adaptation and masked response-only loss generates structured scientific content based on retrieved facts.The conclusion describes this design as constraining generation to address factual hallucinations.Source: §5 · p. 28
  • The authors conclude that the visual-to-knowledge pipeline has practical value for biodiversity monitoring and science popularization in support of the Yangtze River Ten-Year Fishing Ban.Source: §5 · p. 28

CRediT authorship contribution statement

The authorship statement assigns contributions across project leadership, research and software work, investigation, data curation, and writing.

CRediT authorship contribution statement

  • Meng Liang contributed to project administration, data curation, conceptualization, methodology, analysis, software, visualization, investigation, and manuscript drafting and revision.Source: CRediT authorship contribution statement · p. 29
  • Fucheng Wang contributed supervision, resources, conceptualization, software, analysis, visualization, and manuscript revision; Lang Zhao contributed methodology, analysis, visualization, and revision.Source: CRediT authorship contribution statement · p. 29
  • Jun Chen, Ziying Li, Yuxiang Zhao, Haozhuang Chi, Wenkai Fang, Qiong Zhou, Junlin Yang, and Jinming Wu are credited with software, investigation, methodology, data curation, visualization, conceptualization, project administration, funding acquisition, or supervision.The statement assigns different subsets of these roles across the listed contributors.Source: CRediT authorship contribution statement · p. 29

Related papers

Loading 2610.31880v1…