Source-linked AI summary
IchthyoNoma: Nomenclature and Context Sensitivity of Zero-Shot Biological Vision--Language Models for Bangladeshi Freshwater Fish Recognition
Nazim-E-Alam, Tarek Rahman, Md Kishor Morol
TL;DR
The paper asks whether zero-shot fish recognition remains stable across model, language, nomenclature, prompt, and visual-context changes. It audits frozen models on Bangladeshi freshwater fish and finds that performance depends on biological specialization and the text and image conditions, with context effects varying by intervention and species.
Problem
High in-domain accuracy does not establish transferable biological knowledge, and the stability of zero-shot recognition across models, language, nomenclature, prompts, and visual context remains a research question.
Source: §1 · p. 1Method
The study benchmarks frozen vision–language models on seven shared Bangladeshi freshwater-fish categories while varying names, prompts, and visual context, and includes a multilingual control.
Source: §1 · p. 1, §3.2 · p. 4, §4 · p. 5Results
Takeaways & Limitations
Reporting only the best English or scientific prompt can overstate native-language accessibility, and context robustness should be evaluated with multiple interventions rather than one masking condition.
Source: §4.5 · pp. 7–8The evaluation covers seven categories and two sources, and context interventions alter image distribution as well as information content, limiting perfect causal decomposition.
Source: §4.5 · p. 7
Abstract
from arXiv · showhide
Zero-shot vision-language models (VLMs) are increasingly used as training-free species recognizers, but reported accuracy can reflect more than visual species knowledge. We audit CLIP, BioCLIP, BioCLIP2, and a multilingual Jina CLIP v2 control on seven freshwater-fish categories from two Bangladeshi sources (10,321 images). BioCLIP2 reaches 72.36% on BFF-15 with English common names and 68.91% on SylFishBD with scientific names, versus 25.15% and 14.40% for generic CLIP. BioCLIP2 Bengali prompts are near chance in balanced accuracy (14.22-14.29%); Jina partially recovers Bengali discrimination to 21.89% and 16.36%, but bare Bengali names return to 14.29% on both sources. Paired SylFishBD interventions show no significant weak-blur effect, modest losses from stronger blur/gray masking, a larger white-mask artifact, and strong species dependence. Zero-shot biological VLM scores therefore jointly reflect biological specialization, multilingual alignment, nomenclature, prompt formulation, and context.
1 Introduction
The paper asks how stable zero-shot fish recognition is when models, label language and nomenclature, prompts, or visual context change. It benchmarks biological and generic vision–language models and audits those textual and visual factors on Bangladeshi freshwater fish.
1 Introduction
- High in-domain accuracy does not establish transferable biological knowledge, motivating the question of how zero-shot recognition changes with models, language, nomenclature, prompts, and context.The introduction also notes that fish recognition is complicated by subtle taxonomic differences and variation in pose, illumination, occlusion, and background.Source: §1 · p. 1
- The study benchmarks CLIP, BioCLIP, and BioCLIP2 across seven Bangladeshi freshwater-fish categories and two sources, with a multilingual Jina CLIP v2 control.The framework also evaluates standard classification metrics, per-class performance, and statistical significance.Source: §1 · p. 1, Figure 1 · p. 2
- The nomenclature audit varies Romanized, Bengali-script, English-common, and scientific names, prompt templates, and scientific synonyms to test textual-interface sensitivity.Jina CLIP v2 is included to separate multilingual alignment from biological specialization.Source: §1 · p. 1
- Paired interventions on 7,665 SylFishBD images vary visual context through blur, masks, cropping, background-only views, and cross-species background swaps.The study uses paired inference and multiple-testing correction for these context interventions.Source: §1 · pp. 1–2
2 Related Work
Related work motivates evaluating zero-shot fish recognition across textual interfaces, biological specialization, multilingual alignment, and visual context. It distinguishes this frozen-model evaluation from recognition systems adapted to target tasks.
Prompt and nomenclature robustness
- CLIP uses text-derived class prototypes, making prompt context and category naming part of the classifier; CoOp and CoCoOp show prompt changes can alter recognition (Zhou et al., 2022b,a).Common English names can outperform scientific names with generic CLIP, so precise taxonomic labels are not necessarily the most effective interface (Parashar et al., 2023).Source: §2 · p. 2
Biological and ecological vision–language models
- BioCLIP and BioCLIP2 use biologically structured pre-training, while CLIP-FSSC and TaxaBind extend multimodal approaches to fish or broader ecological recognition (Stevens et al., 2024; Gu et al., 2025; Sastry et al., 2025).This study instead examines the stability of a frozen biological VLM under changes to text and visual context.Source: §2 · pp. 2–3
Multilingual vision–language alignment
- Multilingual benchmarks report cross-language variation, especially for lower-resource languages, while uCLIP explores improving multilingual alignment (Geigle et al., 2024; Chung et al., 2026).The study uses Bengali and Jina CLIP v2 to examine multilingual alignment separately from fine-grained biological specialization (Koukounas et al., 2024).Source: §2 · p. 3
Regional fish recognition and visual context
- Bangladeshi fish datasets support local-taxa evaluation, but supervised adaptation does not establish training-free transfer; background-only signals and context interventions motivate a multi-condition robustness test (Xiao et al., 2021).The study keeps model weights and prompts frozen and treats synthetic context changes as interventions rather than perfect causal isolation.Source: §2 · p. 3
3 Methodology
The methodology evaluates frozen zero-shot models on harmonized fish categories while varying names, prompts, and visual context. It uses paired interventions and statistical procedures to assess performance and differences.
3 Methodology
- The methodology combines dataset harmonization, frozen zero-shot classification, a multilingual control, context interventions, and classification metrics with paired statistical tests.The reported measures include accuracy, balanced accuracy, macro-F1, per-class analysis, bootstrap confidence intervals, and significance testing.Source: §3 · p. 3
3.1 Data harmonization
- The shared seven-species benchmark spans 10,321 images (BFF-15: 2,656; SylFishBD: 7,665); 73 low-distance pairs (≤6 bits) were flagged for review, so the sources are not fully independent.Table 1 gives class nomenclature and image counts; each SylFishBD image is paired with a segmentation mask.Source: §3.1 · pp. 3–4, Table 1 · p. 4
3.2 Frozen zero-shot classification and prompt construction
- Frozen models classify images by matching image embeddings to normalized mean text prototypes built from four prompt templates, with nomenclature tests changing only the text-side classifier.Tests include Romanized, English-common, and scientific names, Bengali script for BioCLIP2, individual templates, and specified scientific-name variants.Source: §3.2 · p. 4
3.3 Multilingual Bengali control
- Jina CLIP v2 uses the same images, classes, four-template ensemble, and four label families as a Bengali-alignment control, with a 512-dimensional truncated representation and no benchmark tuning.A name-only Bengali control tests whether multilingual support suffices or interacts with prompt framing.Source: §3.3 · p. 4
3.4 Paired context interventions
- SylFishBD context stress tests alter backgrounds or image composition using graded blur, neutral-color masks, a tight crop, inpainting, and cross-species background swaps.The transformations are not perfect causal isolation because they also introduce distribution shift.Source: §3.4 · p. 4
3.5 Metrics and inference
- Evaluation reports accuracy, balanced accuracy, macro-F1, and per-class effects, using stratified bootstrap intervals, paired tests, FDR correction, Cochran’s Q, and permutation testing.The primary benchmark uses 1,000 bootstrap resamples; robustness and multilingual controls use 2,000.Source: §3.5 · p. 4
4 Results and Discussion
The results compare generic, biology-specialized, and multilingual models across two Bangladeshi fish sources, then test how labels, prompts, and visual context affect zero-shot recognition. Performance varies across these choices, and context effects differ by intervention and species.
4.1 Biology-specialized VLMs outperform the tested generic CLIP baseline
- 72.36% BioCLIP2 accuracy on BFF-15 with English names and 68.91% on SylFishBD with scientific names contrast with CLIP’s 15.40%/14.40% under scientific prompting.BioCLIP2’s advantage over BioCLIP under scientific prompting is 22.18 and 10.03 points, respectively; the authors caution that checkpoint scale and training recipe also differ.Source: Table 2 · p. 5, §4.1 · p. 5
4.2 Nomenclature and prompt formulation are part of the classifier
- 35.77%/72.36%/69.99% BioCLIP2 accuracy on BFF-15 and 37.56%/64.59%/68.91% on SylFishBD correspond to Romanized/English/scientific names; Bengali prompts yield 14.22%/14.29% balanced accuracy.Scientific-template results span 56.06–77.07% on BFF-15 and 61.44–79.71% on SylFishBD; synonym substitutions also change accuracy, so “scientific prompt” is not fully specified without the name and template.Source: §4.2 · p. 5
4.3 Multilingual alignment partially recovers Bengali, but does not replace biological specialization
- 21.89%/16.36% Jina Bengali balanced accuracy on BFF-15/SylFishBD exceeds BioCLIP2’s 14.22%/14.29%, but English remains ahead and bare Bengali names return to 14.29%.English exceeds Bengali accuracy by 4.10 points on BFF-15 and 13.70 points on SylFishBD; the authors treat Jina as an interface diagnostic, not evidence about Bengali understanding generally.Source: §4.3 · pp. 5–6
4.4 Context sensitivity is real but intervention-dependent
- Weak blur has no significant effect; stronger blur and gray masking reduce accuracy, while white masking costs 8.39 points and class-level effects vary.Strong blur improves Katla/Mrigal by 8.91/8.66 points but hurts Rui by 18.38; background-only images retain 17.90% accuracy, and cross-species swaps reduce accuracy to 56.76%.Source: §4.4 · pp. 6–7, Table 4 · p. 7, Figure 3 · p. 7
4.5 Implications and limitations
- The authors warn that reporting only the best English or scientific prompt can overstate accessibility when the native-language interface is weak.They recommend reporting English-common, scientific, and locally used names, evaluating native-language interfaces separately, and testing context with multiple interventions.Source: §4.5 · pp. 7–8
- The study covers seven categories and two sources; pre-training exposure is unknown, and Bengali findings are specific to tested names and templates.It uses one multilingual control, and context interventions alter image distribution as well as information content, so the authors rely on agreement across several controls rather than perfect causal decomposition.Source: §4.5 · p. 7
5 Conclusion
The paper evaluates zero-shot freshwater-fish recognition across generic, biology-specialized, and multilingual models. BioCLIP2 performs strongest overall, but recognition remains sensitive to language, nomenclature, prompts, and visual context.
5 Conclusion
- BioCLIP2 achieves the strongest overall performance, while Jina CLIP v2 only partially improves Bengali discrimination and falls to chance-level performance with bare Bengali class names.The paper also reports pronounced species-specific variation under paired context interventions.Source: §5 · p. 8
- The findings indicate that biological specialization and multilingual alignment address distinct sources of error.The conclusion calls for zero-shot benchmarks to evaluate language, exact nomenclature, prompt formulation, and contextual robustness rather than a single textual or visual configuration.Source: §5 · p. 8
- Paired context interventions show limited average effects from mild visual degradation, but stronger masking introduces larger distribution-shift artifacts and species-specific variation.Source: §5 · p. 8