PaperlayerView exact revision on arXiv

Source-linked AI summary

Explainable Multi-Loss Distillation Framework for Efficient and Interpretable Shrimp Disease Text Classification

Anh Nguyen Quynh, Khang Nguyen Quoc, Luyl-Da Quach

arXiv:2608.29027v1cs.CV

TL;DR

Shrimp disease text classification addresses the limits of image-based detection and the need for efficient, interpretable Vietnamese-language models. SALT combines multi-loss knowledge distillation with LIME and SHAP, achieving teacher-comparable performance with a compact student while supporting explanation analysis.

  • Problem

    Shrimp disease research largely uses image classification, while efficient and interpretable Vietnamese text classification for earlier symptom-based detection remains limited.

  • Method

    SALT combines teacher–student knowledge distillation across multiple objectives with LIME and SHAP analyses for Vietnamese shrimp disease text classification.

  • Results

    The 79.88M-parameter student matches the PhoBERT-base teacher’s classification performance at lower computational cost, while Cosine achieves the best overall performance and explanation consistency.

  • Takeaways & Limitations

    SALT supports efficient and interpretable Vietnamese shrimp disease text classification, with Cosine distillation offering the strongest reported performance–explanation consistency trade-off.

  • Takeaways & Limitations

    The small Vietnamese-only dataset may not represent the diversity of real-world disease stages, farming conditions, and regional language variations.

Abstract

from arXiv · show

Shrimp disease classification has become an urgent issue due to its significant impact on the import-export output of producing countries, particularly Vietnam. Most existing studies focus on image-based classification, which typically operates at the late stage of disease manifestation. Therefore, text-based classification has the potential to enable early and timely disease detection. To address this limitation, we introduce the SALT (Shrimp disease text Analysis with multi-Loss disTillation) framework, which incorporates explainability analysis using Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP) to evaluate model predictions and interpret the learned linguistic features. Experimental results demonstrate that SALT achieves competitive performance across multiple distillation objectives, outperforming supervised baselines while providing a favorable trade-off between predictive performance and computational efficiency. Moreover, it exhibits strong explainability, accurately identifying key linguistic features and semantic patterns relevant to disease descriptions. These findings highlight the potential of knowledge distillation-based text classification for future applications in early shrimp disease diagnosis and related research directions.

Highlights

The section introduces SALT, an explainable multi-loss distillation framework for Vietnamese domain-specific text understanding, and summarizes its evaluation across objectives, baselines, efficiency, and explanation analysis.

  • Framework contribution: SALT unifies teacher-student learning with explainability-driven evaluation for Vietnamese domain-specific text understanding while comparing distribution-based and metric-learning distillation losses.This establishes the framework’s central contribution and multi-objective design.
  • Explainability analysis: The framework integrates LIME for local explanations, aggregated SHAP values for global feature attribution, and keyword deletion to assess explanation faithfulness.These methods jointly evaluate local importance, global attribution, and whether identified keywords affect predictions.
  • Experimental evaluation: Experiments comprehensively compare multiple distillation objectives and strong baselines using classification performance, computational efficiency, and explanation analysis.The evaluation spans predictive, computational, and interpretability dimensions.

1. Introduction

Shrimp disease outbreaks threaten Vietnam’s shrimp production, while existing image-based and black-box language-model approaches leave early text diagnosis constrained by domain coverage, efficiency, and interpretability. SALT addresses these challenges by combining knowledge distillation with LIME and SHAP-based explainability for Vietnamese shrimp-disease text classification.

  • Motivation: Disease outbreaks pose major productivity challenges despite Vietnam ranking among the world’s top three shrimp-exporting countries.Global shrimp production exceeded 3 million tons according to the May 2025 FAO report.
  • Related work: Prior shrimp-disease AI studies primarily use CNNs and transfer learning for image classification, typically targeting visible disease manifestations.One reported study achieved approximately 90% accuracy with InceptionV3 and MobileNet models.
  • Research gap: Vietnamese PLMs such as BERT and PhoBERT have succeeded broadly, but their use for symptom-based shrimp-disease text diagnosis remains limited.The domain requires both domain-specific semantic understanding and computational efficiency for practical deployment.
  • Research gap: Existing PLMs are computationally expensive and often function as black boxes, limiting deployment efficiency and insight into decision-making and feature attribution.These limitations are especially relevant to Vietnamese textual symptom descriptions in aquaculture diagnosis.
  • Contribution: SALT combines teacher–student knowledge distillation with LIME and SHAP explainability analysis to develop lightweight, transparent Vietnamese domain-specific text classification.The framework addresses computational efficiency, limited domain-specific Vietnamese resources, and black-box interpretability.
  • Contribution: The framework compares multiple distillation objectives and evaluates explanations through local LIME, aggregated SHAP attributions, and keyword deletion testing.The evaluation covers classification performance, computational efficiency, and explanation analysis against strong baselines.

2. Materials and method · 2.1. Data collection and preprocessing

The study uses ShrimpCap, a Vietnamese shrimp-disease symptom dataset, and applies preprocessing to standardize descriptions and reduce redundancy. The processed data support semantic analysis and are divided into training, validation, and test sets for experimentation.

  • 2.1. Data collection and preprocessing: ShrimpCap initially contains 1,000 Vietnamese symptom descriptions spanning 10 common shrimp diseases.The diseases include Acute Hepatopancreatic Necrosis Disease, White Feces Disease, White Spot Disease, Luminous Bacterial Disease, Loose Shell Syndrome, Black Gill Disease, Filamentous Bacterial Disease, Vitamin C Deficiency Disease, Yellow Head Disease, and Taura Syndrome.
  • 2.1. Data collection and preprocessing: The dataset was curated from publicly available Vietnamese aquaculture resources and includes English translations of symptom descriptions.Table 1 presents the Vietnamese symptom descriptions alongside their English translations.
  • 2.1. Data collection and preprocessing: Preprocessing removes duplicates and semantically redundant descriptions to improve consistency and minimize bias during model training.The pipeline includes data cleaning, Unicode normalization, and Vietnamese diacritic standardization.
  • 2.1. Data collection and preprocessing: The preprocessing pipeline removes HTML tags, normalizes Unicode characters, and standardizes Vietnamese diacritics to prevent spurious features.Figure 1 illustrates the data preprocessing pipeline, while Figure 2 shows the transformation of a specific raw description.
  • 2.1. Data collection and preprocessing: Sentence-embedding similarity analysis reveals high semantic similarity among several disease pairs, indicating shared symptom descriptions.The inter-class similarity matrix is based on average pairwise cosine similarity of sentence embeddings.
  • 2.1. Data collection and preprocessing: After preprocessing, the dataset contains 979 symptom descriptions partitioned into training, validation, and test sets using a 6 : 2 : 2 ratio.The resulting data statistics are summarized in Table 2.

2.2. Augmentation for baseline evaluation.

The study augments the ShrimpCap training set to evaluate whether conventional linguistic augmentation improves student-model performance. It uses four method-based strategies and a prompt-guided GPT-5.5 framework to generate semantically equivalent disease descriptions with diverse wording and styles.

  • Augmentation purpose: The augmented training set was constructed exclusively for the linguistic augmentation baseline to assess its effect on student-model performance.The augmentation was applied to the ShrimpCap training set.
  • Method-based augmentation: Four strategies were applied: synonym replacement, contextual augmentation, paraphrasing, and symptom mix-up.These methods introduce lexical diversity, preserve meaning, enrich grammatical structure, or combine same-class symptom descriptions.
  • Method-based augmentation: Synonym replacement uses a manually edited dataset dictionary, while contextual augmentation replaces words with PhoBERT-predicted contextually appropriate alternatives.Both methods aim to preserve accuracy, context suitability, or semantic equivalence while increasing lexical diversity.
  • Method-based augmentation: Paraphrasing uses VietAI/vit52base to generate semantically equivalent expressions, and symptom mix-up combines descriptions from the same disease class.These strategies enrich grammatical structure, increase data diversity, and support learning disease-group characteristics.
  • Prompt-guided augmentation: A GPT-5.5 prompt-guided framework generates descriptions with diverse wording and styles using GOAL, KEY, and TRANSLATE components.GOAL specifies semantic equivalence, KEY preserves essential disease characteristics and prevents contradictions or fabricated symptoms, and TRANSLATE controls stylistic transformations.

2.3. Method

The SALT method uses a three-phase teacher–student framework to select a teacher, compare lightweight students under multiple training objectives, and analyze model explanations. Its student is a truncated PhoBERT model with gated fusion, while LIME and SHAP provide local and global interpretability for shrimp disease symptom classification.

  • Overall framework: SALT comprises teacher selection, lightweight model comparison, and explainability analysis phases.Fine-tuned state-of-the-art models are evaluated to select the teacher; lightweight students are trained through distillation or supervised learning; XAI methods assess model behavior.
  • Teacher models: PhoBERT is selected for its Vietnamese contextual modeling, while mBART provides a multilingual comparative baseline.PhoBERT is trained on a large Vietnamese corpus with word-level tokenization, whereas mBART is a multilingual denoising sequence-to-sequence model.
  • Student architecture: The student uses the first four Transformer layers of a truncated PhoBERT encoder, gated fusion, projection layers, and a classifier for 10 disease classes.The projection head includes a linear layer, ReLU activation, and dropout with rate 0.1; the architecture targets efficient inference and Vietnamese-language representation.
  • Distillation objectives: Distillation compares soft-target learning with KL, MSE, Cosine, JSD, SupCon, Center, and Triplet losses.These objectives align teacher–student probability distributions, logits, directions, or feature representations using different mechanisms.
  • Explainability analysis: LIME explains individual predictions, while SHAP aggregates feature contributions to evaluate global explainability.LIME is applied to teacher and student test-set predictions to extract five influential words and attribution weights; SHAP uses 200 training samples as background and computes mean absolute SHAP values on the test set.

2.4. Experimental setup

The experimental setup standardizes training and method-specific configurations to support fair, reproducible comparisons. Evaluation combines classification metrics, efficiency measurements, repeated seeded runs, and comparative experiments covering framework components and baseline methods.

  • Training configuration: All models use identical optimization settings, while knowledge-distillation, metric-learning, and explainability hyperparameters remain fixed unless otherwise specified.Training hyperparameters are summarized in Table 4, and method-specific settings are summarized in Table 5.
  • Evaluation protocol: Accuracy, Precision, Recall, and F1-Score are reported, with teacher-phase model selection based on F1 performance.Validation-based selection uses macro-F1, while the test set is reserved for final evaluation.
  • Efficiency measurement: Efficiency is measured using parameter count, model size, GFLOPs, inference latency, and throughput under batch size 1 and a maximum sequence length of 384 tokens.Experiments use an NVIDIA GeForce RTX 4060 GPU and an Intel Xeon E5-2667 v2 CPU.
  • Experimental protocol: Experiments are repeated five times with different random seeds, and results are reported as the mean ± standard deviation across runs.Model selection is performed exclusively on the validation set using macro-F1.
  • Comparative experiments: Comparative experiments assess individual framework components and benchmark SALT against internal and external alternatives.Setups include PhoBERT-L teacher distillation, gated-fusion ablation, linguistic augmentation, supervised fine-tuning, TF-IDF + DNN, and F2LLM-v2-80M.
  • Baseline methods: External baselines include a reimplemented best-performing TF-IDF + DNN configuration and a frozen F2LLM-v2-80M multilingual text-embedding model.The TF-IDF representation uses word-level bigrams, while F2LLM-v2-80M uses an ℓ2-normalized end-of-sequence hidden state as the sentence embedding.

3. Experimental results … 3.4. Ablation study

Experiments selected PhoBERT as the teacher, with Cosine distillation providing the strongest base-student results and outperforming supervised baselines. Ablations show that larger teachers, gated fusion, and linguistic augmentation affect performance differently, with augmentation consistently reducing results.

  • 3.1. Evaluation of teacher models: PhoBERT achieved 89.08% accuracy and 0.8899 F1-score, outperforming mBART’s 86.43% accuracy and 0.8647 F1-score while requiring 13.34 ms versus 342.16 ms per sample.PhoBERT also showed stronger class discrimination for Loose Shell Syndrome, Black Gill Disease, and Taura Syndrome.
  • 3.2. Evaluation of student models: Cosine loss produced the best PhoBERT-base student results, reaching 89.18% accuracy, 0.8935 F1-score, 0.9051 precision, and 0.8925 recall.Conventional distillation losses achieved 87.65%–89.18% accuracy and 0.8773–0.8935 F1-scores, while metric-learning objectives provided no additional benefits.
  • 3.3. Comparison with baseline methods.: Knowledge distillation outperformed supervised fine-tuning, whose internal baseline achieved 86.53% accuracy and 0.8666 F1-score.The supervised baseline distinguished well-separated diseases but showed greater confusion among clinically similar classes.
  • 3.3. Comparison with baseline methods.: TF-IDF + DNN achieved 88.98% accuracy and 0.8896 F1-score, remaining slightly below the Cosine-distilled student’s 89.18% accuracy and 0.8935 F1-score.Its sparse bag-of-words representation captured lexical cues but modeled semantic similarity among clinically related diseases less effectively.
  • 3.3. Comparison with baseline methods.: F2LLM-v2-80M recorded the lowest baseline performance, with 82.86% accuracy, 0.8287 F1-score, and 5.53 ms inference time per sample.It also showed lower classification rates for Filamentous Bacterial Disease, Loose Shell Syndrome, and Black Gill Disease.
  • 3.4.1. Effectiveness of teacher model scale: PhoBERT-large achieved 91.94±0.94% accuracy and 0.9183±0.0097 F1-score, with stable convergence and strong class-level discrimination.Among its distilled students, Center loss attained the highest mean accuracy of 88.16%, F1-score of 0.8822, precision of 0.8894, and recall of 0.8821.
  • 3.4.2. Effectiveness of gated fusion: Removing gated fusion did not consistently reduce performance: JSD achieved 89.29% accuracy and 0.8929 F1-score, while confusion patterns remained largely consistent with the full framework.JSD and Center improved modestly after removal, whereas KL, MSE, Cosine, and SupCon were comparable or slightly lower; Triplet also improved moderately.
  • 3.4.3. Effectiveness of linguistic augmentation: Linguistic augmentation reduced teacher and student performance: the augmented teacher reached 87.96±0.72% validation accuracy, while its best student, Cosine, achieved 85.71% accuracy and 0.8569 F1-score.The Cosine student’s F1-score decreased from 0.8935 on the original dataset to 0.8569 after augmentation, with Cosine retaining the strongest class-level discrimination.

3.5. Explainability analysis · 3.6. Computational efficiency

The explainability analyses show that distilled students generally preserve teacher-aligned, clinically meaningful features, while attribution strength and predictive performance capture complementary properties. The Cosine student also delivers substantial computational savings and a favorable efficiency–performance balance among transformer-based models.

  • 3.5.1. Qualitative explainability: LIME explanations generally match teacher explanations, with few unmatched samples across distillation objectives and most unmatched cases concentrated in White Feces Disease.KL and Triplet each produce three unmatched samples, JSD produces two, and Cosine and SupCon each produce one.
  • 3.5.1. Qualitative explainability: Across all ten disease classes, Cosine student explanations highlight clinically meaningful disease-specific words and semantic patterns.Examples include “vỏ” for Loose Shell Syndrome, “mang” for Black Gill Disease, and “đuôi” with “đỏ” for Taura Syndrome.
  • 3.5.2. Global explanation using SHAP: JSD has the strongest global feature attribution, with the highest average SHAP value of 0.0212 and the largest SHAP standard deviation of 0.0457.Its SHAP distribution is broadest and has a pronounced right tail, indicating wider variation in attribution magnitudes across samples.
  • 3.5.2. Global explanation using SHAP: Cosine achieves the highest F1-score of 0.8935 despite having one of the lowest average SHAP values, showing that attribution magnitude does not directly determine classification performance.SupCon combines relatively low feature attribution with the lowest F1-score, reinforcing that attribution and performance are complementary.
  • 3.5.3. Keyword deletion faithfulness: LIME keyword deletion supports explanation faithfulness because removing explanation-guided keywords reduces accuracy and predicted-class confidence more than random deletion.The accuracy gap widens as more keywords are removed; JSD produces the largest confidence drop, followed by Cosine.
  • 3.6. Computational efficiency: The distilled student reduces computational demand from 43.49 to 14.50 GFLOPs (66.7%) and GPU latency from 13.34 ms to 3.83 ms per sample.CPU latency falls from 518.01 ms to 175.67 ms, throughput rises from 1.93 to 5.69 samples/s, and model size decreases from 515.01 MB to 304.73 MB.
  • 3.6. Computational efficiency: TF–IDF + DNN is most computationally efficient but has lower predictive performance, whereas F2LLM-v2-80M requires 32.21 versus 14.50 GFLOPs and 5.53 versus 3.83 ms GPU latency relative to the distilled student.The proposed student therefore provides a favorable balance among evaluated transformer-based models, although mobile and resource-constrained deployment remains untested.

4. Discussion

The discussion finds that knowledge distillation enables efficient Vietnamese shrimp disease text classification, with Cosine loss offering a strong performance–explanation trade-off. It also shows that explanations are faithful while noting dataset diversity and deployment limitations.

  • Distillation effectiveness: The 79.88M-parameter distilled student achieves performance comparable to PhoBERT-base while substantially reducing computational cost.PhoBERT-base is identified as a more suitable knowledge source than mBART because it achieves higher predictive performance.
  • Distillation objectives: Cosine loss achieves the best overall classification performance and the highest LIME-based explanation consistency with the teacher.The discussion attributes this advantage to preserving angular similarity between teacher and student representations.
  • Teacher selection: Larger teacher capacity does not consistently improve student performance, suggesting that more complex representation spaces may be harder for lightweight students to approximate.PhoBERT-large has higher standalone performance but does not consistently yield better student results.
  • Baseline comparisons: Knowledge transfer, rather than increased model capacity, explains the distilled student’s gains because supervised fine-tuning with the same architecture consistently underperforms it.TF–IDF + DNN remains competitive but relies on sparse lexical representations that struggle with diseases having overlapping clinical descriptions.
  • Explainability: LIME shows that distilled students generally preserve teacher-like explanation patterns, while keyword deletion causes larger performance degradation than random deletion.Cosine combines high teacher–student explanation consistency with the best overall classification performance.
  • Limitations: The ShrimpCap dataset may not represent the diversity of disease stages, farming conditions, and regional linguistic variations, and deployment on mobile or embedded devices remains unevaluated.The experiments were conducted on desktop hardware.

5. Conclusion

The study presents SALT, an explainable multi-loss knowledge distillation framework for Vietnamese shrimp disease classification from textual symptom descriptions, alongside the ShrimpCap dataset. Its findings show that explanation quality and predictive performance should be evaluated jointly across multiple dimensions.

  • SALT is an explainable multi-loss knowledge distillation framework for Vietnamese shrimp disease classification based on textual symptom descriptions.
  • 979 manually validated samples comprise ShrimpCap, described as the first Vietnamese dataset dedicated to shrimp disease symptom descriptions.
  • JSD achieved the highest average SHAP score despite not producing the best classification performance, indicating distinct attribution patterns across distillation objectives.
  • Keyword deletion experiments supported the faithfulness of generated explanations, motivating joint evaluation of performance, consistency, attribution, and faithfulness.

6. Future work

Future work will broaden the dataset and evaluate the student model in real-world environments to expand applicability and validate its practical advantages.

  • Future research should diversify data collection across farming areas, disease stages, and expression styles.
  • Deploying and evaluating the student model in real-world environments is necessary to confirm its speed advantages.

Declarations

The declarations assign Anh Nguyen Quynh roles spanning project development, implementation, visualization, and original-draft writing. Nguyen Quoc and Luyl-Da Quach contributed to methodology, supervision, validation, and review-editing responsibilities.

  • Author contributions: Anh Nguyen Quynh contributed to conceptualization, data curation, methodology, software, visualization, and original-draft writing.
  • Author contributions: Nguyen Quoc and Luyl-Da Quach contributed to methodology, supervision, validation, and review-editing responsibilities.
Loading 2608.29027v1…