Source-linked AI summary
Lightweight Vision Transformer Compression for On-Device Plant Disease Detection in Resource-Constrained Agricultural Field Conditions
Mahadev Sunil Kumar, Bhavika Gondi, Desaisetty Venkata Satya Sai Swapnith, Gangireddy Rahul Jogi, Sudheesh Manalil, Arnab Raha, Amitava Mukherjee, Parthasarathy Seethapathy, G. Gopakumar
TL;DR
Plant-disease ViTs are accurate but too large for resource-constrained agricultural devices, and prior compression studies leave combined-method interactions insufficiently explored. The paper evaluates and sequentially integrates Hessian-aware pruning, quantization, and attention-based distillation, achieving substantial compression while preserving baseline-level accuracy. It also compares the pipeline with a same-size directly trained student and identifies hardware validation as a study boundary.
Problem
High-accuracy Vision Transformers are difficult to deploy on resource-constrained agricultural devices, while combined pruning, quantization, and distillation remain insufficiently evaluated for this setting.
Method
The paper independently evaluates compression techniques, then integrates H-BAC Hessian-aware block pruning, quantization, and attention-based knowledge distillation in a sequential deployment pipeline.
Results
54.5x size reduction, from 327.42 MB to 6.01 MB, was achieved by the integrated pipeline at 95.13 +/- 2.32% accuracy across four tested configurations.
Takeaways & Limitations
The study shows that combined compression can produce near-baseline agricultural ViT models, while a same-size directly trained student provides a relevant comparison for judging added compression complexity.
Takeaways & Limitations
Inference measurements used an Apple M4 Pro CPU and NVIDIA RTX 4060 GPU, while validation on actual ARM smartphone hardware remains outside the study's scope.
Abstract
from arXiv · showhide
Chilli (Capsicum annuum) is one of India's most economically significant crops, yet its productivity is persistently threatened by diseases that are difficult to identify without expert intervention. While Vision Transformers (ViTs) have achieved high classification accuracy, their large computational footprint makes deployment on resource constrained devices challenging. Existing compression approaches typically address pruning, quantization, and knowledge distillation in isolation, leaving the potential benefits and interactions of their combined application insufficiently explored. We propose a unified Vision Transformer compression framework that combines Hessian-Balanced Adaptive Block Pruning (H-BAC), guided by second-order sensitivity estimation, with quantization and attention-based knowledge distillation. To systematically identify the most effective configuration within each compression family, each technique is first evaluated independently through controlled ablation studies, after which the best-performing components are integrated into a sequential deployment pipeline tailored to real-world agricultural constraints. On a chilli 3-class village-split dataset with a genuine cross-village, cross-device out-of-distribution test split, the resulting compressed models match or exceed the 95.13% FP32 baseline's accuracy, alongside 74-98% model size reduction, and the fully integrated compression pipeline achieves a 54.5x size reduction (327.42 MB to 6.01 MB) at 95.13 +/- 2.32% accuracy across four tested configurations. A direct comparison further reveals that, on this dataset, a directly-trained student of the same final size, without pruning or distillation, reaches comparable accuracy of 94.87%, at the same 6.01 MB INT8 size, indicating where H-BAC and knowledge distillation are, and are not yet shown to be, worth their computational cost.
1. Introduction
Chilli diseases threaten a major agricultural crop, while expert diagnosis is too slow and costly for rural-scale deployment. The paper addresses this gap by combining curvature-aware pruning, attention-based distillation, and quantization for ViT compression under field constraints.
- Motivation: Chilli diseases can cause losses ranging from 20% to total crop failure, while manual or laboratory diagnosis is too slow and costly for rural farming.Early-stage diseases may also look visually similar, increasing the need for automated field detection.
- Motivation: 327 MB and nearly 28 ms per CPU inference make a standard ViT-B/16 incompatible with low-end mobile devices used by many rural farmers.The paper frames the gap between laboratory accuracy and field deployability as the central deployment problem.
- Research gap: Existing pruning, quantization, and distillation studies largely treat the techniques independently and often overlook transformer-block sensitivity and real-world deployment constraints.Conventional distillation can also suffer feature mismatch across heterogeneous architectures.
- Contributions: H-BAC combines Hutchinson-estimated block-level Hessian curvature with first-order Taylor-based component pruning to adapt pruning across ViT blocks.The method is presented as a curvature-aware adaptive block-pruning approach.
- Contributions: Attention-based knowledge distillation transfers relational reasoning through attention maps, bypassing feature-dimension mismatch between teacher and student models.This design addresses a limitation of conventional feature-based distillation for heterogeneous architectures.
- Contributions: The paper provides a joint evaluation of pruning, quantization-aware training, and knowledge distillation for ViT compression under agricultural deployment constraints.The techniques are systematically compared rather than evaluated only as isolated interventions.
2. Related work and research gaps
Related work established the accuracy potential of CNNs and ViTs but left efficient, field-ready deployment insufficiently addressed. The paper identifies a gap in co-designing compression methods for agricultural ViTs, especially around block sensitivity and feature mismatch.
- Related work: Traditional hand-crafted features and earlier hybrid methods were unable to generalize reliably under actual field conditions.Deep learning subsequently became the dominant approach for plant-disease image classification.
- Related work: Vision Transformers model local and global leaf context through self-attention, supporting plant-disease detection but increasing deployment complexity.The cited motivation is that infection patterns must be related to the overall leaf structure.
- Efficient deployment: 323 MB full-scale ViTs can reach 96% accuracy yet remain unsuitable for resource-limited settings, showing that architectural efficiency alone is insufficient.The related work therefore motivates post-training compression methods.
- Compression methods: Prior compression research spans pruning, quantization, and knowledge distillation, including Hessian-based structural saliency and sequential combinations of techniques.These studies establish relevant components but do not resolve the agricultural ViT setting addressed here.
- Research gaps: The research gap is systematic co-design of pruning, quantization, and distillation for agricultural ViTs under changing light, class imbalance, and farmer-facing constraints.Existing work also neglects differences in transformer-block sensitivity and feature-space mismatch between substantially different teacher and student models.
- Research gaps: The proposed pipeline uses second-order Hessian curvature for non-uniform block pruning and attention maps to transfer relational reasoning across mismatched feature dimensions.This design directly targets the two stated gaps in conventional ViT pruning and feature-based distillation.
3. Methodology
The study evaluates chilli disease detection under village- and device-separated field conditions, using a ViT-B/16 baseline and a compression workflow combining pruning, quantization, and distillation. H-BAC allocates pruning according to transformer-block curvature, while the integrated pipeline progressively reduces model size under accuracy constraints.
- Dataset and preprocessing: 17,655 training images and a 760-image OOD split span distinct villages and smartphone devices, testing cross-domain generalization.All reported accuracy, precision, recall, and F1 results are measured on the OOD split.
- Baseline model: 95.13% OOD accuracy establishes the fine-tuned ViT-B/16 baseline, but its 85.80M parameters limit suitability for low-computation smartphones.The baseline uses an ImageNet-1K-pretrained ViT-B/16 adapted to three chilli disease classes.
- H-BAC pruning: H-BAC assigns non-uniform pruning rates using second-order Hessian curvature and performs first-order Taylor component pruning within each transformer block.The method is designed to reduce ViT FLOPs while preserving classification accuracy for edge deployment.
- Quantization: 94.34% accuracy and 84.42 MB are reported for PTQ-Dynamic, which is selected over PTQ-Static at 94.61% and 85.78 MB because the difference is within run-to-run noise.PTQ-Dynamic is used consistently in the subsequent pipeline for simplicity.
- Integrated compression pipeline: 54.5× size reduction yields a 6.01 MB deployed model after H-BAC, attention-based KD, and PTQ-Dynamic, but the described 50% pipeline stage reaches 91.97% accuracy.The pipeline begins at 95.13% and 327.42 MB, with intermediate KD producing a 21.15 MB student at 93.68% accuracy.
- Integrated compression pipeline: 95.13 ± 2.32% final-stage accuracy across four configurations shows run variability, while direct TinyViT training reaches 94.87% at the same 6.01 MB size without pruning or distillation.The direct-training alternative therefore provides a comparable accuracy-size point with fewer compression stages.
4. Results
Across pruning, quantization, distillation, and integrated deployment tests, compressed ViTs retained near-baseline OOD accuracy while substantially reducing model size or computation. Results also show that recovery fine-tuning stabilizes aggressive pruning, quantization mainly reduces storage on the tested hardware, and direct training can match the smallest pipeline models.
- H-BAC pruning: 91.84–97.37% post-finetune OOD accuracy remained stable across 10–90% H-BAC pruning, including 91.84% at 90% pruning.One epoch of recovery fine-tuning removed nearly all visible accuracy dependence on pruning ratio in this sweep.
- H-BAC pruning: 95.53% post-finetune accuracy versus 93.16% showed curvature-guided allocation outperforming flat allocation by 2.37 points.The comparison used models with 43.70M versus 43.30M active parameters.
- Quantization: ∼74% model size reduction came from both PTQ methods, with accuracy losses of −0.79 points for PTQ-Dynamic and −0.52 points for PTQ-Static relative to FP32.PTQ-Dynamic was adopted for simplicity because it requires no calibration dataset.
- Quantization: 1.00× latency showed no significant INT8 speedup on the tested CPU, with 7.21 ms versus 7.18 ms for FP32 inference.The tested Core ML weight-only INT8 path dequantized weights for matrix multiplication, so quantization primarily reduced storage rather than latency.
- Knowledge distillation: 96.71±1.03% was the highest mean among four distillation recipes, only 0.13 points above the 96.58 ± 0.26% no-KD control.Attention-Based KD had the widest run-to-run spread, so its advantage was not guaranteed in every run.
- Deployment comparison: 94.87% accuracy at the same 6.01 MB INT8 size showed that direct TinyViT training can approach the integrated pipeline without pruning or distillation.The four-run pipeline mean was 95.13±2.32%, whereas the direct-training alternative was 2.89 points above one matched pipeline run.
- Constrained search: 94.08% accuracy at 7.27 MB satisfied every tested size and accuracy budget in the constrained search grid.The search evaluated 10 combinations across 10–30 MB targets and 2% or 3% allowable accuracy drops, with all combinations satisfied.
- Integrated compression: 95.13 ± 2.32% accuracy accompanied the integrated pipeline’s 98.2% size reduction, from 327.42 MB to 6.01 MB, across four configurations.The pipeline achieved a 54.5× compression ratio while matching the FP32 baseline on average.
5. Conclusion
The unified compression pipeline combines attention-based knowledge distillation, structured pruning, and low-precision quantization to reduce Vision Transformer size while maintaining diagnostic accuracy. Its deployment claims are qualified by hardware-dependent latency behavior and the need for target-platform validation.
- 5. Conclusion: The unified system combines attention-driven knowledge distillation, structured pruning, and low-precision quantization for resource-limited edge deployment.The framework is designed to shrink model size and computational footprint while maintaining accurate disease diagnosis.
- 5. Conclusion: 54.5× compression reduces the model from 327.42 MB to 6.01 MB while the integrated pipeline maintains 95.13±2.32% accuracy across four configurations.The pipeline combines all three compression techniques and matches the 95.13% FP32 baseline across the tested configurations.
- 5. Conclusion: INT8 quantization improves model size rather than latency on the Apple M4 Pro because the tested weight-only implementation dequantizes weights before matrix multiplication.The study therefore does not establish that the observed latency behavior transfers to other ARM or GPU platforms.
- 5. Conclusion: Latency-based deployment claims require conversion and validation with ONNX Runtime Mobile or CoreML on the intended target platform.The paper leaves this platform-specific validation to future work.