Cross-Dataset Generalization of Bangladeshi Rice Leaf Disease Classifiers: Benchmark, Diagnosis, and Mitigation

Motivation

• Models trained on curated rice leaf collections can degrade substantially under different acquisition conditions, undermining deployment across real-world conditions faced by farmers and extension officers.The mechanisms driving this transfer failure remain incompletely understood; background and lighting are often assumed to confound performance, but have rarely been tested through direct intervention at scale.Source: §1 · pp. 1–2

Study aims

• The study benchmarks three Bangladeshi rice leaf collections and tests transfer-failure hypotheses through segmentation, a self-supervised transformer control, and AdaBN analysis.It also documents frozen data artifacts, prespecified analysis criteria, and decision gates committed before results were observed.Source: §1 · p. 2

Primary findings

• +0.070 F1 (p < 0.001) was the statistically significant improvement from strong augmentation across 18 transfer pairs.The findings also report that masking improves performance directionally but is not conventionally significant, while label-prior mismatch predicts AdaBN harm and Grad-CAM fails to distinguish transfer failures from successes.Source: §1 · p. 2

2.1 Domain Shift in Computer Vision

• Domain shift spans visual, semantic, and label-prior dimensions, and prior work has characterized susceptibility and partial mitigation in deep vision models.The cited literature includes adversarial training as a partial mitigation for domain shift.Source: §2.1 · p. 2

2.2 Plant Disease Classification and Transfer

• Cross-dataset transfer between RiceLeafBD and Dhan-Shomadhan has not been systematically studied, and the mechanisms behind transfer failure remain unclear.The datasets differ in acquisition setting: RiceLeafBD uses controlled white backgrounds, whereas Dhan-Shomadhan was field-collected under natural illumination.Source: §2.2 · pp. 2–3

2.3 This Work

• The paper diagnoses transfer failure through controlled interventions and documents results with reproducibility discipline, rather than proposing a novel architecture or adaptation algorithm.Its stated approach is an evidence-based empirical mechanism study.Source: §2.3 · p. 3

3.1 Datasets

• RiceLeafBD contains about 1,500 laboratory-controlled images, Dhan-Shomadhan about 2,000 field images, and BRRI about 1,900 heterogeneous field images including pest damage.RiceLeafBD uses a white background, while Dhan-Shomadhan has natural illumination and backgrounds.Source: §3.1 · p. 3
• A transfer pair was included only when both datasets shared disease classes; listed directions include RiceLeafBD↔Dhan-Shomadhan and Dhan-Shomadhan↔BRRI.The study specifies six ordered transfer pairs overall.Source: §3.1 · p. 3
• Train, validation, and test splits followed a frozen manifest with no overlap or re-splitting.This split protocol was specified for all data.Source: §3.1 · p. 3

Models

• The CNN backbones were ResNet50, EfficientNet-B0, and MobileNetV2; frozen DINOv2 ViT-B/14 features with logistic regression served as an architecture-independence control.The CNNs used ImageNet pretrained weights from timm.Source: §3.2 · p. 4

Training protocol

• CNN training used source-class inverse-frequency weighted cross-entropy, a three-epoch frozen-backbone head warmup, then up to 30 full-model epochs with validation macro-F1 early stopping.The learning-rate schedule used ReduceLROnPlateau with patience 5.Source: §3.2 · p. 4

Augmentation and evaluation

• Training augmentation applied horizontal flips, 90-degree rotations, and ColorJitter; validation and test inputs were resized to 224 × 224 and normalized without augmentation.A separate strong-augmentation experiment applied the listed spatial and color transforms during training only and tested the frozen backbone and head unchanged.Source: §3.2 · p. 4

3.3.2 Cross-Dataset Transfer Matrix

• Six source-to-target pairs, three CNNs, and three seeds produce 54 trained models evaluated on shared classes.The in-dataset baseline also identifies weak source models that could inflate the cross-dataset gap.Source: §3.3.2 · p. 4, §3.3.1 · p. 4

3.3.3 Strong Augmentation

• The augmentation experiment reruns the transfer matrix, changing only training-time augmentation while holding hyperparameters, seeds, and splits fixed.Source: §3.3.3 · p. 4

3.3.4 Masking Intervention

• Masking isolates leaf regions by zeroing non-leaf pixels, using MobileSAM vegetation-score masks and HSV thresholding on ExG.A stratified 60-image audit applies an 80% per-dataset acceptance gate before masked-image transfer runs at seeds 42 and 2024.Source: §3.3.4 · p. 5

3.3.5 Adaptive Batch Normalization (AdaBN)

• AdaBN updates batch-normalization statistics on the target test set, evaluates on that same set, and relates performance changes to label-prior divergence.Source: §3.3.5 · p. 5

3.3.6 DINOv2 Architecture Control

• The DINOv2 control fits logistic-regression heads to frozen ViT-B/14 features, testing whether transfer failure is CNN-specific or dataset-wide.The regularization parameter C is selected on the source validation split.Source: §3.3.6 · p. 5

3.3.7 Grad-CAM Attribution

• Grad-CAM border-attention enrichment is compared between six correct and six incorrect predictions sampled from cross-domain predictions.Border attention is the mean weight on pixels within 15% of the image boundary; the comparison uses a Mann–Whitney U test.Source: §3.3.7 · p. 5

• Macro-F1 is the unweighted average of per-class F1 scores; accuracy is secondary, and paired and unpaired comparisons use Wilcoxon signed-rank and Mann–Whitney U tests.Source: §3.4 · p. 5

• Frozen results have SHA-256 checksums; the v1 baseline is verified byte-identical, and decision gates and analysis criteria are recorded before results are observed.Source: §3.5 · p. 5

4.1 In-Dataset Performance

• F1 is 0.90 for RiceLeafBD, compared with 0.62 for Dhan-Shomadhan and 0.64 for BRRI in-domain.The text attributes RiceLeafBD's strong result likely to its controlled studio setting and white background, and notes that weaker source models may contribute to the cross-dataset gap.Source: §4.1 · p. 6, Table 1 · p. 6

4.2 Cross-Dataset Baseline

• 0.436 mean cross-dataset macro-F1 compares with a pooled in-dataset mean of 0.719, a 39% relative performance drop.Transfer is asymmetric: BRRI →RiceLeafBD achieves F1 0.60, while RiceLeafBD →BRRI achieves F1 0.37.Source: §4.2 · p. 6, Figure 1 · p. 6

4.3 Strong Augmentation

• +0.070 F1 is the statistically significant mean gain from strong augmentation, with 15 of 18 transfer pairs improving.The per-pair figure shows the largest gain for BRRI →RiceLeafBD (+0.41 F1) and a slight regression for RiceLeafBD →BRRI with MobileNetV2.Source: Figure 2 · p. 7, Table 2 · p. 7, §4.3 · p. 8

4.4 Masking Intervention

• +0.0661 F1 is the pooled SAM masking improvement (n = 36, p = 0.0621), while gains vary across transfer directions.SAM generally surpasses raw performance across seeds; the text characterizes masking as a partial, imperfect intervention, with segmentation errors and cropping patterns as possible confounds.Source: §4.4 · p. 8, Figure 3 · p. 8, Table 3 · p. 8

4.5 Architecture Independence (DINOv2 Probe)

• DINOv2 cross-dataset performance (0.451) is nearly identical to CNN performance (0.436), despite comparable in-dataset performance (0.721 versus 0.719).The result indicates that transfer failure is not specific to convolutional architecture or ImageNet-based pretraining.Source: §4.5 · p. 9, Figure 4 · p. 9, Table 4 · p. 9

4.6 AdaBN Failure and Label-Shift Diagnosis

• AdaBN harms ResNet50 (mean ∆F1 = −0.119) and EfficientNet-B0 (−0.060), while MobileNetV2 shows marginal benefit (+0.013).Label-prior mismatch predicts AdaBN harm (Spearman ρ = 0.621, p = 0.0089); the text relates greater harm to larger label shift and deeper models.Source: §4.6 · pp. 9, 11, Figure 5 · p. 10

4.7 Grad-CAM Attribution Diagnosis (Negative Result)

• Grad-CAM border-attention enrichment does not significantly differ between correct and incorrect predictions (p = 0.462, n = 6 each).Correct predictions have higher mean border attention (0.875 versus 0.765), opposite to the stated hypothesis; most values are below 1.0, and the analysis uses 12 sampled predictions.Source: §4.7 · p. 11, Table 7 · p. 11, Figure 6 · p. 12

5.1 Summary of Findings

• +0.070 F1 (p < 0.001) is the strong-augmentation effect, robust across all 18 transfer pairs and the most actionable finding.Among tested adaptation methods, strong augmentation is the only one with consistent, statistically significant benefit.Source: §5.1 · p. 13, §5.4 · p. 14
• Masking yields pooled ∆F1 = +0.066 (p = 0.062) across two segmentation methods and two seeds, consistent with acquisition-condition shift as a partial contributor.Transfer-pair heterogeneity and possible cropping or segmentation errors make masking an imperfect intervention.Source: §5.1 · p. 13, §5.2 · p. 13

5.2 Mechanisms of Transfer Failure

• AdaBN uniformly harms transfer, and its harm magnitude correlates with source–target label-prior divergence (Spearman ρ = 0.621, p = 0.009).The discussion cautions that adaptation methods may backfire when source and target label distributions differ.Source: §5.1 · p. 13, §5.2 · p. 13, §5.4 · p. 14

5.1 Summary of Findings

• Grad-CAM does not distinguish cross-domain successes from failures at the sample sizes and conditions tested.The discussion cautions that attribution-based diagnostics can show an effect in the opposite direction from the hypothesized one.Source: §5.1 · p. 13, §5.4 · p. 14

5.2 Mechanisms of Transfer Failure

• Weak in-domain F1 (0.62–0.64) for Dhan-Shomadhan and BRRI, versus 0.90 for RiceLeafBD, contributes to the cross-dataset gap; DINOv2 features perform equivalently to CNNs.The discussion attributes the collapse primarily to dataset and acquisition conditions, rather than CNN-specific inductive bias.Source: §5.2 · p. 13

5.3 What Remains Unresolved

• The study leaves field deployment, generalization beyond Bangladesh, and the mechanism behind LODO failure unresolved.The datasets are static image collections; real in-canopy phone photography may add failure modes, and LODO's mechanism is not explored.Source: §5.3 · pp. 13–14
• Why masking varies across transfer directions remains unclear: background importance and systematic segmentation or cropping bias are both possible explanations.The discussion notes that segmentation errors and differing cropping patterns may confound the intervention.Source: §5.3 · p. 14

5.4 Implications

• The discussion recommends strong augmentation among tested strategies and cautions that AdaBN can backfire under label-prior mismatch.It also warns against trusting Grad-CAM for domain-shift diagnosis without adequate sample sizes and appropriate baselines.Source: §5.4 · p. 14

• The study covers only Bangladeshi datasets, so generalization to other regions is not established.Source: §6 · p. 14
• Segmentation-based background removal is an imperfect intervention because segmentation errors and systematic cropping differences confound the background effect.Source: §6 · p. 14
• The masking intervention used two seeds rather than three, leaving variance estimates incomplete relative to the baseline and augmentation arms.Source: §6 · p. 14
• Grad-CAM analysis used only 12 predictions, and correct predictions’ higher border attention reverses the hypothesized direction.Source: §6 · p. 14
• Results apply to static image datasets; in-canopy capture, phone photography, and agronomist-confirmed labels were not evaluated.Source: §6 · p. 14

• Cross-domain macro-F1 averages 0.44 against pooled in-domain performance of 0.72, a 0.28-point gap reflecting multiple interacting mechanisms.Source: §7 · p. 15
• +0.070 F1 improvement (p < 0.001) makes strong data augmentation the most reliable recovery strategy across transfer pairs.Source: §7 · p. 15
• +0.066 F1 from masking (p = 0.062, n = 36) is a directional benefit that does not reach conventional significance.Source: §7 · p. 15
• AdaBN uniformly harms transfer with mismatched label distributions, and harm correlates with model depth (Spearman ρ = 0.621, p = 0.009).Source: §7 · p. 15
• Frozen DINOv2 validation points away from CNN inductive bias as the primary driver, and the results are reproducible through public artifacts and prespecified criteria.Source: §7 · p. 15
• Future work should evaluate field-captured images with agronomist-confirmed labels, multi-region generalization, and adaptation strategies accounting for label shift and acquisition differences.Source: §7 · p. 15