PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Group Splits: Specimen-Level Cross-Validation and Visual Attribution for Remaining-Shelf-Life Regression in Climacteric Fruit

Rovhona Mudau, Jean Frederic Isingizwe Nturambirwe, Clement Nthambazale Nyirenda

arXiv:2610.09726v1cs.CV

TL;DR

Repeated images of the same fruit can make image-level evaluation an incomplete test of remaining-shelf-life prediction on unseen specimens. The paper compares partition protocols and lightweight versus heavier frozen visual backbones, while auditing spatial attribution with Grad-CAM. Observation-level splits report lower RMSE than specimen-disjoint evaluation, while the lightweight backbone is comparably accurate to ResNet-18 with higher throughput.

  • Problem

    Repeated photographs of the same specimen can cross image-level train/test splits, making it important to assess reported RSL accuracy on unseen fruit.

    Source: §I · p. 1

  • Method

    The study evaluates frozen ImageNet-pretrained visual backbones with a lightweight regression head across observation-level and specimen-disjoint protocols, and examines Grad-CAM attributions.

    Source: §I · p. 1, §VI · p. 6

  • Results

    RMSE was 2.37 ± 0.03 d across ten observation-level splits and 3.12 ± 0.11 d under specimen-disjoint 5-fold cross-validation.

    Source: §V · p. 5, §VI · p. 6

  • Takeaways & Limitations

    For this dataset and model configuration, observation-level partitioning gives a substantially more optimistic estimate, while MobileNetV3-Small offers accuracy comparable to ResNet-18 with higher throughput.

    Source: §VI · p. 6

    The backbones are frozen feature extractors, and the findings characterize within-dataset behavior rather than external generalization.

    Source: §VI · p. 6

Abstract

from arXiv · show

Estimating remaining shelf life (RSL) from images could provide affordable decision support for perishable produce, but evaluation protocols can substantially affect reported performance when repeated images are available from the same biological specimen. We use the Hass Avocado Ripening dataset, comprising 8,834 image-RSL pairs from 426 fruits across three storage regimes, to evaluate a frozen ImageNet-pretrained visual backbone with a lightweight regression head. Our contributions are threefold: we quantify the effect of observation-level versus specimen-disjoint evaluation, compare lightweight and heavier visual backbones under specimen-disjoint cross-validation, and examine their spatial attributions using Grad-CAM. Across ten observation-level random splits, the model achieves a mean RMSE of 2.37 days with a standard deviation of 0.03 days, whereas specimen-disjoint 5-fold cross-validation yields a mean RMSE of 3.12 days with a standard deviation of 0.11 days. The corresponding mean coefficient of determination is 0.553. A matched per-specimen comparison confirms higher error under specimen-disjoint evaluation, with a probability value below 0.001 across 426 specimens, showing that observation-level partitioning gives a substantially more optimistic estimate for this dataset and model configuration. Under specimen-disjoint evaluation, MobileNetV3-Small (0.93 million parameters) achieves accuracy comparable to ResNet-18 while providing substantially higher throughput, and Grad-CAM reveals differences in spatial attribution between the lightweight backbones. These results support specimen-disjoint evaluation and attribution analysis when assessing lightweight vision models for longitudinal shelf-life prediction.

I. INTRODUCTION

Repeated photographs of individual fruits can make image-level evaluation look stronger than performance on unseen specimens. This study examines that evaluation gap alongside backbone efficiency and whether models attend to biologically meaningful fruit regions.

I. INTRODUCTION

  • Repeated images of the same fruit can cross image-level train/test splits, making reported performance reflect interpolation rather than generalisation to unseen specimens.The study asks how much accuracy survives when whole specimens are held out.Source: §I · p. 1
  • The paper compares whether heavier backbones improve frozen-feature transfer and whether the most accurate models attend to biologically meaningful fruit regions.It also frames Grad-CAM as a way to examine spatial attribution.Source: §I · p. 1

II. RELATED WORK

  • Shelf-life regression estimates remaining time, unlike ripeness classification, which assigns fruit to a discrete class.The cited precedent trained CNNs on the same longitudinal Hass avocado imagery for ripening-stage prediction and subsequent shelf-life estimation.Source: §II · p. 2
  • Prior fruit studies use differing partition protocols, which address distinct generalization issues and should not be treated as interchangeable.Specimen-disjoint evaluation tests unseen specimens within represented conditions, whereas external validation and condition-level holdout address broader data shifts.Source: §II · p. 2
  • Related work links deployment efficiency to compact MobileNet models and notes that saliency maps can be unfaithful.The discussion motivates quantified attribution and perturbation-based corroboration such as Score-CAM.Source: §II · p. 2

III. MATERIALS AND METHODS

The study constructs an image-level remaining-shelf-life target from daily photographs of Hass avocados under three storage regimes. It retains 8,834 image–RSL pairs from 426 fruits.

A. Dataset and RSL Target Construction

  • 8,834 image–RSL pairs from 426 fruits remain after excluding images beyond end of shelf life and fruits that never reach Ripening Index 4.The source dataset contains 14,710 photographs of 478 fruits.Source: A. Dataset and RSL Target Construction · p. 2
  • RSL is the number of days from image acquisition until the fruit first reaches Ripening Index 4, defined as full spoilage.Only images taken on or before that end-of-shelf-life day are retained.Source: A. Dataset and RSL Target Construction · p. 2
  • The dataset tracks individual avocados with daily photographs under cold storage at 10 °C, T20, and ambient conditions.Models are compared against a constant-mean predictor with a reference RMSE of 4.68 days.Source: A. Dataset and RSL Target Construction · p. 2

B. Model and Training

The pipeline combines frozen ImageNet-pretrained visual feature extractors with a trained regression head and compares image-, condition-, and specimen-based evaluation protocols. The specimen-disjoint protocol keeps all observations from each avocado together when testing on unseen fruit.

B. Model and Training

  • Each model uses a frozen ImageNet-pretrained backbone as a feature extractor and trains only a lightweight two-layer MLP to regress RSL.Training-fold statistics standardize features and targets; predictions are inverse-transformed to days before scoring.Source: B. Model and Training · p. 2

C. Evaluation Protocols

  • The study compares image-level random splitting, leave-one-storage-regime-out evaluation, and specimen-disjoint 5-fold cross-validation.The random image-level split is treated as a leakage-prone diagnostic because images from one avocado may occur in both training and test sets.Source: C. Evaluation Protocols · p. 3
  • Specimen-disjoint folds keep every observation from an avocado together, testing generalization to previously unseen specimens within represented conditions.Out-of-fold predictions are combined across the five held-out folds for all 8,834 observations.Source: C. Evaluation Protocols · p. 3
  • The paired protocol comparison uses identical image rows and averages absolute errors within each avocado before testing.The specimen-disjoint predictions are selected for the matched comparison, with one paired error summary per avocado averaged over ten random repetitions.Source: C. Evaluation Protocols · p. 3

D. Attribution and the Mass-on-Fruit Measure

Grad-CAM heatmaps for the two MobileNet backbones are summarized by the proportion of saliency energy inside an avocado silhouette. The measure gauges spatial concentration on fruit relative to a chance-level reference.

D. Attribution and the Mass-on-Fruit Measure

  • Mass-on-fruit is the percentage of total Grad-CAM saliency energy falling inside the avocado silhouette.It is used to assess whether predictions rely on the fruit rather than the background.Source: D. Attribution and the Mass-on-Fruit… · p. 3
  • A mass-on-fruit value near 0.17 suggests chance-level attention because the fruit occupies about 17% of the frame.Values substantially above 0.17 indicate spatial concentration on the fruit.Source: D. Attribution and the Mass-on-Fruit… · p. 3
  • The silhouette is obtained by thresholding brightness and saturation against the light background.Mass-on-fruit is computed for 40 images at each of four ripening stages.Source: D. Attribution and the Mass-on-Fruit… · p. 3

E. Evaluation Metrics

The section reports image-level RMSE, MAE, and R2, alongside fruit-balanced error metrics that give each avocado equal overall weight. It cautions that R2 can be unstable when held-out target variance is low.

E. Evaluation Metrics

  • Image-level RMSE, MAE, and R2 are reported for held-out image observations.The section defines the predicted RSL, true RSL, mean RSL, and number of held-out observations used in the metrics.Source: E. Evaluation Metrics · pp. 3–4
  • Fruit-balanced RMSE and MAE give each avocado equal overall weight despite differing numbers of longitudinal images.All held-out observations are retained, but each avocado contributes the same total weight.Source: E. Evaluation Metrics · p. 4
  • R2 can become unstable when a held-out fold has little target variance, so RMSE and MAE are more interpretable for heterogeneous storage-condition comparisons.The section particularly favors fruit-balanced variants for those comparisons.Source: E. Evaluation Metrics · p. 4

A. Evaluation Protocol

Evaluation protocol substantially affected reported performance: observation-level random splits had lower error than specimen-disjoint cross-validation. The difference persisted in matched per-specimen comparisons and fruit-balanced metrics.

A. Evaluation Protocol

  • RMSE was 2.37±0.03 d under observation-level random splits and 3.12 ± 0.11 d under specimen-disjoint 5-fold cross-validation.The corresponding R2 values were 0.744±0.006 and 0.553 ± 0.023, respectively; the mean RMSE difference was approximately 0.76 d.Source: A. Evaluation Protocol · p. 4
  • Fruit-balanced RMSE/MAE were 2.42 ± 0.04/1.76 ± 0.02 d for random splits, 3.07/2.30 d for specimen-disjoint cross-validation, and 4.68/3.60 d for leave-one-storage-regime-out evaluation.The protocol effect persisted when each fruit contributed equal total weight.Source: A. Evaluation Protocol · p. 4
  • Matched per-specimen MAE was 1.77 d for random observation-level splits and 2.32 d for specimen-disjoint evaluation, a significant paired difference of 0.55 d (p < 0.001, n = 426).Specimen-disjoint RMSE ranged from 2.96 to 3.27 d across folds, so the higher error was not caused by a single held-out fold.Source: A. Evaluation Protocol · p. 4
  • Leave-one-storage-regime-out RMSE/MAE were 5.92/4.87 d for T10, 3.82/2.85 d for T20, and 3.71/2.85 d for ambient; MAE is more interpretable when held-out target variance is limited.T20 and ambient had mean RSL of approximately 3 d and relatively limited target variance, while T10 had a mean RSL of 7.75 d and the largest absolute errors when held out.Source: A. Evaluation Protocol · p. 4

B. Backbone Comparison

Under specimen-disjoint 5-fold evaluation, the tested heavier backbones did not improve predictive performance over the lightweight alternatives. MobileNetV3-Small paired comparable accuracy with substantially higher throughput.

B. Backbone Comparison

  • MobileNetV3-Small has 0.93 M parameters, throughput of 76 images/s, and R2 = 0.596 ± 0.027 while being at least as accurate as the larger alternatives.The backbone comparison uses identical specimen-disjoint 5-fold evaluation.Source: B. Backbone Comparison · p. 4, Table III · p. 5
  • Differences among the three backbones were small relative to fold-to-fold variability, so greater backbone capacity did not translate into better accuracy in this setting.The authors interpret this as consistent with frozen features and modest data, where the larger network cannot exploit its extra capacity.Source: B. Backbone Comparison · p. 4
  • The authors identify the lightweight architecture as the rational default for the targeted resource-constrained transfer setting, on both accuracy and compute.They note that evidence for heavier backbones under fine-tuning need not apply to frozen transfer.Source: B. Backbone Comparison · pp. 4–5

C. Visual Attribution

Grad-CAM attribution differed between the lightweight backbones despite their predictive accuracy. MobileNetV2 localized attention to the fruit more often at earlier ripening stages, while MobileNetV3-Small maps remained near chance across stages.

C. Visual Attribution

  • MobileNetV2 mass-on-fruit reached 0.44 at stage 1, or 2.6× chance, then decreased monotonically to 0.12 at full spoilage.The authors interpret the decrease as saliency dispersing as spoiled avocados become uniformly dark and lack distinguishing surface characteristics.Source: C. Visual Attribution · p. 5
  • MobileNetV3-Small mass-on-fruit remained near chance at every stage (0.12–0.28), with substantial variance and no discernible trend, localizing to the background.The authors describe it as more accurate but less spatially verifiable than MobileNetV2.Source: C. Visual Attribution · p. 5
  • The authors caution that diffuse maps may reflect Grad-CAM's limits with squeeze-and-excitation architectures rather than the model's evidence, and leave Score-CAM corroboration for future work.They note that saliency maps are not guaranteed to be faithful.Source: C. Visual Attribution · p. 5

V. DISCUSSION

In this repeated-measures dataset, evaluation protocol has a larger effect on reported performance than backbone choice: observation-level partitioning is more optimistic than specimen-disjoint evaluation. The discussion also qualifies the scope of this finding and notes that prediction accuracy, efficiency, and attribution quality need not align.

V. DISCUSSION

  • RMSE was 2.37 ± 0.03 d under observation-level random splits versus 3.12 ± 0.11 d under specimen-disjoint evaluation, a difference larger than any between-backbone difference.The matched comparison found mean per-specimen MAE of 1.77 d versus 2.32 d (p < 0.001; n = 426).Source: §V · p. 5
  • The observed protocol gap is specific to this dataset and model configuration, not a universal correction for other shelf-life datasets.The discussion recommends considering predictive performance, computational efficiency, and attribution behaviour jointly when selecting lightweight models.Source: §V · p. 6
  • Fruit-balanced MAE was 1.76 d under random observation-level assessment and 2.30 d under specimen-disjoint evaluation; fruit-balanced RMSE was 2.42 d and 3.07 d, respectively.The protocol pattern persists when each avocado has identical total weight; MAE and RMSE are easier to interpret than R2 across heterogeneous storage regimes.Source: §V · p. 6
  • MobileNetV3-Small achieved RMSE of 2.97 ± 0.14 d versus 3.09 ± 0.08 d for ResNet-18, despite its smaller size and higher throughput.This comparison applies to the assessed frozen-feature transfer-learning setting; task-specific fine-tuning could benefit a larger backbone.Source: §V · p. 6
  • MobileNetV3-Small had the best prediction performance, but its Grad-CAM maps localized to the avocado less reliably than MobileNetV2’s.The discrepancy’s source is uncertain, so Grad-CAM should be treated as an attribution audit rather than evidence of causal properties; spatially plausible maps may help identify obvious failure modes.Source: Figure 1 · p. 5, §V · p. 6

VI. CONCLUSION

Observation-level partitioning gives a more optimistic estimate than specimen-disjoint evaluation for this dataset and model configuration. The backbone and Grad-CAM comparisons show trade-offs in accuracy, throughput, and spatial attribution, while the findings remain bounded to the study’s setting.

VI. CONCLUSION

  • For this dataset and model configuration, observation-level random partitioning yielded a substantially more optimistic performance estimate than specimen-disjoint 5-fold cross-validation, which yielded 3.12 ±0.11 d.Source: §VI · p. 6
  • MobileNetV3-Small, with 0.93 M parameters, achieved accuracy comparable to ResNet-18 while providing substantially greater throughput.Greater model capacity did not necessarily improve predictive performance under frozen-feature transfer.Source: §VI · p. 6
  • MobileNetV3-Small had the highest predictive efficacy across assessed architectures, while MobileNetV2 produced more consistently localized Grad-CAM visualizations.The Grad-CAM analysis revealed spatial attribution differences not captured by predictive metrics alone.Source: §VI · p. 6
  • The frozen-feature setup, heuristic fruit-silhouette estimate, single produce type, and acquisition setting constrain the findings to within-dataset evaluation behavior.The authors identify fine-tuning, independent produce and imaging conditions, and complementary perturbation-based attribution methods as future work.Source: §VI · p. 6

Related papers

Loading 2610.09726v1…