Source-linked AI summary
Multi-Scale Fruit Capsules: Dilated Convolutions and Dynamic Routing for In-the-Wild Explainable Fruit Recognition
Subhankar Chattoraj, Sawon Pratiher, Samiran Das, Hubert Konik
TL;DR
Fruit recognition in the wild must handle large appearance variation within classes and small visual differences between classes, while pooling-based CNNs discard pose and region-location information. FruitCapsNet uses dilated convolutions inside a capsule front end, followed by dynamic routing and Bayesian hyper-parameter optimisation. Across four benchmarks it outperforms ten fine-tuned transfer-learning backbones at one-third the depth, with the largest margin on PD-19 and whole-fruit Grad-CAM attribution.
Problem
Fruit recognition in the wild involves wide intra-class and narrow inter-class variation, while pooling discards region pose and location.
Method
FruitCapsNet uses dilated convolutions in the capsule front end, dynamic routing for part–whole agreement, and Bayesian optimisation for hyper-parameter selection.
Results
FruitCapsNet beats ten fine-tuned backbones across four benchmarks at one-third the depth, with a +2.67% margin on PD-19.
Takeaways & Limitations
Grad-CAM through DigitCaps attributes decisions to whole-fruit regions rather than edges, including under occlusion, clutter, and multi-object conditions.
Abstract
from arXiv · showhide
The same fruit appears in a bunch, unpicked, peeled, bagged in plastic, or sliced on a dish, so automated fruit classification in the wild (AFCW) must absorb wide intra- class and narrow inter-class variability in shape, size, colour and texture. Convolutional networks route information through pooling, which discards the pose and location of the region of interest and therefore generalises poorly across these presentations. We propose FruitCapsNet, a capsule network whose Fruit Capsules replace the standard convolutional front end with dilated convolutions: the receptive field grows exponentially at constant parameter cost, so each capsule encodes multi-scale context before dynamic routing resolves part whole spatial agreement. Hyper-parameters, including the dilation factor, are selected by Bayesian optimisation rather than grid search. On three public datasets (SMP, FruitsGB, Fruits-360) and a new 19-class, 10,639-image in-the-wild dataset (PD-19), FruitCapsNet exceeds ten fine-tuned transfer-learning backbones at one-third the depth, with the largest margin (+2.7% over the nearest competitor) on the hardest set. Grad-CAM saliency propagated from the DigitCaps layer shows that the improvement comes from attributing decisions to whole-fruit regions rather than to object edges, giving post-hoc evidence that the gain is not a dataset artefact.
I. INTRODUCTION
Fruit classification in the wild remains difficult because fruit appearance varies widely within classes while differing classes can look similar. FruitCapsNet addresses this with dilated capsule front ends, Bayesian hyper-parameter search, and evaluation on challenging in-the-wild data.
- Motivation: Fruit grading, sorting, and cataloguing remain largely manual, labour-intensive, expert-dependent, and difficult to staff.The paper motivates automation for IoT-enabled farms, packing lines, and retail checkout.
- Motivation: Fruit appearance changes with size, shape, colour, texture, ripening, provenance, and presentation, producing wide intra-class variation and narrow inter-class separation.Examples include peeled, sliced, washed, bagged, and on-branch fruit.
- Proposed approach: Dilated Fruit Capsules expand receptive fields exponentially at constant kernel size and aggregate multi-scale context before dynamic routing.Section IV-D isolates the dilated front end from the rest of the architecture.
- Proposed approach: Bayesian optimisation searches margin-loss constants, dilation, and training-schedule parameters instead of relying on hand tuning.The search addresses interacting hyper-parameters in the model and training procedure.
- Results: Grad-CAM propagated from DigitCaps attributes decisions to whole-fruit regions rather than object edges.The paper presents this as post-hoc evidence concerning the source of the performance gain.
II. RELATED WORK
Prior fruit-recognition systems used handcrafted descriptors, non-visible sensing, or conventional deep architectures. The paper positions FruitCapsNet as addressing the absence of dilated convolutions inside capsule-layer front ends.
- Prior approaches: Handcrafted pipelines combine colour, texture, shape, geometric, rule-based, fuzzy, and shallow-classifier features for fruit recognition and grading.Examples include PCA, SVMs, LDA, trees, neural networks, and ripeness rules.
- Prior approaches: Non-visible sensing methods include hyperspectral, spectroscopic, multispectral, terahertz, and reflectance-based systems, but the passage states none is deployable at retail cost.These methods trade cost for invariance according to the cited discussion.
- Deep models: Earlier deep models established CNN and fine-tuned VGG-16 comparisons, while capsule networks used standard convolutions in their front ends.The cited capsule applications span scene classification, remote sensing, fault diagnosis, and apoptosis classification.
- Deep models: FruitCapsNet addresses the stated gap that dilated convolutions had not previously been placed inside capsule layers.Its architecture is an encoder–decoder with three dilated convolutional layers, PrimaryCaps, DigitCaps, and a reconstruction decoder.
B. Dilated Fruit Capsules
FruitCapsNet places dilated convolutions before capsule routing so capsules receive multi-scale spatial context. This expands receptive fields without increasing parameter count or reducing resolution, supporting part–whole agreement in complex scenes.
- Capsule representation: A capsule activity vector encodes class instantiation parameters: length represents class likelihood, while direction carries pose, deformation, and albedo.This distinguishes capsule representations from CNN feature maps that retain neither pose nor part–whole structure.
- Architecture: Three dilated convolutional layers feed PrimaryCaps, followed by DigitCaps and an input-reconstructing decoder.The decoder functions as a regulariser.
- Capsule representation: Each capsule contains a weight matrix encoding part–whole spatial relationships before parent capsules aggregate child predictions.The weight matrix maps a child capsule output into a parent-capsule prediction.
- Dilated Fruit Capsules: Dilated convolution replaces standard convolution by sampling kernel positions with spacing d, expanding the receptive field exponentially at fixed parameter count and resolution.The resulting capsules aggregate context across scales before routing decides part–whole agreement.
- Dilated Fruit Capsules: The expanded context spans the fruit and its immediate background rather than only a local patch.The paper connects this design to occlusion and multi-object conditions in PD-19.
C. Routing and Objective
Dynamic routing replaces pooling for capsule outputs, weighting child predictions according to agreement with parent outputs. Training combines margin and reconstruction losses, with the decoder loss controlled by a weighting factor.
- Routing: Dynamic routing aggregates transformed child predictions into parent capsules using routing coefficients determined by softmax logits.Agreement updates increase a child’s contribution to a matching parent and suppress competing parents.
- Routing: The squashing function ties capsule-vector length to presence probability while preserving orientation.Capsule outputs therefore retain directional information alongside activation magnitude.
- Objective: Training minimises encoder margin loss together with decoder reconstruction loss.The reconstruction term regularises the encoder objective.
- Objective: Tj equals 1 when class j is present, and α prevents the decoder from dominating the combined objective.The objective uses the target indicator to define the class-specific margin-loss terms.
D. Bayesian Hyper-parameter Search
Bayesian optimisation replaces manual hyper-parameter tuning by searching interacting loss, architecture, and training variables with a Gaussian-process surrogate.
- Bayesian optimisation searches four loss constants plus architecture and training variables using a Gaussian-process surrogate.The search includes depth, filter size, dilation, learning rate, batch size, dropout, L2 strength, and ADAM moment decay rates.
A. Datasets
The study evaluates three public datasets and PD-19, spanning increasingly realistic fruit-recognition conditions from isolated specimens to cluttered, occluded, multi-category scenes.
- SMP contains fruit images with varying illumination and pose, including reflection, shadow, and plastic-bag occlusion.
- FruitsGB covers six Indian fruit varieties with good/bad quality labels, yielding 12 classes photographed against diverse backgrounds.
- Fruits-360 provides pre-segmented single fruits on a rotating shaft against a white background, without illumination variation.
- PD-19 contains 10,639 web images across 19 classes, with multiple categories, inhomogeneous backgrounds, occlusion, and varied fruit presentations.Examples include uncut, halved, bagged, plated, and unpicked fruit.
- SMP, FruitsGB, and PD-19 use 0.6 : 0.2 : 0.2 train/validation/test splits, while Fruits-360 uses its official 41,322/13,877 partition.The official Fruits-360 split avoids leakage from consecutive frames of the same rotating specimen.
C. Results
FruitCapsNet achieves near-perfect agreement with ground truth and outperforms ten fine-tuned ImageNet backbones, with its strongest advantage on PD-19.
- κ > 0.98 on all four datasets indicates near-perfect agreement with ground truth, including the uneven-prior PD-19 dataset.
- FruitCapsNet is best on every dataset at depth 7 versus 201 for the transfer-learning backbones, with comparable parameter count.
D. Ablation: Isolating the Dilated Front End
Ablations isolate dilation and Bayesian tuning as complementary contributors, with dilation factor 2 providing the largest gain and factor 3 degrading performance.
- The ablation holds depth, capsule dimensions, routing iterations, and training budget constant while comparing the dilated front end and Bayesian search against standard CapsNet.
- d = 2 accounts for 50%–62% of total improvement on public datasets and 70% (+1.09%) on PD-19.Its benefit is larger in multi-object, cluttered, and occluded scenes than in isolated Fruits-360 specimens.
- d = 3 reduces performance by 0.19%–0.46% points across all datasets despite fixed parameter counts.The reported explanation is gridding artifacts and integration of background noise.
- The Bayesian search identifies d = 2 as optimal after selecting the dilation factor among the searched hyper-parameters.
- Bayesian optimisation adds +0.29% to +0.69% across datasets, while on PD-19 dilation and tuning provide the remaining +1.56% of the total +2.67% gain.An unmodified CapsNet contributes +1.11% of the PD-19 gain over the best transfer baseline at 95.61%.
E. Visual Explanation
Grad-CAM comparisons indicate that FruitCapsNet bases predictions on whole-fruit regions, while DenseNet concentrates saliency on edges. This attribution remains focused on fruit regions under pose, illumination, occlusion, cropping, and multi-object conditions.
- FruitCapsNet attributes decisions to whole-fruit regions spanning texture and boundary, whereas DenseNet saliency collapses onto object edges.
- FruitCapsNet’s attribution stays on the fruit under pose change, illumination change, multiple fruits, cropping, and occlusion.
- +2.67% on PD-19 versus +0.32% on FruitsGB is consistent with accuracy degrading least where spatial structure matters most.
V. CONCLUSION
FruitCapsNet addresses the loss of spatial relations in pooling-based CNNs by combining dilated convolutions, capsule routing, and Bayesian hyperparameter optimisation. Across four benchmarks, it outperforms ten fine-tuned backbones at one-third the depth, with the largest margin on PD-19 and whole-object Grad-CAM attribution.
- Pooling-based CNNs discard spatial relations, whereas dilated capsule layers aggregate multi-scale context before dynamic routing resolves part–whole agreement.
- Bayesian optimisation makes the resulting six-way hyperparameter interaction tractable.
- +2.67% on PD-19 is the largest margin, while gains are negligible on saturated public sets.
- Grad-CAM through DigitCaps shows whole-object rather than edge-local attribution that survives occlusion, clutter, and multi-object frames.
- PD-19 label noise is unquantified, and routing cost scales with capsule count, constraining edge deployment.