A100: A Hybrid Vision Foundation Model for Agricultural Plant Image Analysis

1 INTRODUCTION

• Agricultural plant-image analysis supports crop surveillance, agricultural automation, and farmer decision support, but expertly annotated data are scarce.The introduction motivates self-supervised foundation-model pretraining as an approach for settings with limited annotations.Source: §1 · pp. 1–2
• Existing agricultural foundation models optimized for remotely sensed imagery struggle with fine-grained details in proximal images, while Agri-FM+ uses a small corpus and a pure CNN backbone.The passage identifies limited representational diversity and lack of global-context modeling as constraints of Agri-FM+.Source: §1 · p. 2
• A100 uses a hybrid CNN-Transformer backbone with MBConv stages for local texture extraction and Transformer stages for global context, pretrained on 3 million unlabeled images spanning over 100 crop species.The pretraining corpus is approximately 20× larger than Agri-FM+'s 147K-image corpus.Source: §1 · p. 2
• A100 improves crop identification by 2.56% over ImageNet-12K supervised features under linear probe and PlantDoc by 4.30% over the identical ImageNet-12K-initialized model.The introduction also reports 98.01% crop-classification accuracy with 1% labeled data and a 14.2–20.2-point lead over FoMo4Wheat and LeafVision under the same 1%-label protocol.Source: §1 · p. 2

2 RELATED WORK

• The related-work section reviews SSL strategies, existing vision foundation models, and hybrid CNN-Transformer architectures.Source: §2 · p. 2

2.1 Self-Supervised Learning for Plant Image Analysis

• Agricultural SSL work includes plant-classification models and LeafVision's six-backbone comparison, while A100 combines DINO self-distillation, a 3M-image corpus, and evaluation across three tasks.The passage says prior agricultural SSL models evaluate individual backbones in isolation and target single-task classification.Source: §2.1 · p. 2

2.2 Agricultural Vision Foundation Models

• Agricultural foundation models include wheat-focused FoMo4Wheat and other vision and multimodal systems, while A100 uses RGB images with a hybrid CNN-Transformer architecture pretrained on 3M images.The passage notes that multimodal foundation models require textual descriptions that are rarely available in the field.Source: §2.2 · p. 2

2.3 Hybrid CNN-Transformer Architectures

• Existing agricultural hybrid models are fully supervised, task-specific classifiers trained on small datasets; A100 introduces a self-supervised hybrid foundation model pretrained on 3M images.The paper describes A100 as the first self-supervised hybrid CNN-Transformer foundation model for agricultural vision.Source: §2.3 · p. 2

3 METHODOLOGY

• A100 analyzes proximally sensed plant images with a hybrid CNN-Transformer backbone trained via multi-objective self-supervised SADG pretraining.Source: §3 · pp. 2–3
• Figure 2 shows representative samples for self-supervised pretraining, crop classification, disease detection, and plant phenotyping.Source: Figure 2 · p. 3
• The methodology describes datasets, problem formulation, model architecture, SADG pretraining, and task-specific fine-tuning.Source: §3 · p. 3

3.1 Datasets and Data Preparation

• The pretraining corpus contains approximately 3M images from 17 sources spanning 100+ crop species, disease symptoms, growth stages, and field conditions.Approximately half consists of proprietary in-field images collected across farming regions in India; the rest comes from public datasets.Source: §3.1 · p. 3
• Approximately 90K duplicate images were removed using filename and file-size heuristics, leaving a deduplicated corpus of approximately 3M images.The images were organized into 380 WebDataset TAR shards of approximately 8K images each.Source: §3.1 · p. 3
• Downstream evaluation uses 1,142,326 crop images across 32 classes, 51,994 disease images across 44 classes, and approximately 88K GroMo25 multiview images with leaf-count annotations.GroMo25 covers four crops and images across multiple growth stages; external disease benchmarks include PlantDoc, PlantVillage, and PlantWild.Source: §3.1 · p. 3
• Crop and disease datasets use fixed-seed, stratified 80/10/10 splits, while GroMo25 uses its official split and external benchmarks use published splits.The 1% and 10% label subsets come only from training data, and downstream evaluation images were excluded from the pretraining corpus.Source: §3.1 · p. 3

• The task setup first pretrains the model with self-supervised learning and then fine-tunes it for a specific task.Source: §3.2 · p. 4

3.2.1 Self-supervised pretraining

• The pretraining objective is to produce informative representations for downstream tasks from the unlabeled corpus.Source: §3.2 · p. 4
• A feature encoder is pretrained on approximately 3 million unlabeled images using SADG to learn representations without labels.Source: §3.2 · p. 4

3.2.2 Fine-Tuning

• Crop identification uses a classifier that maps pretrained image representations to one of 32 crop classes.Source: §3.2.2 · p. 4
• Disease detection uses a classifier over 44 disease classes.Source: §3.2.2 · p. 4
• Leaf counting uses a regression head to predict a continuous leaf-count value from the pretrained representation.Source: §3.2.2 · p. 4

3.3.1 Hybrid Design.

• The hybrid design addresses fine-grained patterns and whole-image context through different stages of the backbone.Source: §3.3.1 · pp. 4–5
• S0–S1 use MBConv blocks for local features, while S2–S3 use Transformer blocks for global context.Source: §3.3.1 · pp. 4–5

3.3 A100 Architecture

• The backbone processes 224×224 images through four stages, ending with two Transformer blocks at 7 × 7 resolution.S0 and S1 contain MBConv blocks; S2 and S3 contain Transformer blocks.Source: §3.3.1 · p. 5
• Global average pooling of S3 yields a 1024-dimensional representation, and the backbone has 73M parameters.Source: §3.3.1 · p. 5

3.4 Stage-Aware Dual-Granularity (SADG) Pretraining

• SADG combines complementary self-supervised objectives with agriculture-specific design choices to learn representations at scale.The framework uses DINO, iBOT, and KoLeo objectives on a hybrid CNN-Transformer backbone.Source: §3.4 · p. 5

3.4.1 DINO Self-Distillation (Global).

• The DINO objective learns global image representations through cross-view consistency between a teacher and student.The teacher uses EMA-updated soft targets, and training uses Sinkhorn-Knopp normalization.Source: §3.4.1 · pp. 5–6

3.4.3 KoLeo Uniformity Regularization (Global).

• KoLeo uses a nearest-neighbor entropy estimator to encourage uniform coverage of the representation space and prevent feature collapse.Source: §3.4.3 · p. 6

3.4.4 Projection Heads.

• Separate projection heads produce the distributions used by the DINO and iBOT objectives.The iBOT head applies 1×1 convolutions across the 14×14 feature map; both heads are discarded after pretraining.Source: §3.4.4 · p. 6

3.4.5 Combined SADG Objective.

• The combined SADG objective adds DINO and iBOT losses to a KoLeo loss weighted by λ_k = 0.1.The objectives provide global and local supervision while KoLeo maintains uniform feature dispersion.Source: §3.4.5 · p. 6

• AdamW training used a cosine learning-rate schedule, a 1×10−4 base rate scaled by effective batch size, and ten warm-up epochs.The full learning rate was applied to the backbone for adaptation from ImageNet-12K initialization.Source: §3.5 · p. 6
• The augmentation pipeline used two 224×224 global crops, reduced Gaussian blur, ±15° rotation, and stronger color jitter on the student view.Crop scales were 0.5–1.0; weaker teacher-view jitter provided cleaner targets.Source: §3.5 · p. 6
• Pretraining used eight NVIDIA H200 GPUs with distributed data-parallel bfloat16 mixed-precision training and an effective batch size of 2048.Source: §3.5 · p. 6

3.6 Task-Specific Fine-Tuning

• Fine-tuning adds a LayerNorm, Dropout, and Linear classification head, first freezes the feature extractor, then unfreezes it for end-to-end optimization.Training uses AdamW, a cosine schedule, label smoothing, MixUp, CutMix, and validation-accuracy early stopping; class-imbalanced datasets use a weighted sampler and class-weighted cross-entropy.Source: §3.6 · p. 6
• The linear probe freezes the backbone and trains only a linear head to measure pretrained representation quality without task-specific adaptation.Source: §3.6 · pp. 6–7
• Table 3 compares A100 with ImageNet-12K across four evaluation protocols using the same architecture but varying supervision.Source: Table 3 · p. 7
• Figure 4 covers crop identification and disease detection at 1% and 10% labels, alongside fully supervised and frozen-backbone setups.Source: Figure 4 · p. 7
• Table 4 reports test accuracy on external benchmarks; the cited note states that all benchmarks were excluded from A100 pretraining.Source: Table 4 · p. 7, §3.6 · p. 7

4.1 Comparison with State-of-the-art

• At 1% labels, A100 reaches 98.01% crop-identification accuracy vs 95.20% and 86.10% disease-detection accuracy vs 83.90%; with full fine-tuning, results are near-identical.With full fine-tuning, crop accuracy is 99.90% for both initializations, and disease accuracy is 99.02% vs 99.00%.Source: §4.1 · p. 7

4.2 Comparison with Agricultural Foundation Models

• At 1% crop-identification labels, A100 leads FoMo4Wheat by 14.2 points and both LeafVision models by about 19–20 points.The existing agricultural models also score below the ImageNet-12K initialization in this setting; their macro-F1 ranking is identical.Source: Table 5 · p. 7, §4.2 · p. 8

4.3 Label Efficiency

• With 1% labels, A100 achieves 86.10% disease-detection accuracy versus ImageNet-12K’s 83.90%, using 418 images total, about 9.5 per class.At 10% labels, crop-identification accuracy is 99.72% vs 98.69%, and disease-detection accuracy is 95.94% vs 94.82%.Source: §4.3 · p. 8

4.4 External Benchmark Evaluation

• On PlantDoc, A100 reaches 64.89% accuracy, exceeding ImageNet-12K (60.59%) and CvT-13 (63.56%).A100 also reaches 99.96% on PlantVillage, versus CvT-13 at 99.73%, and 76.96% on PlantWild, versus ImageNet-12K at 75.75%.Source: §4.4 · p. 8

4.5 Representation Quality: Linear Probe Analysis

• Frozen-backbone linear probing gives A100 95.37% crop-classification accuracy and 96.76% disease-detection accuracy.The linear-probe gains over ImageNet features are +2.56% for crop identification and +0.97% for disease detection.Source: §4.5 · p. 8

4.6 Beyond Classification: Plant Phenotyping (Leaf Counting)

• On GroMo25 leaf counting, A100’s linear-probe RMSE is 7.28 versus ImageNet-12K’s 8.13, a 10.4% lower error.A100 also reduces RMSE by 16.4% at 1% labels (12.20 vs 14.59) and retains a full-fine-tuning advantage (7.23 vs 7.42); GroMo25 uses controlled-condition images.Source: §4.6 · pp. 8–9, Figure 5 · p. 9

4.7 Discussion

• Holding architecture fixed, A100’s agricultural pretraining adds +2.81 points at 1% labels, +2.56 in linear probing, and +4.30 on PlantDoc.Comparisons with narrower public agricultural-model corpora indicate that corpus breadth, rather than agricultural pretraining alone, is the dominant factor for multi-crop field recognition.Source: §4.7 · p. 9
• A100’s practical deployment claim is production-grade accuracy (>98%) with fewer than 300 labeled images per class.The study evaluates only CoAtNet-2, lacks component-wise ablations of DINO, iBOT, and KoLeo, and deduplicates images using file-name and file-size heuristics.Source: §4.7 · p. 9

5 CONCLUSION

• 99.90% crop classification and 99.02% disease detection with 73M parameters are comparable or superior to an ImageNet-12K supervised baseline across tasks.These are fine-tuned results reported in the conclusion.Source: §5 · p. 9
• +2.56% on crop identification and +0.97% on disease detection in linear-probe evaluation show the frozen backbone’s representation is superior without fine-tuning.The conclusion also reports 10.4% lower GroMo25 leaf-counting error over ImageNet.Source: §5 · pp. 9–10
• On external benchmarks, A100 improves all baseline models on PlantDoc by +4.30% over ImageNet-12K and on PlantWild by +1.21%, with 99.96% on PlantVillage.The conclusion also reports a 14.2-point lead over FoMo4Wheat with only 1% of labels.Source: §5 · p. 10
• The authors plan to release pretrained A100 weights and extend the model to multi-label settings, additional crop types, and vision-language capabilities for zero-shot diagnosis.The stated aim is to support agricultural researchers adapting the model, including with limited annotated data.Source: §5 · p. 10