CropCop: An Auditable 120-Class Plant-Health Model from Benchmark Reconstruction to a Quantised Runtime Artifact

Rana Muhammad Ahmed; Sabahat Abbas

Source-linked AI summary

CropCop: An Auditable 120-Class Plant-Health Model from Benchmark Reconstruction to a Quantised Runtime Artifact

Rana Muhammad Ahmed, Sabahat Abbas

arXiv:2608.25539v1Aug 26, 2026cs.CVcs.LG

TL;DR

Plant-health scores can be misleading when duplicate image families, long-tailed labels, or unevaluated runtime artifacts undermine the evidence. CropCop reconstructs a leakage-controlled 120-class benchmark and traces model states through compact transfer, quantisation, and direct PTE execution, achieving strong internal results while leaving external farms, devices, and acquisition pipelines untested.

Problem
Plant-health recognition lacks sufficient evidence when benchmark leakage, heterogeneous class support, and runtime identity are not controlled together.
Method
CropCop reconstructs a 120-class benchmark and evaluates a DINOv3 reference, a compact MobileNetV4 lineage, and the final quantised ExecuTorch/XNNPACK artifact through paired states.
Results
98.46% accuracy and 96.23% macro-F1 were achieved by the directly executed 22.60 MiB PTE, with six top-1 decisions differing from the converted INT8 graph.
Takeaways & Limitations
CropCop establishes leakage-controlled internal recognition and software-runtime fidelity for a closed-set plant-health classifier.
Takeaways & Limitations
The evidence does not establish performance on unseen farms, regions, cultivars, cameras, acquisition protocols, or physical Android hardware.

Abstract

from arXiv · show

A plant-health score can appear precise while resting on duplicated image families, a long-tailed label space, or a runtime file that was never evaluated. We present CropCop, a closed-set recognition system spanning 120 operational plant-health classes and an evidence chain from corpus reconstruction to direct execution of the final quantised artifact. Starting from 117,546 audited images, we rejected the inherited partition after confirming 3,233 duplicate relationships across split boundaries and froze a 109,107-image benchmark with zero crossings among the audited trusted leakage groups and a 151.7 largest-to-smallest class ratio. A fully fine-tuned DINOv3 ConvNeXt-Tiny reference achieved 98.51% accuracy and 96.87% macro-F1 on the locked internal test. A compact MobileNetV4 Conv-Medium derivative achieved 98.46% accuracy and 96.27% macro-F1 without being presented as evidence for a new distillation method. Validation-only post-training quantisation selected dynamic activations with per-channel weights, and the final 22.60 MiB ExecuTorch/XNNPACK PTE achieved 98.46% accuracy and 96.23% macro-F1 when executed directly. Only six of 16,363 top-1 decisions changed between the converted INT8 graph and the PTE, while paired analysis showed a modest class-balanced loss; an exploratory post hoc fruit-label slice localized a larger recall decline than aggregate accuracy revealed. CropCop establishes strong leakage-controlled internal recognition and software-runtime fidelity; it does not establish performance on unseen farms, camera pipelines, or physical Android hardware.

1 Introduction

CropCop frames plant-health recognition as an evidence-chain problem: benchmark identity, leakage control, compact-model retention, and direct runtime evaluation must remain connected. The study reconstructs a difficult 120-class benchmark and evaluates identifiable model states without extending its claims to external field validity.

Runtime fidelity: CropCop treats checkpoints, compact models, quantised graphs, lowered programs, and runtime files as distinct computational objects requiring paired evaluation.The final runtime claim is tied to one exact PTE identified by its bytes and SHA-256 digest.
Benchmark validity: 117,546 audited images yielded a reconstructed 109,107-image, 120-class benchmark after 3,233 duplicate relationships crossed historical partitions.The final split had zero crossings among the audited trusted leakage groups and retained a 151.7 largest-to-smallest class ratio.
Research questions: The study asks whether duplicate families can be separated, class-balanced performance retained during compact transfer, and quantised execution preserve predictions and ground-truth performance.These questions are deliberately narrower than whether the model works in the field.
Model results: 98.51% accuracy and 96.87% macro-F1 were achieved by the DINOv3 ConvNeXt-Tiny reference on the locked internal test.The compact MobileNetV4 float state retained 98.46% accuracy and 96.27% macro-F1.
Runtime results: 98.46% accuracy and 96.23% macro-F1 were achieved by the directly executed final PTE, with six top-1 decisions differing from the converted INT8 graph.The contribution is an auditable connection of benchmark repair, class-balanced evaluation, compact transfer, validation-only PTQ selection, and direct runtime execution—not a new backbone, loss, or quantiser.
Positioning: CropCop makes transitions jointly observable for one model rather than claiming that earlier literature should implement every stage.The cited literature supplies foundations for data, learning, calibration, and runtime, while CropCop tests whether their outputs remain attached to one identifiable result.

3 Dataset and Benchmark Construction

CropCop reconstructed a heterogeneous 120-class plant-health benchmark by replacing a contaminated historical split, enforcing leakage-group-aware partitioning, and documenting severe long-tail and acquisition-cue risks.

3.1 Source pool and operational ontology: 120 operational labels span specific diseases, broad conditions, and healthy states rather than 120 uniform disease entities.The class map includes 72 specific named diseases or conditions, 9 broad categories, and 39 healthy-state labels.
3.3 Historical partition audit: 3,233 trusted duplicate relationships crossed the historical train, validation, and test boundaries, so the inherited split was discarded.The audit identified 2,876 exact and 357 near-duplicate cross-split relationships.
3.5 Leakage-group-aware split and final freeze: 109,107 images formed the frozen benchmark across 120 classes, with zero crossings among audited trusted leakage groups.The final split contained 76,376 training, 16,368 validation, and 16,363 test images.
3.6 Model-readiness diagnosis and residual risk: 85.39% accuracy versus 68.70% macro-F1 in the frozen DINOv3 probe exposed minority-performance loss before task-specific adaptation.Acquisition-associated probes also exceeded the 0.83% chance accuracy across 120 classes, while split-membership prediction remained near chance.

4 Model and Training Methodology

The methodology separates a fully fine-tuned DINOv3 reference from a compact MobileNetV4 lineage and preserves deterministic, fingerprinted evaluation across model states.

4.1 DINOv3 ConvNeXt-Tiny reference: The DINOv3 ConvNeXt-Tiny backbone was fully fine-tuned with a 120-way classifier at 256×256 resolution.Validation macro-F1 selected the checkpoint, reflecting the observed divergence between accuracy and class-balanced performance.
4.2 Preprocessing and augmentation: Deterministic evaluation fixed EXIF-aware RGB conversion, square padding, bicubic resize, and the frozen normalisation contract.Augmentation remained conservative for disease morphology, using controlled crop, affine, illumination, JPEG, blur, and noise variation.
4.3 Reproducible training execution: Resumable training preserved model, optimiser, scheduler, AMP, EMA, random-number-generator, progress, and configuration-fingerprint state.Startup checks validated dataset, class-map, split, duplicate, and leakage-group identities before continuation.
4.4 Compact MobileNetV4 derivative: The compact state used MobileNetV4 Conv-Medium with a 120-way linear classifier, ImageNet normalisation, and 256×256 input.The archived selected state contains 8,588,232 parameter tensors excluding batch-normalisation running statistics and counters.
4.5 Paired state evaluation: Four states remained separate: DINOv3 reference, float MobileNetV4, converted INT8 graph, and serialised PTE.Paired comparisons used identical locked examples and class ordering, while agreement was kept distinct from accuracy.

5 Quantisation and Runtime Methodology

CropCop selected quantisation on validation data, then directly evaluated the serialised ExecuTorch/XNNPACK artifact to test runtime fidelity rather than inheriting graph metrics.

5.1 Validation-only PTQ selection: Three predeclared XNNPACK-compatible PTQ candidates were evaluated on the full validation split before test access.Selection was pass-first and macro-F1-first, with dynamic activation quantisation and per-channel weights among the candidates.
5.1 Validation-only PTQ selection: The dynamic candidate achieved the strongest validation macro-F1, class-balanced result, and float agreement without calibration rows.The static alternatives included a 2,880-row candidate that passed but was weaker and a 1,440-row candidate that failed the gate.
5.2 Converted-graph evaluation: 99.951% of rows produced identical float and converted-INT8 top-1 predictions, with macro-F1 changing by −0.022 points.The paired 95% confidence interval for converted-minus-float macro-F1 was −0.097 to +0.024 percentage points.
5.3 Artifact serialisation: 22.5986 MiB was the size of the serialised ExecuTorch/XNNPACK PTE, 31.92% smaller than the selected prequantised state file.The comparison is reported as an artifact-size reduction between different containers, not as a theoretical fourfold INT8 compression claim.
5.3 Artifact serialisation: 16,363 direct PTE executions generated a new prediction record, so PTE metrics were measured from the serialised program itself.The PTE was not assigned converted-graph metrics by inheritance.
5.4 Runtime environment and quantisation coverage: Android latency, memory, energy, delegate fallback, and thermal behaviour remained unmeasured.Operator-level delegation details and host-environment records were also incomplete, limiting runtime reproduction claims.

6 Results

CropCop’s reconstructed benchmark supported strong internal performance across the reference, compact, quantised, and directly executed runtime states. Aggregate accuracy was nearly preserved, while paired and class-level analyses revealed modest, concentrated reductions in class-balanced performance.

Model-state performance: 98.51% accuracy and 96.87% macro-F1 were achieved by the DINOv3 ConvNeXt-Tiny reference on the locked internal test.
Model-state performance: 98.46% accuracy and 96.23% macro-F1 were achieved by the final PTE, with macro-F1 0.64 percentage points below the reference.The table compares distinct model states rather than an architecture tournament.
Benchmark reconstruction: 3,233 duplicate relationships crossed historical partitions, motivating a 109,107-image replacement with zero crossings among audited trusted leakage groups.The reconstructed split still retains source bias and ontology ambiguity.
Benchmark reconstruction: 85.39% accuracy and 68.70% macro-F1 from frozen DINOv3 features showed visible label structure before fine-tuning, while metadata, low-frequency imagery, and borders remained predictive.Split membership remained at chance, indicating internal split coherence despite label-correlated acquisition cues.
Paired class-balanced analysis: −0.64 percentage points was the PTE macro-F1 change, with a 95% interval of −1.24 to −0.05 points, while accuracy and balanced-accuracy intervals spanned zero.An exploratory fruit-label slice showed a 6.97-point mean-recall decline and was descriptive rather than preregistered.
Support-bin analysis: 2.56 macro-F1 points and 2.78 points were lost by classes with fewer than 100 and 100–199 training examples, respectively.The three higher-support bins changed by less than one tenth of a point in magnitude, except for a slight improvement in the ≥1,000 bin.
Per-class analysis: 0.7778 to 0.6000 was banana_disease_fruit’s F1 change on 11 test images, illustrating the instability of small-class rankings.Several classes had only 7–30 test examples, so small TP, FP, or FN changes could move F1 sharply.
Runtime fidelity: Six of 16,363 rows changed between converted INT8 and PTE predictions; three corrected INT8 errors, two reversed correct predictions, and one moved between wrong classes.The PTE gained one correct prediction but had macro-F1 0.023 points lower; all six changes were low-margin.

7 Discussion

CropCop establishes traceable retention of strong internal recognition through benchmark reconstruction, compact transfer, quantisation, backend lowering, and direct runtime execution. The remaining evidence is bounded by class-balanced losses, missing matched controls, same-pool acquisition cues, and the absence of source-independent or physical-device evaluation.

What the paper establishes: 98.46% accuracy remained attached to a specific split, class map, model lineage, prediction record, PTE file, and SHA-256 digest.This traceability lets reviewers locate changes through paired rows rather than speculation.
The model result is strong, but not lossless: 0.05 accuracy points separated the PTE from the reference, but macro-F1 declined by 0.64 points with a paired interval excluding zero.Accuracy therefore preserved near-identical row-level correctness while obscuring redistribution across classes.
The model result is strong, but not lossless: Low-support groups and broad fruit-condition labels lost more than high-support groups, while some classes improved.The paper connects these patterns to limited examples and ontology ambiguity, without claiming a single causal explanation for individual errors.
What foundation transfer and teacher guidance do not prove: No matched non-DINO ConvNeXt control or matched direct MobileNetV4 baseline was included, so DINOv3 improvement and causal distillation gain were not isolated.The study describes a teacher-guided training path without claiming a causal distillation gain.
Internal validity and the next evidence boundary: The reconstructed split removes known duplicate-family leakage but does not establish performance on new acquisition domains or physical Android hardware.The next evaluation proposed is a source-independent smartphone cohort followed by actual Android measurements.

8 Limitations

CropCop’s evidence is bounded by internal data, incomplete provenance, historical recipe gaps, single-seed training, and software-only runtime validation. These constraints limit claims about external deployment, causal attribution, reproducibility, and consequential use.

Zero audited leakage-group crossings do not exclude missed duplicates or establish performance on unseen farms, regions, cultivars, cameras, or acquisition protocols.
The archive lacks complete source URLs, annotation authority, cultivar, geography, severity, and redistribution rights, while the 120 labels mix disease specificity and visual target types.
The evidence cannot isolate whether the compact result arose from distillation, feature transfer, or DINOv3 pretraining because matched controls are absent.
The historical mobile run cannot be reproduced exactly because its optimiser, schedule, distillation temperature, and objective coefficients are unavailable in a complete human-readable configuration.
Each final lineage has one archived training seed, so uncertainty covers the fixed test sample rather than retraining variation.
Direct PTE execution establishes software-runtime predictions in the archived host environment, not Android latency, memory, energy, fallback, or thermal behaviour.
The final test is consumed and must not guide new architectures, losses, thresholds, or quantisation recipes; later comparisons require validation-only ablations and a new external cohort.
CropCop is closed-set and unevaluated on unsupported crops, novel diseases, non-plant images, and source-independent field cohorts; consequential use requires uncertainty, abstention, human review, and local expertise.

11 Reproducibility Statement

The reproducibility package preserves the evidence needed to reconstruct the final dataset, model states, metrics, paired analyses, and runtime artifact. Exact historical reproduction of the compact model remains limited by missing training settings.

The evidence package preserves dataset and class fingerprints, group-safe split counts, model hashes, row identifiers, predictions, per-class reports, PTQ records, and direct PTE outputs.
The compact model’s exact historical optimiser and objective coefficients remain the principal reproducibility gap.
The final 22.60 MiB PTE reached 98.46% accuracy and 96.23% macro-F1, with six decisions differing from the converted INT8 graph.
Machine-readable reference, PTE, and paired-delta files preserve class order and full numerical precision.
Table 12 records the model and dataset identity registry.

C.1 Reference-to-PTE paired bootstrap

The paired bootstrap summary compares reference and PTE states across operational metrics, while the disagreement audit identifies prediction changes between converted INT8 and serialized PTE execution.

The paired bootstrap analysis is summarized as a reference-to-PTE state transition.
The reported paired deltas include -0.006433, -0.012386, and -0.000467 across the summarized metrics.
Additional paired deltas include -0.021017, -0.057207, and +0.013036.
The converted-INT8-to-PTE disagreement audit is presented as a separate paired-runtime analysis.

C.3 Reference-to-PTE operational error taxonomy

The operational error taxonomy organizes errors in the reference-to-PTE comparison.

Table 15 presents the reference-to-PTE operational error taxonomy.

D Runtime and quantisation environment

The archived runtime and quantisation environment records the software stack, full quantisation scope, absence of explicit module exclusions, and one lowered backend module with one delegate token.

PyTorch 2.12.1 and torchvision 0.27.1 define the recorded core framework environment.
timm, FlatBuffers 1.0.26, and version 25.12.19 are included in the archived environment.
Full quantisation was requested with no explicit module exclusions.
The lowered backend configuration records 1 module and 1 delegate token.

E Operational class taxonomy and support inventory

The operational taxonomy preserves class order and frozen train/test support across 120 plant-health labels, using three broad label types while flagging pest conditions and quarantining a non-operational candidate.

Operational class taxonomy and support inventory: Table 17 preserves authoritative class order and frozen train and test support for every operational label.The supports agree row-for-row with the per-class metric tables and reference-to-PTE derived tables.
Class inventory: The inventory spans healthy, broad-condition, and specific-disease labels across crops including apple, banana, bean, cherry, corn, grape, and tomato.Examples include apple_black_rot, banana_healthy_fruit, bean_fungal_disease, cherry_powdery_mildew, corn_northern_leaf_blight, grape_esca, and tomato_late_blight.
Three-way naming logic: Labels containing healthy are classified as generic healthy states, while generic fruit, leaf, fungal, and nutrient-deficiency labels are broad condition categories.
Three-way naming logic: Remaining non-healthy labels are assigned to the specific named disease category.
Taxonomy caveats: Pest or arthropod conditions are flagged because the three-way scheme has no separate pest category.Examples include grape mites, tomato leaf miner, tomato spider mites, and walnut gall mite.
Taxonomy caveats: rice_neck_blast was quarantined before the final freeze and is not one of the 120 operational classes.It was a one-image candidate category.