PaperlayerView exact revision on arXiv

Source-linked AI summary

Open-Set Cattle Muzzle Identification: A Leakage-Controlled Benchmark and Evaluation Protocol

Lalit BC, Dharmendra Singh Chaudhary, Shovit Nepal

arXiv:2608.28663v1cs.CV

TL;DR

Existing cattle muzzle-recognition systems largely assume animals are already enrolled, although practical herds also require rejection of unseen animals. This paper introduces open-set gallery-based identification and a leakage-controlled evaluation protocol, and reports that both embedding configurations exceed 90% detection at every operating point.

  • Problem

    Most muzzle-recognition work assumes every query animal is enrolled, leaving open-set identification—recognizing enrolled animals while rejecting unseen ones—a central task for real herds.

    Source: §1 · p. 2

  • Method

    The paper reformulates muzzle identification as an open-set gallery task and evaluates it with identity-disjoint splits, per-fold retraining, held-out threshold calibration, duplicate removal, and bootstrap confidence intervals.

    Source: §1 · p. 2

  • Results

    Both embeddings clear 90% detection at every operating point, and the paper reports reliable rejection at strict false-accept rates using muzzle images alone on its single-herd dataset.

    Source: §1 · p. 3, §5 · p. 12

  • Takeaways & Limitations

    On this single-herd dataset, muzzle images alone supported high open-set performance across two embedding configurations, while the formulation and protocol are the paper's contribution.

    Source: §5 · p. 12

    The evaluation covers a single herd at one site, supporting an internal benchmark but not a claim of cross-farm generalization.

    Source: §3.2 · p. 5

Abstract

from arXiv · show

Reliable individual cattle identification supports disease surveillance, vaccination records, breeding management, and livestock insurance. Although the bovine muzzle provides a stable, non-contact biometric, existing muzzle-recognition systems largely assume a closed set of enrolled animals, limiting their practical deployment. We reformulate cattle muzzle biometrics as an open-set, gallery-based identification problem that can reject previously unseen animals and support incremental enrollment without model retraining. We introduce a leakage-controlled evaluation protocol based on identity-disjoint splits, per-fold retraining, held-out threshold calibration, verified duplicate removal, and bootstrap confidence intervals. We evaluate the framework using two contrasting embedding configurations: a hybrid CNN-ViT metric-learning model and the MegaDescriptor-L foundation model. Under oracle threshold selection, the hybrid model achieves detection-and-identification rates of 98.3%, 96.4%, and 93.6% at target false-acceptance rates of 10^(-1), 10^(-2), and 10^(-3), respectively, while MegaDescriptor-L achieves 99.3%, 98.1%, and 96.1%. However, deployable threshold calibration reveals a substantial difference between oracle and calibrated performance: the hybrid model achieves a false-acceptance rate of 1.03% at a 1% target, whereas MegaDescriptor-L reaches 2.44%. Incremental enrollment further achieves Rank-1 accuracy above 91% with a single reference image and up to 97.3% with eight reference images, without retraining the model or degrading the existing gallery. These results demonstrate that threshold calibration, leakage control, and embedding quality are critical for reliable open-set cattle identification and provide a practical evaluation framework for deployment-oriented animal biometric systems.

1 Introduction

Cattle muzzle recognition is typically evaluated as a closed-set task, although farms must also reject unenrolled animals and enroll new ones. This paper reformulates the task as open-set gallery-based identification and introduces a leakage-controlled benchmark protocol.

1 Introduction

  • Muzzle patterns have been reported as individually distinctive and relatively stable, while prior deep-learning results largely come from closed-set, dataset-specific experiments.The paper contrasts the muzzle with physical identifiers including ear tags, tattoos, brands, and RFID transponders.Source: §1 · p. 2
  • Existing muzzle-recognition studies largely assume every query animal is enrolled, so they return a known identity rather than reject an unseen animal.New purchases, transferred animals, missing animals, and unenrolled calves make open-set identification relevant to real herds.Source: §1 · p. 2
  • Open-set identification requires known identities to cluster tightly, unknown animals to remain outside them, and a threshold between them, motivating metric learning rather than classification.The authors note that a plain cosine score suffices given a well-trained embedding.Source: §1 · pp. 2–3
  • Prior work had not evaluated muzzle-only open-set identification with strict-FAR reporting under leakage control, and the authors report an early protocol defect that reduced DIR at FAR=10−3 to 23.8 ± 20.9%.The early result was attributed entirely to protocol defects, including evaluating unknown animals already seen during model development.Source: §1 · p. 2
  • The paper reformulates muzzle identification as gallery-based recognition that rejects unseen animals and permits enrollment without retraining.This changes the task from selecting among known labels to identifying enrolled animals while supporting later additions to the gallery.Source: §1 · p. 2
  • The primary contribution is a publicly released leakage-controlled protocol using identity-disjoint splits, per-fold retraining, held-out threshold calibration, duplicate removal, and bootstrap confidence intervals.The framework is validated with a purpose-built hybrid CNN–ViT metric-learning model and the MegaDescriptor-L foundation model.Source: §1 · p. 2
  • Both embeddings clear 90% detection at every operating point, and the paper claims the muzzle alone supports reliable rejection at strict false-accept rates.The authors describe the contribution narrowly: a publicly released, open-set gallery-based benchmark protocol for muzzle identification, not the first open-set cattle method or CNN–Transformer hybrid.Source: §1 · p. 3

2 Related Work

Prior muzzle systems report strong closed-set classification, while open-set animal-recognition studies have not combined muzzle-only identification, strict-FAR reporting, and leakage control. The paper situates its approach in open-set recognition and metric learning.

2.1 Muzzle Biometrics and CNN–Transformer Hybrids

  • Muzzle systems have reached high closed-set accuracy, including CNN–Transformer results exceeding 99%, but do not measure rejection of unenrolled animals.The paper targets that missing capability rather than a better feature-fusion module.Source: §2.1 · p. 3

2.2 Open-Set Recognition

  • Open-set recognition methods include OpenMax and losses reserving probability mass for unknowns; a well-trained embedding can also serve as a strong open-set detector.The paper uses DIR@FAR, OSCR, and EER, measures common in face recognition, speaker verification, and person re-identification but not previously used for the muzzle.Source: §2.2 · p. 3

2.3 Open-Set Recognition in Animal Biometrics

  • Prior animal open-set studies report single OSCR or AUROC values on small unknown sets, without DIR@FAR at controlled false-accept rates or a demonstrated leakage-controlled protocol.No prior study combines muzzle modality, open-set identification, strict-FAR reporting, and leakage control.Source: §2.3 · p. 3

2.4 Metric Learning

  • Metric learning places same-identity images close and different identities far apart, making enrollment vector storage and matching a distance comparison.The paper's hybrid combines a CNN and ViT with GeM pooling and a BNNeck head; MegaDescriptor is a Swin-based animal-re-identification foundation model.Source: §2.4 · pp. 3–4

3 Materials and Methods

The method represents muzzle images as embeddings, matches them to enrolled-animal prototypes, and rejects probes below a threshold. Its evaluation uses cleaned data, separate identity pools, and held-out calibration.

3.1 Overview

  • Each image maps to a 512-dimensional unit embedding; an animal is enrolled as a gallery prototype, and probes are accepted above a similarity threshold or rejected as unknown.Adding an animal stores another prototype without retraining.Source: §3.1 · p. 4, §3.3 · p. 5

3.2 Dataset

  • The cleaned database contains 249 identities and 4,643 images after dropping identities with fewer than four images and removing 280 verified duplicate images.It is one herd at one site, so it supports an internal benchmark, not a claim of cross-farm generalization.Source: §3.2 · pp. 4–5

3.3 Open-Set Formulation

  • Probes are scored by maximum cosine similarity to gallery prototypes and returned as the top identity only when that score meets the rejection threshold.The embedding must rank the correct identity first while separating known and unknown probes around the threshold.Source: §3.3 · p. 5

3.4 Hybrid CNN–ViT Embedding

  • The hybrid embedding combines ResNet-50 local-texture features and ViT global-shape features through cross-attention, then uses gated fusion and quality weighting.Its established components provide an embedding for open-set identification rather than a new fusion design.Source: §3.4 · p. 5, Figure 2 · p. 6

3.5 Data Augmentation

  • Training uses CLAHE preprocessing and image augmentations, while inference applies deterministic preprocessing and averages embeddings across five views.Training augmentations include geometric and photometric transforms, degradation noise, random erasing, and CutMix.Source: §3.5 · p. 6

3.6 Training and Scoring

  • The hybrid model uses sub-center AdaFace with center and supervised-contrastive auxiliary losses; MegaDescriptor-L replaces its backbone while keeping the remaining setup identical.At test time, cosine, margin, entropy, combined, AS-norm, and OpenMax scores are compared against prototypes and thresholds.Source: §3.6 · p. 6

3.7 Leakage-Controlled Protocol

  • Identity-disjoint folds use separate known, validation-unknown, and test-unknown pools, with per-fold retraining, held-out threshold calibration, and duplicate removal.The test-unknown pool is reserved for the final report; the protocol also reports DIR@FAR, OSCR-AUC, open-set AUROC, EER, and identity-clustered bootstrap intervals.Source: §3.7 · p. 7, Figure 3 · p. 7

4 Results

Both embeddings attain high open-set detection-and-identification rates across false-accept targets, but held-out threshold transfer differs from oracle performance. Incremental enrollment works without retraining while preserving existing-gallery performance.

4.1 Closed-Set Accuracy Is Saturated

  • Closed-set Rank-1 is 99.53 ± 0.42% for the hybrid and 99.86 ± 0.13% for MegaDescriptor-L, with verification EER of 0.18% and 0.07%, respectively.Both embeddings are described as essentially perfect on the closed-set task.Source: §4.1 · p. 7

4.2 Open-Set Identification

  • With plain cosine scoring and oracle threshold, the hybrid embedding achieves DIR@FAR of 98.3/96.4/93.6% and the foundation embedding 99.3/98.1/96.1% at FARs 10^-1/10^-2/10^-3, respectively.Both embeddings stay above 90%.Source: §4.2 · p. 7, Table 2 · p. 8

4.3 Evaluation Across Embedding Configurations

  • Replacing only the embedding with MegaDescriptor-L reproduces the open-set picture with uniformly stronger numbers, showing the protocol holds across embedding configurations.Enrollment, scoring, and protocol are kept identical.Source: §4.3 · p. 9, Figure 6 · p. 9

4.4 Scoring: Plain Cosine Is Enough

  • Plain cosine is best or numerically closest at every operating point, while OpenMax trails by nearly eight points at 10^-2.A known–unknown feature-norm gap is also present: means 22.7 versus 19.6, Cohen’s d = 2.2.Source: §4.4 · p. 10, Table 3 · p. 11

4.5 Deployment and Consistency

  • With a threshold calibrated on val-unknown and transferred unchanged, hybrid detection is 96.5% at realised FAR 1.03%, while MegaDescriptor-L detection is 98.2% at realised FAR 2.44%.MegaDescriptor-L has better oracle numbers, but its realised FAR is roughly 2.4× the 1% target.Source: §4.5 · p. 11, Figure 9 · p. 11

4.6 Incremental Enrollment

  • With eight enrollment images, Rank-1 reaches 97.1%/97.3%; original-gallery Rank-1 remains above 99.3% and rejection of remaining unknowns above 98.7%.A single enrollment image already yields Rank-1 above 91% for both embeddings, though detection at the operating threshold is lower.Source: §4.6 · pp. 11–12, Table 4 · p. 12

5 Discussion

Muzzle-only embeddings supported open-set identification on this single-herd dataset, but the discussion emphasizes calibration, gallery growth, and scope limits as deployment considerations. The visual analyses show embedding separation and representative failure cases.

5 Discussion

  • Muzzle images alone supported high open-set performance on this single-herd dataset, without establishing a general claim against face fusion.The authors identify the formulation and leakage-controlled protocol, rather than the embeddings, as the contribution.Source: §5 · p. 12
  • MegaDescriptor-L had stronger oracle numbers but worse calibration, making realised FAR under transferred thresholds a key deployment result.The authors argue that on-farm recalibration should be expected rather than optional.Source: §5 · p. 12
  • Plain cosine scoring performed comparably to more elaborate alternatives and scales linearly with the number of gallery prototypes.Scoring against M prototypes requires one M ×512 matrix–vector product and remains feasible into the tens of thousands of identities before approximate search is warranted.Source: §5 · p. 13
  • Previously unseen test identities reached high Rank-1 from few images; gallery expansion essentially left existing identities unaffected, while fixed-threshold realised FAR rose.The authors caution that the incremental-enrollment and gallery-growth experiments do not by themselves establish a full incremental-enrollment claim.Source: §5 · p. 13
  • Evaluation is limited to one herd, camera, and acquisition period, with breed diversity restricted to Angus and two crosses.Identity-disjoint splits do not rule out acquisition-session, background, or camera-signature cues, which were not separately audited.Source: §5 · p. 13
  • Other stated boundaries include FAR=10^-3 resolution, gallery growth tested only to 179 identities in one ordering, and no prospective field deployment.The study also did not evaluate age-related pattern stability, muzzle soiling or moisture, or presentation and spoofing attacks.Source: §5 · p. 13
  • The t-SNE visualization depicts known identities as colored clusters and unknown samples in gray, illustrating the separation relevant to open-set rejection.The figure caption describes the visualization as demonstrating separation necessary for rejection.Source: Figure 11 · p. 13
  • Representative hybrid-model errors include misidentifications and false rejections in cases with poor image quality, occlusion, or extreme poses.The examples are presented as failure modes rather than a quantified error breakdown.Source: Figure 12 · p. 14

6 Conclusions

The conclusion presents an open-set, gallery-based cattle muzzle identification framework and a leakage-controlled protocol. Its results cover oracle and calibrated thresholds, incremental gallery enrollment, and cosine scoring within a single-herd dataset.

6 Conclusions

  • The framework identifies enrolled animals and rejects unenrolled ones, while supporting new gallery enrollment without model retraining.It combines a leakage-controlled protocol with hybrid CNN–ViT and MegaDescriptor-L embeddings.Source: §6 · p. 14
  • Under oracle thresholds, the two embeddings achieved 94–98% detection across false-accept budgets from 10^-1 to 10^-3.With held-out-unknown calibrated thresholds transferred unchanged, detection was 96.5%/98.2% and realised FAR was 1.03%/2.44%.Source: §6 · p. 14
  • Gallery enrollment added previously unseen identities from one to eight images without retraining or degrading the existing gallery.Within this single-herd dataset, muzzle images alone supported open-set identification without face fusion, and plain cosine scoring was strongest or comparable among evaluated methods.Source: §6 · p. 14

8 Author Contributions

The author-contribution statement assigns conceptualization, methodology, original-draft preparation, and validation to Lalit BC, with additional methodology and validation roles for Dharmendra Singh Chaudhary and Shovit Nepal.

8 Author Contributions

  • Lalit BC contributed conceptualization, methodology, original draft preparation, and validation.Source: §8 · p. 14
  • Dharmendra Singh Chaudhary contributed methodology, data curation, data analysis, and validation.Source: §8 · p. 14
  • Shovit Nepal contributed validation, editing, and draft preparation.All authors read and agreed to the published version.Source: §8 · p. 14

9 Funding

The paper reports that it received no external funding.

9 Funding

  • This research received no external funding.Source: §9 · p. 14
Loading 2608.28663v1…