Source-linked AI summary
What Does Animal Re-Identification Learn? Linear Biological Concepts and Their Origins in Visual Representations
Robert Nolting, Alexandra Schild, Moritz Weckbecker, Maximilian Schall, Gerard de Melo
TL;DR
Animal Re-ID can help make large camera-trap collections useful for monitoring, but the biological cues behind model decisions are difficult to interpret. The authors analyze a fine-tuned DINOv3 gorilla Re-ID model and find that biological concepts are represented and used in ways that can be examined through probing, steering, and attribution.
Problem
Re-ID models trained with metric learning receive no biological-concept supervision, leaving it unclear whether they encode biological axes or whether those concepts influence identity predictions.
Source: §2.2 · p. 3Method
The authors probe and intervene on layer-wise activations of a DINOv3 backbone fine-tuned for Western lowland gorilla Re-ID, and compare representations and training-example attributions.
Source: §1 · p. 2Results
Takeaways & Limitations
Interpretability can expose biological structure, how it is used, and potential failure modes, supporting more auditable wildlife-monitoring systems.
Source: §6 · p. 8The analysis focuses primarily on age and sex in gorillas because few wildlife datasets combine many known identities, multiple images per identity, and reliable concept labels.
Source: A. Limitations and Future Work · p. 11
Abstract
from arXiv · showhide
Conservation increasingly relies on camera traps that collect more wildlife imagery than experts can manually analyze, making animal re-identification (Re-ID) essential for monitoring individuals and populations. Yet understanding which cues drive model decisions is challenging for ViT-based Re-ID models, whose metric-learning objectives provide no explicit supervision for biological concepts. We ask whether such models nonetheless organize their representations along biologically meaningful axes. Using a DINOv3 backbone fine-tuned for Western lowland gorilla Re-ID with triplet-margin loss, we find that sex and age emerge as linear directions that generalize to held-out individuals, reaching up to 0.91 AUROC and being recoverable from a single image per individual. Activation steering further shows that the sex direction is causally used by the model, flipping a significant fraction of predictions to the opposite sex. Comparing off-the-shelf and fine-tuned backbones shows that Re-ID training does not create these concepts, but relocates them across the network. Finally, data attribution reveals that the representation we find reflects a graded biological axis, is redundantly encoded across the population and shaped by visually ambiguous individuals. Together, these findings show how interpretability can uncover both the biological structure and failure modes of Re-ID representations, providing a step toward auditable computer vision for wildlife monitoring.
1. Introduction
This study asks whether animal Re-ID models trained only to distinguish individuals encode biologically meaningful concepts, a question tied to understanding whether their predictions rely on sensible cues. In a Western lowland gorilla Re-ID model, the authors find that sex and age are linearly represented and examine how those concepts are used and shaped.
Motivation
- Camera traps collect more wildlife observations than experts can manually analyze, making Re-ID useful for turning imagery into information about individuals and populations.For conservation, understanding the cues behind predictions matters alongside accuracy.Source: §1 · p. 1
Research question
- The study addresses whether metric-learning models trained only to distinguish individuals nonetheless organize representations along biological axes such as sex and age.Prior evidence for linear concepts has mainly come from models trained with explicit class supervision or generative objectives.Source: §1 · p. 1
Approach
- The authors study a DINOv3-based Western lowland gorilla Re-ID model fine-tuned with triplet-margin loss, testing concept directions, interventions, backbone geometry, and training-example attribution.Their analyses ask whether concepts generalize to unseen individuals, are used by the model, and are shaped by particular examples.Source: §1 · p. 2
Findings
- Sex and age emerge as linearly recoverable concepts, with directions that form graded biological axes and are redundantly encoded across the population.Visually ambiguous blackbacks disproportionately shape the sex direction, although removing any individual has little effect on held-out performance.Source: §1 · p. 2
- Activation steering shows that the sex direction is functionally implicated in retrieval, shifting cross-identity predictions toward the opposite sex.Feature attribution also highlights the nursing infant in some female predictions as a potential shortcut.Source: §1 · p. 2, Figure 1 · p. 2
- Re-ID fine-tuning reshapes where existing concepts reside across the network rather than creating them.Pretrained and fine-tuned representations have similar concept separability, while their directions diverge across layers.Source: §1 · p. 2
2. Related Work
Prior work has advanced deep and transformer-based animal Re-ID, but questions remain about the visual cues behind predictions and whether biological concepts guide retrieval. This paper connects those questions to linear concept representations and data attribution.
2.1. Animal Re-Identification
- Animal Re-ID has moved from hand-crafted patterns toward deep metric learning and transformer-based representations, while the cues driving individual predictions remain an open question.Prior studies report that Re-ID models can rely on spurious correlations, and biological concepts in vision representations have not been explored for their role in re-identification.Source: §2.1 · pp. 2–3
2.2. Linear Concept Representations
- Linear concept research studies semantic directions in neural activations, but whether such concepts emerge and influence predictions under Re-ID metric learning remains largely unexplored.Linear probes test recoverability; activation steering offers a stronger test of whether a direction is functionally relevant to the downstream task.Source: §2.2 · p. 3
2.3. Data Attribution
- Data attribution either decomposes model outputs into training-example contributions or measures prediction changes after examples are removed and the model retrained.The study applies representer-point attribution to ViT concept probes, building on work attributing linear concept probes in a ResNet-18 to ImageNet training data.Source: §2.3 · p. 3
3. Method
The method extracts layer-wise image representations, probes them for biological concepts, and tests whether concept directions affect retrieval. Attribution analyses then examine which training examples and image regions contribute to the learned concepts.
3.1. Residual-Stream Representation
- Residual-stream activations are extracted at each transformer layer and averaged across tokens to form an image-level representation; the study compares pretrained and Re-ID-fine-tuned backbones.The final Re-ID representation is projected into the embedding space used for identity retrieval.Source: §3.1 · p. 3
3.2. Linear Concept Probing
- Linear probes use logistic regression, linear SVM, or a difference-in-means direction to test whether biological concepts are encoded in the activations.Concept separability is evaluated with AUROC on training and held-out test splits.Source: §3.2 · p. 3
3.3. Activation Steering
- Activation steering adds a scaled concept direction to residual-stream activations, then tests whether cross-identity KNN retrieval shifts toward the steered concept.The evaluation excludes same-identity gallery images and uses an embedding-manifold diagnostic to distinguish targeted changes from representation degradation.Source: §3.3 · p. 4
3.4. Concept Attribution
- Representer-point attribution decomposes probe predictions into training-example contributions, which are aggregated by individual and biological class.The constituting set is the SVM support vectors or, for logistic regression, the smallest set accounting for 90% of attribution mass.Source: §3.4 · p. 4
3.5. Counterfactual Data Attribution
- Identity-level leave-one-out retraining removes each individual’s images, refits the probe, and measures the change in concept-prediction performance.Together with representer attribution, this assesses how sensitive predictive performance is to removing an individual.Source: §3.5 · p. 4
3.6. Feature-Level Attribution
- Feature-level attribution uses relevance propagation to localize image regions associated with a learned concept and inspect potential shortcuts.For the ViT, the authors use AttnLRP with CP-LRP to obtain pixel-level relevance maps.Source: §3.6 · p. 4
4. Experimental Setup
The experiments probe biological concepts in a DINOv3 model fine-tuned for gorilla Re-ID, then test how those concepts behave under activation steering and data attribution. The setup uses expert-labeled gorilla images, held-out individuals, and complementary image- and identity-level analyses.
4.1. Dataset and Concepts
- 45,232 images of 132 individually identified Western lowland gorillas form a deduplicated dataset with expert-assigned identity and age–sex labels.The analysis also runs on Jaguar and Polar Bear datasets; the concepts are binary sex (male/female) and age (adult/non-adult).Source: §4.1 · p. 4, Table 1 · p. 5
4.2. Representations and Concept Probing
- At each DINOv3-Base ViT layer, probes and a difference-in-means direction measure concept separability by AUROC under held-out-individual and identity-cross-validation protocols.The model is fine-tuned for gorilla Re-ID with triplet-margin loss; probes are also tested using one randomly sampled training image per identity across 32 repetitions.Source: §4.2 · pp. 4–5, Figure 2 · p. 5
4.3. Activation Steering
- Steering interventions add a unit-normalized probe direction at each layer, then assess cosine-similarity retrieval in the 256-dimensional Re-ID embedding space.Each identity contributes an 80% unsteered gallery and 20% query split; same-identity gallery images are excluded, and the main sweep uses K = 5 across specified steering strengths.Source: §4.3 · p. 5
4.4. Data and Feature Attribution
- Representer-point attribution decomposes probes into influential training images, with identity-level influence checked by leave-one-out retraining and attribution aggregated by age–sex class and identity.Selected high-attribution test–training pairs are further analyzed with pixel-level relevance maps; Figure 3 compares concept separability using the full training set versus one image per individual, with shading for one standard deviation across 32 samples.Source: §4.4 · pp. 5–6, Figure 3 · p. 6
5. Results
Gorilla Re-ID representations encode biological concepts that generalize beyond training individuals, and fine-tuning reshapes existing concepts rather than creating them. Steering, probe scores, and data attribution reveal how the sex direction affects retrieval, reflects a graded biological axis, and is shaped by visually ambiguous individuals.
5.1. Re-ID features encode biological concepts
- Held-out probes reach 0.91 image-level AUROC for sex and 0.89 for age, showing both concepts are linearly decodable in the Re-ID representation.Identity-level AUROC reaches up to 0.97.Source: §5.1 · p. 6
- The sex direction remains recoverable from one image per individual, and similar concept recoverability appears in jaguar and polar bear Re-ID, with performance depending on trait distinguishability.Single-image probes were trained on 46 images across 32 random samplings.Source: §5.1 · p. 6
5.2. Re-ID fine-tuning reshapes biological concepts
- Probe transfer remains strong through most layers but degrades near the output, consistent with fine-tuning reshaping rather than creating biological concepts.Probe-direction cosine similarity falls from around 0.95 in the first layer to 0.25 in the final layer.Source: Figure 4 · p. 6, §5.2 · p. 6
5.3. Steering reveals causal use of the sex direction
- At α = 16, steering flips 24% of queries toward an opposite-sex identity while fewer than 1% leave the defined embedding region.Random-direction steering yields a flip-toward value of <4%; stronger interventions cause representation collapse.Source: Figure 5 · p. 7, §5.3 · p. 7
5.5. Difficult individuals shape the direction, but no individual is essential
- Removing a single individual changes held-out AUROC by up to 0.016, indicating population-level redundancy; constitution and leave-one-out influence are distinct.Some individuals ranked highly by representer-point methods have positive ΔAUROC, meaning their removal would improve the probe.Source: §5.5 · pp. 7–8
5.4. The learned sex representation forms a graded biological axis
- Held-out individuals form a graded sex axis: silverbacks and adult females occupy opposite extremes, while blackbacks cluster between them.This pattern is consistent with visual sex-status classification becoming reliable only with maturity.Source: §5.4 · p. 7, Figure 6 · p. 8
5.5. Difficult individuals shape the direction, but no individual is essential
- Pixel-level relevance usually concentrates on faces and sex-associated anatomy, but sometimes localizes on an infant in images of nursing females, a potential shortcut.Source: §5.5 · p. 8
- Blackbacks are enriched 4× among the sex probe’s highest-attributing examples relative to their dataset share, consistent with their position near the decision boundary.The enrichment is measured among support vectors / k90; prototypical silverbacks and adult females contribute relatively little to defining the boundary.Source: §5.5 · p. 7, Figure 7 · p. 8
6. Conclusion
The conclusion presents biological concepts in Re-ID as structured and functionally relevant, while data and feature attribution expose how ambiguous cases shape representations and where models may rely on shortcuts. These interpretability tools can help make wildlife-monitoring systems more auditable and scientifically trustworthy.
6. Conclusion
- Sex and age emerge as linearly recoverable directions that generalize to unseen individuals and remain detectable from one image per individual.The sex direction is functionally implicated in retrieval, while stronger steering causes representation collapse.Source: §6 · p. 8
- Data attribution reveals a graded sex axis shaped disproportionately by visually ambiguous individuals, though no individual is essential; female-image relevance can also localize on an infant.Constitution and counterfactual influence do not necessarily coincide.Source: §6 · p. 8
- Interpretability tools can expose biological structure and potential failure modes, helping assess what cues Re-ID models use in wildlife monitoring.The paper frames this as a step toward more auditable and scientifically trustworthy systems.Source: §6 · p. 8
A. Limitations and Future Work
Concept-level analysis is constrained by scarce wildlife datasets with enough identities, repeated images, and reliable concept labels, while this study examines only age and sex rather than the broader cue range used in animal Re-ID.
A. Limitations and Future Work
- Concept-level analyses need datasets combining many known identities, multiple images per identity, and reliable concept labels.Few available wildlife datasets satisfy all three requirements.Source: A. Limitations and Future Work · p. 11
- The study centers on gorillas, with jaguar and polar bear checks offering initial evidence rather than broad cross-species validation.The authors identify broader validation across species, datasets, and Re-ID architectures as future work.Source: A. Limitations and Future Work · p. 11
- Only age and sex are analyzed, so richer concept annotations are needed to disentangle them from correlated cues such as body size, morphology, and appearance.Animal Re-ID also uses coat color, texture, markings, and pose.Source: A. Limitations and Future Work · p. 11
B.1. Age Concept
Age probes show graded biological structure, but steering toward the opposite age is less effective than sex steering and becomes substantial only alongside degradation; checks in other species and robustness analyses are limited or supplementary.
B.1. Age Concept
- Age steering is less effective than sex steering, with substantial opposite-age switching emerging only after off-manifold degradation becomes pronounced.Age-labeled individuals form tighter embedding clusters, so steering out of them frequently causes representation collapse; steering effects increase with α.Source: §B.1 · p. 11, Figure 8 · p. 11
- Age-probe logits form a graded axis, with age-ambiguous blackbacks and Adolescent & Juvenile individuals near the decision boundary.Distinctive silverbacks and infants occupy the extremes.Source: §B.1 · p. 11
- Age heatmaps highlight common Re-ID regions, especially the face and other anatomical areas.The figure also shows the same Re-ID-relevant regions for an infant gorilla.Source: §B.1 · p. 11
B.2. Concepts in Representations of Other Species
- Jaguar sex probes reach a peak AUROC of 0.864, while polar-bear leave-one-zoo-out evaluation reaches 1.0 on only two held-out individuals.The polar-bear estimate is preliminary because zoo background is confounded with sex and the held-out set is very small.Source: §B.2 · pp. 11–12, Table 2 · p. 12
C. Additional Analyses
- Robustness analyses use layer 9 because it offers the best steering trade-off between effect strength and performance degradation.The layer also corresponds to mid-to-late layers commonly found most informative in probing studies.Source: C. Additional Analyses · p. 12
C.1. Probe Types and Regularization
Probe and attribution checks show that optimized linear probes agree closely, while parameter-free mean-difference probes recover concepts less well; fine-tuning, steering settings, and attribution-set composition are examined for robustness.
C.1. Probe Types and Regularization
- LR and linear SVM probes track closely, with near-collinear directions in later layers, whereas the sex DiM probe peaks at only 0.77 AUROC.LR and SVM AUROCs nearly coincide across the second half of the network; DiM therefore does not recover the concepts as well as optimized decision boundaries.Source: §C.1 · pp. 12–13, Figure 13 · p. 14
- Changing ℓ2 regularization shifts held-out AUROC by only 0.008 on average in the network’s second half, although it changes which training images enter the probe.Stronger regularization lowers AUROC in the earliest layers, where the representation is already weak.Source: §C.1 · p. 13, Figure 12 · p. 13
C.2. Pretrained versus Fine-Tuned Representations
- Off-the-shelf DINOv3 probes achieve similar AUROC to fine-tuned probes, while their learned directions diverge across layers.The comparisons cover linear separability and cosine similarity between directions.Source: §C.2 · p. 13, Figure 14 · p. 14, Figure 15 · p. 14
C.3. Sensitivity of Activation Steering
- Steering behavior is stable across K values, but age steering requires a larger dose and takes effect only after substantial degradation, unlike sex steering.Off-manifold degradation curves turn upward at α = 32 and saturate near 100% at α = 64 across K values.Source: §C.3 · p. 14, Figure 16 · p. 15
C.4. Robustness of Attribution-Set Composition
- SVM support-vector sets and LR k90 sets contain similar numbers of images and yield similar qualitative results.The comparison concerns attribution-set composition for sex and age concepts.Source: §C.4 · p. 14, Figure 17 · p. 15
C.5. Individual-Level Composition
- After normalizing for image counts, the most selected individuals are enriched only 2 to 3 times over the uniform baseline, and both methods select the same top ten.This is far from the near-exclusive concentration expected if probes relied on memorized identities.Source: §C.5 · p. 16, Figure 18 · p. 16
D. Implementation Details: DualXDA Attribution
DualXDA relevance is propagated through a ViT-specific LRP setup, with image-mean color used as the pixel reference; highly similar Test heatmaps limit image-specific interpretation because of representation geometry.
D. Implementation Details: DualXDA Attribution
- For DINOv3’s Vision Transformer, DualXDA uses AttnLRP with the lxt library and CP-LRP within attention blocks.CP-LRP treats queries and keys as constants and prevents relevance propagation through softmax; the configuration improves relevance conservation on this backbone relative to the full uniform rule.Source: D. Implementation Details: DualXDA… · p. 16
- Pixel-level relevance uses each image’s mean color as the reference point because a black reference produces uninformative maps for dark gorillas against brighter backgrounds.Activations are extracted from unnormalized images with pixel values in [0, 1].Source: D. Implementation Details: DualXDA… · p. 16
- Test-row heatmaps can be highly similar across images, with their sign determined by the concept label.This limitation arises from the geometry of the learned representations.Source: D. Implementation Details: DualXDA… · p. 16
- Aligned support-vector representations produce similar relevance seeds, so similar Test maps are not independent evidence from each support vector.The limited map diversity follows from representation geometry rather than image-specific support-vector evidence.Source: D. Implementation Details: DualXDA… · p. 16