PaperlayerView exact revision on arXiv

Source-linked AI summary

DINOv3

Oriane Siméoni, Huy V. Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timothée Darcet, Théo Moutakanni, Leonel Sentana, Claire Roberts, Andrea Vedaldi, Jamie Tolan, John Brandt, Camille Couprie, Julien Mairal, Hervé Jégou, Patrick Labatut, Piotr Bojanowski

arXiv:2508.10104v1cs.CVcs.LG

TL;DR

DINOv3 addresses the difficulty of scaling self-supervised learning across massive, diverse image collections while preserving dense feature quality. It combines data and model scaling with Gram anchoring and post-training strategies, then distills the resulting 7B model into practical variants. The resulting vision foundation model achieves strong performance across tasks and domains, with especially large gains on dense prediction.

  • Problem

    Scaling self-supervised learning to massive datasets and models while retaining high-quality dense features remains challenging.

  • Method

    DINOv3 scales curated data and model capacity, uses Gram anchoring to mitigate dense-feature degradation, and distills a 7B teacher into smaller models.

  • Results

    DINOv3 achieves state-of-the-art performance across challenging downstream tasks, including COCO detection mAP 66.1 and ADE20k segmentation mIoU 63.0 with a frozen backbone.

  • Takeaways & Limitations

    DINOv3 provides a versatile family of off-the-shelf vision models that generalizes across tasks, domains, resolutions, and resource constraints.

  • Takeaways & Limitations

    Continued training can produce outliers in addition to degrading dense features.

Abstract

from arXiv · show

Self-supervised learning holds the promise of eliminating the need for manual data annotation, enabling models to scale effortlessly to massive datasets and larger architectures. By not being tailored to specific tasks or domains, this training paradigm has the potential to learn visual representations from diverse sources, ranging from natural to aerial images -- using a single algorithm. This technical report introduces DINOv3, a major milestone toward realizing this vision by leveraging simple yet effective strategies. First, we leverage the benefit of scaling both dataset and model size by careful data preparation, design, and optimization. Second, we introduce a new method called Gram anchoring, which effectively addresses the known yet unsolved issue of dense feature maps degrading during long training schedules. Finally, we apply post-hoc strategies that further enhance our models' flexibility with respect to resolution, model size, and alignment with text. As a result, we present a versatile vision foundation model that outperforms the specialized state of the art across a broad range of settings, without fine-tuning. DINOv3 produces high-quality dense features that achieve outstanding performance on various vision tasks, significantly surpassing previous self- and weakly-supervised foundation models. We also share the DINOv3 suite of vision models, designed to advance the state of the art on a wide spectrum of tasks and data by providing scalable solutions for diverse resource constraints and deployment scenarios.

1 Introduction

DINOv3 advances self-supervised vision at scale through data and model scaling, Gram anchoring for dense features, and distillation into practical model variants. It achieves strong cross-task performance, especially on dense prediction, while generalizing across domains without task-specific annotation.

  • Motivation: Self-supervised learning trains on raw, diverse image collections without requiring high-quality metadata, supporting foundation models across domains.The paper highlights applications spanning natural, aerial, medical, biological, astronomical, and other specialized imagery.
  • Motivation: Scaling self-supervised learning remains difficult because useful data collection, optimization horizons, and dense-feature quality become challenging at large scale.Long training can improve global benchmarks while degrading dense-task performance through patch-level inconsistencies.
  • Dense Features: Gram anchoring mitigates dense-feature collapse caused by large models and long schedules, producing cleaner high-resolution maps and better dense representations than DINOv2.The method targets the conflict between high-level semantic objectives and dense geometric feature quality.
  • Model Family: DINOv3 distills a 7B teacher into ViT Small, Base, Large, and ConvNeXt variants, with ViT-L achieving performance close to the original teacher across tasks.The family is designed for diverse resource constraints and deployment scenarios.
  • Scaling SSL: DINOv3 combines automatic data curation, specialized-data mixing, a 7B ViT architecture, modern position embeddings, and constant schedules for 1M iterations.These choices address data scaling, model architecture, and training design without relying on manual annotations.
  • Results: With a frozen backbone, DINOv3 reaches COCO detection mAP 66.1 and ADE20k segmentation mIoU 63.0, while surpassing prior approaches on satellite imagery.The paper reports state-of-the-art results on these dense tasks and comparable performance to CLIP derivatives on global tasks.

2 Related Work

Related work spans self-supervised representation learning, vision foundation models, dense transformer features, and distillation-based approaches. DINOv3 is positioned against efforts to scale unsupervised pretraining and improve local feature quality across resolutions.

  • Self-Supervised Learning: Self-supervised learning trains visual representations directly from images without requiring manual annotations or metadata.
  • Vision Foundation Models: Vision foundation models evolved from supervised ImageNet pretraining toward larger datasets, weak supervision, and multimodal image-text training.
  • Vision Foundation Models: Scaling unsupervised image pretraining has received less attention than supervised and weakly supervised scaling, with DINOv2 representing a major prior advance.
  • Dense Transformer Features: Dense transformer features support applications including detection, segmentation, depth estimation, 3D understanding, video, robotics, multimodal, and generative models.
  • Dense Transformer Features: Prior methods improve local features through register tokens, fine-tuning objectives, patch sorting, or high-resolution feature-map generation.
  • Dense Transformer Features: Distillation-based agglomerative methods combine encoders trained with supervised, weakly supervised, and self-supervised signals into unified backbones.

3 Training at Scale Without Supervision

DINOv3 scales self-supervised training through larger models and datasets, careful data preparation, and constant-schedule optimization. The section also identifies degradation of dense features during long training as a central challenge motivating the method’s design.

  • DINOv3 targets robust and flexible visual representations by scaling model capacity and training data by an order of magnitude.
  • Data Preparation: Naive data scaling can fail to improve model quality, so DINOv3 uses curation and sampling strategies for its large web-image pretraining dataset.
  • Data Preparation: The data ablation compares clustering, retrieval, raw, and mixed datasets using downstream performance after a shorter 200k-iteration schedule.
  • Large-Scale Training with Self-Supervision: Training uses a discriminative objective combining DINO, iBOT, Sinkhorn-Knopp normalization, and a Koleo regularizer for global and local representation learning.
  • Large-Scale Training with Self-Supervision: As training progresses, patch-to-class and patch-to-patch similarities increase or become noisier, while dense-task performance and feature localization decline.
  • Model Architecture: The model scales to 7B parameters while retaining 40 blocks, increasing embedding dimension to 4096 and using 16-pixel patches.
  • Large-Scale Training with Self-Supervision: Constant learning rate, weight decay, and teacher EMA momentum remove the need to specify an optimization horizon in advance.

4 Gram Anchoring: A Regularization for Dense Features

Extended training improves global metrics but degrades dense-task performance through patch-level inconsistencies. Gram anchoring regularizes patch-feature relationships, rapidly restoring dense performance while preserving global representations.

  • 4.1 Loss of Patch-Level Consistency Over Training: Extended training improves global benchmarks but degrades dense prediction as patch-level inconsistencies emerge.Cosine maps become less smooth and localized after 600k iterations and beyond.
  • 4.2 Gram Anchoring Objective: Gram anchoring matches the student’s patch-feature Gram matrix to an early teacher, preserving similarity structure while allowing individual features to change.The loss uses L2-normalized local features and operates on pairwise dot products between patches.
  • 4.2 Gram Anchoring Objective: The refinement loss is applied on global crops after 1M iterations, with the Gram teacher updated every 10k iterations.Late application can repair substantially degraded local features.
  • 4.2 Gram Anchoring Objective: Gram anchoring significantly improves dense-task performance within the first 10k refinement iterations, while global benchmarks show mild impact.The method also produces notable gains on ADE20k after Gram-teacher updates.
  • 4.3 Leveraging Higher-Resolution Features: High-resolution Gram anchoring distills smoothed, higher-resolution feature correlations into the student and adds +2 mIoU on ADE20k over LRef.The procedure computes the Gram matrix from down-sampled high-resolution teacher features.

5 Post-Training

DINOv3 uses post-training adaptation to support variable resolutions, efficient model sizes, and image-text alignment. These stages retain dense-feature quality at higher resolutions, distill the 7B model into smaller variants, and align global and local visual features with text.

  • 5.1 High-Resolution Adaptation: High-resolution adaptation uses mixed-resolution global-crop pairs to support inference across different input sizes.The base model is trained at resolution 256 before this adaptation phase.
  • 5.1 High-Resolution Adaptation: Gram anchoring with the 7B teacher is essential during high-resolution adaptation because removing it significantly degrades dense prediction.It encourages consistent feature correlations across spatial locations in high-resolution inputs.
  • 5.1 High-Resolution Adaptation: Resolution scaling gives a small ImageNet classification gain and relatively stable performance across evaluation resolutions, while also improving broader resolution generalization.The before-and-after comparison covers ImageNet, ObjectNet, ADE20k, and DAVIS.
  • 5.2 Model Distillation: The ViT-7B teacher is distilled into ViT-S, ViT-B, ViT-L, ViT-S+, and ViT-H+ variants for different compute budgets.The standard variants contain 21M, 86M, and 0.3B parameters; custom variants contain 29M and 0.8B.
  • 5.2 Model Distillation: Parallel multi-student distillation shares teacher inference across nodes and adjusts student groups so training steps have equal duration.This reduces idle synchronization time while training multiple students concurrently.
  • 5.3 Aligning DINOv3 with Text: The text-alignment stage freezes the visual backbone, trains a text encoder contrastively, and concatenates mean-pooled patch embeddings with the CLS token.Two transformer layers are added above the frozen visual backbone to align global and local visual features with text.

6 Results

DINOv3’s frozen representations deliver strong performance across dense, global, 3D, video, retrieval, detection, and segmentation tasks. The results show broad gains over prior self-supervised models and competitive performance against weakly supervised and specialized systems.

  • Overall evaluation: DINOv3 achieves state-of-the-art results with a frozen backbone across object detection, semantic segmentation, 3D view estimation, and relative monocular depth estimation.The evaluation uses lightweight protocols or little additional effort on top of the frozen representation.
  • Dense prediction: 81.1 mIoU on Cityscapes surpasses AM-RADIOv2.5 by 2.5 points and all other backbones by at least 5.5 points.On ADE20k, DINOv3 exceeds self-supervised baselines by more than 6 mIoU points and weakly supervised baselines by more than 13 points.
  • 3D correspondence: DINOv3 improves geometric correspondence recall over DINOv2 by 4.3% and leads on semantic correspondence recall.It also exceeds DINOv2 by 2.6% and AM-RADIO by 1.9% on semantic correspondences.
  • Object discovery: DINOv3 improves object discovery by 5.9 CorLoc on VOC 2007 and outperforms prior self-, weakly supervised, and agglomerative backbones.Its clean, precise feature maps address artifacts associated with DINOv2 in this task.
  • Global recognition: DINOv3 gains +10% on ImageNet-R, +6% on ImageNet-Sketch, and +13% on ObjectNet over DINOv2.It achieves the best corruption robustness on ImageNet-C and remains comparable to weakly supervised models on several classification settings.

7 Evaluating the Full Family of DINOv3 Models

The DINOv3 family spans diverse architectures and compute budgets while maintaining strong dense and global vision performance. Distillation transfers the 7B teacher’s capabilities to smaller ViT and ConvNeXt variants, with additional support for resolution scaling and text alignment.

  • Family overview: The DINOv3 family significantly outperforms other model collections on dense prediction while achieving similar classification results across compute budgets.This positions the family as a broad alternative to specialized model collections.
  • Vision Transformers: ViT variants range from compact ViT-S models for resource-constrained devices to 840-million-parameter ViT-H+ models for demanding applications.The family includes ViT-S, S+, B, L, and H+ models.
  • Vision Transformers: Over 6 mIoU points on ADE20k separate DINOv3 ViT-L from DINOv2, while ViT-B gains approximately 3 mIoU points over the next-best competitor.Depth estimation evaluations also show consistent gains over competing approaches.
  • Distillation: The largest student achieves performance on par with the 8-times-larger ViT-7B teacher.The comparison supports distillation as a way to produce smaller models with comparable performance.
  • ConvNeXt models: DINOv3 ConvNeXt models scale with increased input resolution while supervised ConvNeXts significantly degrade at 512 resolution.At resolution 256, DINOv3 models slightly lag supervised models on in-distribution classification but show significant out-of-distribution gaps in their favor.
  • Text alignment: Text-aligned DINOv3 ViT-L substantially improves over text-aligned DINOv2 on all reported benchmarks and remains competitive in global alignment tasks.Its dense alignment performance is especially strong, while it is slightly behind SigLIP2 and Perception Encoder globally.

8 DINOv3 on Geospatial Data

DINOv3 extends self-supervised learning to satellite imagery and achieves strong results across metric, semantic, and detection tasks. Domain-specific satellite pretraining is especially beneficial for canopy-height estimation, while web pretraining remains effective for semantic geospatial tasks.

  • Satellite pretraining: A DINOv3 7B model trained on satellite imagery demonstrates that the self-supervised recipe transfers to a domain with different textures, sensor noise, and focal views.The satellite model is trained on SAT-493M, containing 493 million 512 × 512 RGB images.
  • Canopy height estimation: DINOv3 satellite models set state-of-the-art canopy-height results, reducing MAE from 2.4 to 2.2 on SatLidar1M validation and from 2.42 to 2.02 on Open-Canopy.The satellite ViT-L performs comparably to the 7B model and reaches MAE 2.4 on Neon test.
  • Earth observation benchmarks: Frozen DINOv3 satellite and web models achieve state-of-the-art results on 12 of 15 Earth-observation classification, segmentation, and horizontal detection tasks.The models use RGB inputs while surpassing prior approaches that use six or more optical bands and task-specific fine-tuning.
  • Earth observation benchmarks: General-purpose self-supervised learning can match or exceed satellite-specific approaches on tasks depending on precise object boundaries.The result supports strong generalization from domain-agnostic pretraining in specialized downstream domains.
  • Domain specificity: Domain-specific pretraining benefits metric tasks such as canopy-height estimation, whereas web pretraining achieves state-of-the-art results on semantic geospatial tasks.The two models therefore show complementary strengths across geospatial applications.

9 Environmental Impact

The paper estimates the energy and carbon footprint associated with DINOv3 training, using fixed assumptions for grid carbon intensity and power usage effectiveness. It also compares DINOv3 with high-resolution geospatial models and reports a qualitative satellite-model comparison.

  • Geospatial evaluation: Table 19 compares DINOv3 with Prithvi-v2, BillionFM, and SkySense V2 on high-resolution geospatial segmentation and detection tasks.The reported metrics are mIoU on LoveDA and iSAID and mAP on DIOR.
  • Geospatial evaluation: On Open Canopy, DINOv3 produces more accurate qualitative maps than Tolan et al. (2024), including tree heights in a field.Both models use decoders trained on 448×448 input images.
  • Environmental impact: 47 MWh is the estimated energy required to train one DINOv3 model.This is roughly equivalent to 240,000 km of driving with an average electric vehicle.
  • Environmental impact: Roughly 2600 tCO2eq is the estimated carbon footprint of the whole project based on 9M GPU hours.The estimate uses a PUE of 1.1 and a US-average carbon intensity factor of 0.385 kg CO2eq/KWh.
  • Environmental impact: The project estimate considers GPU electricity but ignores emissions from cooling, manufacturing, and disposal.This defines the scope of the reported carbon-footprint estimate.

10 Conclusion

DINOv3 combines scaling, Gram anchoring, and post-hoc strategies to improve versatile self-supervised vision models. The conclusion reports state-of-the-art performance across visual tasks without fine-tuning the image encoder, while the section also discusses high-norm outlier mitigation.

  • Conclusion: DINOv3 scales dataset and model size through careful data preparation, design, and optimization, while Gram anchoring mitigates dense-feature degradation over long training.The conclusion presents these as central strategies for advancing self-supervised learning.
  • Conclusion: Post-hoc high-resolution post-training and distillation yield state-of-the-art performance across a wide range of visual tasks without fine-tuning the image encoder.The DINOv3 suite is intended for varied resource constraints and deployment scenarios.
  • High-Norm Patch Outliers: High-norm patch outliers are characterized as high-norm tokens often located in low-information background regions.The section reports that this phenomenon appears in DINOv3 7B and also affects supervised and other self-supervised models.
  • High-Norm Patch Outliers: Four register tokens take over internal communication between patches and the CLS token to mitigate patch outliers.The authors adopt registers for all reported experiments after observing their effectiveness.
  • High-Norm Patch Outliers: Value gating eliminates clear outliers but generally produces higher patch norms, while attention bias does not completely resolve the issue.The best performance is reported with register tokens.

A.2 Feature Dimension Outliers

Large DINOv3 models exhibit outliers across feature dimensions, distinct from patch outliers. These dimensions matter during training but appear largely non-informative at inference, while earlier-layer features require care.

  • Feature-dimension outliers: A small subset of feature dimensions in trained 7B models attains exceptionally large magnitudes across patches and transformer layers.This is a feature-dimension outlier phenomenon rather than an outlier occurring across patches.
  • Feature-dimension outliers: L2 regularization that suppresses feature-dimension outliers causes a performance drop, indicating that these dimensions play a significant role during training.The evidence concerns experiments designed to neutralize the dimensions during training and inference.
  • Feature-dimension outliers: Removing feature-dimension outliers at inference does not significantly change performance, suggesting that they primarily carry trivial or non-informative signals.Batch normalization can also suppress them because their elevated values are consistent across patches and images.
  • Practical handling: Earlier-layer features can be ill-conditioned because their outliers are not necessarily well handled by the final layer normalization.Performance decreases for some tasks, while batch normalization or principal component analysis can help.
  • Practical handling: Intermediate-layer features are used with batch normalization in semantic segmentation and depth estimation experiments.This is an implementation choice for experiments using intermediate representations.

B.2 Per-Layer Analysis

DINOv3 feature quality varies across backbone layers and downstream tasks. Classification and dense-task performance rises smoothly, whereas geometry-heavy tasks peak around layer 32.

  • Layer-wise performance: Performance for classification and dense tasks increases smoothly over the backbone layers.The analysis evaluates classification and segmentation alongside depth, tracking, and 3D correspondence tasks.
  • Layer-wise performance: Depth estimation, tracking, and 3D correspondence estimation peak around layer 32.These tasks are described as having a significant geometric component.
  • Layer-wise performance: Intermediate layers only slightly improve over the last layer overall, making the last layer a good default choice.Earlier layers remain useful when downstream geometry is especially important.

B.3 Additional Results to Main Results Section

Additional experiments report DINOv3 results across intermediate-layer features, fine-grained classification, instance recognition, and semantic segmentation benchmarks.

  • Instance recognition: Table 23 provides full instance-recognition results with additional metrics.These results complement the main instance-recognition evaluation.
  • Intermediate-layer features: Figure 21 evaluates intermediate-layer DINOv3 7B features on five benchmarks using linear and non-parametric methods.Evaluations (a–c) use a linear layer, while (d, e) use a non-parametric approach.
  • Fine-grained classification: Table 22 reports per-dataset fine-grained classification results on small datasets using linear probing.The evaluation follows the Fine-S protocol.
  • Semantic segmentation: Table 24 compares semantic-segmentation mIoU on COCO-Stuff, PASCAL VOC 2012, and Cityscapes against prior state-of-the-art systems.Results are reported under single-scale and multi-scale test-time augmentation setups.

B.4 Classification on OCR-Heavy Datasets

DINOv3 is evaluated on OCR-heavy classification tasks involving street signs, logos, and products, where self-supervised learning remains challenging.

  • Evaluation setup: The evaluation covers street-sign, logo, and product classification tasks requiring character recognition.DINOv3 is evaluated at 512-pixel resolution, with other models adjusted for patch size.
  • Results: DINOv3 drastically outperforms DINOv2 on OCR-heavy datasets but remains behind weakly supervised PE-core.The authors attribute the remaining gap to the absence of pair image-text data and leave closing it for future work.

B.5 Fairness Analysis

The fairness analysis finds broadly consistent DINOv3 performance across income categories and regions, with measurable gaps between the lowest and highest-performing groups.

  • Income buckets: DINOv3 performance drops 23% in the low-income bucket relative to the highest-income bucket.The medium- and high-income buckets show comparable performance.
  • Regions: The Europe–Africa relative performance difference exceeds 14%, improving on DINOv2’s difference of more than 17%.DINOv3 otherwise achieves relatively good scores across regions.

C Implementation Details

The implementation combines large-scale multi-crop SSL training, Gram anchoring, and empirically selected high-resolution adaptation settings.

  • Training setup: Training uses two 256×256 global crops and eight 112×112 local crops, producing a 3.7M-token sequence for the student.The teacher EMA processes only the global crops; masking and the LDINO, LiBOT, and LDKoleo losses are also applied.
  • Gram anchoring: Gram anchoring uses loss weight wGram = 2 and updates the Gram teacher every 10k steps for at most three updates.These are the reported implementation settings for the Gram anchoring step.
  • Resolution adaptation: High-resolution adaptation samples five global/local/Gram-teacher resolution combinations with probabilities ranging from 0.05 to 0.3.The reported values were obtained empirically.

D Experimental Details

DINOv3 is evaluated across image, video, geometric, correspondence, and satellite benchmarks using frozen-backbone protocols and lightweight downstream heads. The experiments also test resolution and normalization choices, with high-resolution inference improving COCO detection performance.

  • Semantic Segmentation: Semantic segmentation is evaluated by linear probing frozen DINOv3 features on ADE20k, VOC12, and Cityscapes using mIoU.The linear classifier operates on normalized patch output features from the frozen backbone.
  • Geometric Tasks: Geometric feature quality is measured on NYUv2 and KITTI using Root Mean Squared Error.These benchmarks evaluate the dense features for geometric tasks.
  • Correspondence: Correspondence evaluation uses NAVI and SPair, reporting recall under patch-matching protocols with resolution determined by model patch size.NAVI pairs object views within a maximum 120-degree rotation and evaluates top-1000 cosine-similarity matches.
  • Unsupervised Object Discovery: Unsupervised object discovery generates one bounding box per image and evaluates Correct Localization on VOC07, VOC12, and COCO20K.TokenCut constructs a patch-similarity graph, applies Normalized Cut, and fits a box to salient patches.
  • Video Classification: Video classification encodes sampled frames with a frozen backbone, concatenates patch features, and trains a shallow transformer classifier with labeled videos.Training samples 16 frames and uses shared spatial crops, timestamps, spatial coordinates, and positional embeddings.
  • Detection Inference: 65.4 and 65.6 COCO mAP are obtained when resizing the short side to 1536 and 2048, respectively, during test-time inference.The experiments also consider flipping, multiresolution processing, prediction merging, and SoftNMS.
Loading 2508.10104v1…