Source-linked AI summary
C-RADIOv4 (Tech Report)
Mike Ranzinger, Greg Heinrich, Collin McCarthy, Jan Kautz, Andrew Tao, Bryan Catanzaro, Pavlo Molchanov
TL;DR
The paper addresses how to combine heterogeneous vision foundation models into a unified student without losing their distinct capabilities. C-RADIOv4 uses multi-teacher distillation with updated teachers and improved resolution training, delivering stronger downstream performance and expanded SAM3 compatibility, with a remaining gap in matching SAM3 more closely.
Problem
Agglomerative models must combine heterogeneous teachers while avoiding resolution-dependent behavior and unequal teacher influence during distillation.
Method
C-RADIOv4 distills dense features and summarization tokens from SigLIP2, DINOv3, and SAM3 across stochastic low- and high-resolution partitions, using shift-equivariant MESA and balanced summary loss.
Results
C-RADIOv4 improves classification at low resolution, generally surpasses earlier C-RADIO models, supports replacing SAM3’s vision encoder, and can run faster than SAM3’s ViT-L+ encoder in the stated SO400M configuration.
Takeaways & Limitations
The released SO400M and H variants extend one student model across multi-resolution classification and SAM3-based applications, with a commercially permissive license.
Takeaways & Limitations
Improving C-RADIOv4’s ability to better match SAM3 remains an open research direction.
Abstract
from arXiv · showhide
By leveraging multi-teacher distillation, agglomerative vision backbones provide a unified student model that retains and improves the distinct capabilities of multiple teachers. In this tech report, we describe the most recent release of the C-RADIO family of models, C-RADIOv4, which builds upon AM-RADIO/RADIOv2.5 in design, offering strong improvements on key downstream tasks at the same computational complexity. We release -SO400M (412M params), and -H (631M) model variants, both trained with an updated set of teachers: SigLIP2, DINOv3, and SAM3. In addition to improvements on core metrics and new capabilities from imitating SAM3, the C-RADIOv4 model family further improves any-resolution support, brings back the ViTDet option for drastically enhanced efficiency at high-resolution, and comes with a permissive license.
1. Description
C-RADIOv4 updates the agglomerative foundation-model approach with SigLIP2, DINOv3, and SAM3 teachers while improving resolution flexibility and high-resolution efficiency.
- Agglomerative foundation models create a new foundation model by distilling feature representations from heterogeneous teacher models.
- Training all teachers at both resolutions addressed the earlier mode-switching problem, which caused resolution-dependent inference behavior.
- C-RADIOv4 replaces the previous core teachers with SigLIP2, DINOv3, and SAM3.The updated teachers provide improved text alignment, semantic segmentation, and SAM3-compatible capabilities.
- C-RADIOv4 improves any-resolution operation and restores ViTDet mode for substantially faster inference on high-resolution images.ViTDet mode lets most transformer blocks operate in windowed mode.
2. Method Updates
C-RADIOv4 updates the agglomerative distillation pipeline with new teachers, smoother multi-resolution training, shift-equivariant losses, MESA regularization, and balanced summary-token supervision. These changes target cleaner feature behavior, stronger resolution scaling, and better balancing of heterogeneous teacher signals.
- Evaluation: C-RADIOv4 is competitive with DINOv3 on dense tasks at a fraction of the parameters, while its feature visualizations show cleaner object boundaries than C-RADIOv3-H.The supplied table caption states the dense-task comparison, and Figure 1 reports cleaner boundaries in the newer model.
- Resolution Training: C-RADIOv4 samples six low-resolution and four high-resolution training sizes to enable smoother resolution scaling and improve low-resolution quality.The low-resolution set is {128, 192, 224, 256, 384, 432}; the high-resolution set is {512, 768, 1024, 1152}.
- Shift Equivariant Loss: Shift-equivariant feature loss uses independently shifted student and teacher crops plus spatial alignment to reduce learning of position-specific teacher noise.Shifts are sampled in patch-size increments, and the loss averages squared aligned feature differences over common spatial positions.
- Shift Equivariant MESA: Shift-equivariant MESA matches differently cropped student and EMA-student outputs through a spatial transform, further combating fixed-pattern noise and promoting flat, robust solutions.MESA is described as converging weights toward flat regions where input perturbations do not cause chaotic output changes.
- Balanced Summary Loss: Balanced summary loss normalizes angular differences by teacher dispersion so the student focuses on relative directions and prevents one teacher from dominating.The reported dispersion differs significantly between SigLIP2 and DINOv3; without balancing, DINOv3 would dominate this loss term.
3. Results
C-RADIOv4 improves resolution-dependent classification and kNN performance while extending SAM3 backbone replacement and efficient high-resolution inference through ViTDet mode. The report also identifies an open direction in matching SAM3 more closely.
- Metrics: C-RADIOv4 strongly improves ImageNet-1K zero-shot classification at low input resolutions and reaches its maximum score at 1024px with aspect-preserving resizing.The comparison includes RADIOv2.5 and multiple C-RADIO generations.
- Metrics: Starting at 256px, C-RADIOv4-H matches or surpasses DINOv3 on ImageNet-1K kNN classification.C-RADIOv4 also generally exceeds C-RADIOv3 on this task, while higher-resolution DINOv3 performance degrades after its 192–256px scaling range.
- SAM3: C-RADIOv4 can replace SAM3’s Perception Encoder backbone, supporting SAM3-based segmentation workflows and creative applications such as RADSeg.On SA-Co/Gold instance segmentation, it becomes the second-best model but has an uneven ability to replace SAM3’s vision encoder.
- Efficiency: ViTDet mode uses mostly windowed attention with a few global layers, substantially reducing the growth factor at high resolution despite retaining O(T^2) complexity.The mode supports window sizes from 6 × 6 to 32 × 32 tokens, subject to the stated divisibility constraint.
- Efficiency: On an A100, SO400M with window size ≤12 is faster than SAM3’s encoder, while ViT-H with window size 8 is nearly as fast.The latency difference between ViTDet window sizes 8 and 16 is negligible.
- SAM3: C-RADIOv4 resolves SAM3’s reported “person” query failure in the tested RADIO configurations, including global attention and ViTDet with window size 8.The report suggests the difference between the vision encoders creates a thresholding effect for this query.
4. Conclusion
C-RADIOv4 builds on improved foundation models and distillation to deliver substantial gains, while extending practical use through SAM3 vision-encoder replacement and high-resolution efficiency options.
- C-RADIOv4 benefits from stronger base foundation models, particularly DINOv3, and improvements to its distillation algorithm.
- C-RADIOv4 can replace SAM3’s vision encoder while preserving the SAM3 demonstration’s masking and text-query functionality.
- The report evaluates instance-segmentation results using ViTDet window sizes and global attention configurations on SA-Co/Gold.
- The SO400M variant with ViTDet window size ≤12 is faster than SAM3’s ViT-L+ Perception Encoder.