nvidia/C-RADIOv4-H

1---2license: other3license_name: nvidia-open-model-license4license_link: https://developer.download.nvidia.com/licenses/nvidia-open-model-license-agreement-june-2024.pdf5library_name: transformers6---7 8# Model Overview9 10## Description11 12This model performs visual feature extraction.13For instance, RADIO generates image embeddings that can be used by a downstream model to classify images.14 15C-RADIOv4 models are available in multiple sizes:16* Shape-Optimized (431M parameters).17* Huge (653M parameters).18 19C-RADIOv4 was trained using an updated set of teach models:20* [SigLIP2-g](https://huggingface.co/google/siglip2-giant-opt-patch16-384)21* [DINOv3-7B](https://huggingface.co/facebook/dinov3-vit7b16-pretrain-lvd1689m)22* [SAM3](https://huggingface.co/facebook/sam3)23 24This model is ready for commercial/non-commercial use.25 26### License/Terms of Use27 28GOVERNING TERMS: Use of this model is governed by the [NVIDIA Open Model License Agreement](https://developer.download.nvidia.com/licenses/nvidia-open-model-license-agreement-june-2024.pdf).29 30## Deployment Geography31 32Global33 34## Use Case35 36The embeddings generated by this model are expected to be used by a downstream application.37For example:38 39* Image-level understanding (image classification, curation, etc.).40* Dense processing (semantic segmentation, depth estimation, etc.).41* Integration into a Vision-Language Model.42 43## Release Date44 45Hugging Face: 01/27/2026 via [RADIO Collection of Models](https://huggingface.co/collections/nvidia/radio-669f77f1dd6b153f007dd1c6).46 47## References48 49* [AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into One](https://arxiv.org/abs/2312.06709)50* [PHI-S: Distribution Balancing for Label-Free Multi-Teacher Distillation](https://arxiv.org/abs/2410.01680)51* [RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models](https://arxiv.org/abs/2412.07679)52* [FeatSharp: Your Vision Model Features, Sharper](https://arxiv.org/abs/2502.16025)53* [C-RADIOv4 (Tech Report)](https://arxiv.org/abs/2601.17237)54 55## Model Architecture56 57**Architecture Type:** Neural Network  <br>58**Network Architecture:** Vision Transformer <br>59**Number of model parameters:** -SO400M size: 431M, -H size: 653M <br>60 61## Input62 63**Input Type(s):** Image <br>64**Input Format(s):** Red, Green, Blue (RGB) <br>65**Input Parameters:** Two Dimensional (2D) <br>66**Other Properties Related to Input:** Image resolutions up to 2048x2028 in increments of 16 pixels <br>67 68## Output69 70**Output Type(s):** Embeddings <br>71**Output Format:** Tensor <br>72**Output Parameters:** Two Dimensional 2D <br>73**Other Properties Related to Output:** Downstream model required to leverage image features. Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>74 75## Usage:76 77RADIO will return a tuple with two tensors.78The `summary` is similar to the `cls_token` in ViT and is meant to represent the general concept of the entire image.79It has shape `(B,C)` with `B` being the batch dimension, and `C` being some number of channels.80The `spatial_features` represent more localized content which should be suitable for dense tasks such as semantic segmentation, or for integration into an LLM.81 82```python83import torch84from PIL import Image85from transformers import AutoModel, CLIPImageProcessor86 87hf_repo = "nvidia/C-RADIOv4-H"88 89image_processor = CLIPImageProcessor.from_pretrained(hf_repo)90model = AutoModel.from_pretrained(hf_repo, trust_remote_code=True)91model.eval().cuda()92 93image = Image.open('./assets/radio.png').convert('RGB')94pixel_values = image_processor(images=image, return_tensors='pt', do_resize=True).pixel_values95pixel_values = pixel_values.cuda()96 97summary, features = model(pixel_values)98```99 100Spatial features have shape `(B,T,D)` with `T` being the flattened spatial tokens, and `D` being the channels for spatial features. Note that `C!=D` in general.101Converting to a spatial tensor format can be done using the downsampling size of the model, combined with the input tensor shape. For RADIO, the patch size is 16.102 103```Python104from einops import rearrange105spatial_features = rearrange(spatial_features, 'b (h w) d -> b d h w', h=x.shape[-2] // patch_size, w=x.shape[-1] // patch_size)106```107 108The resulting tensor will have shape `(B,D,H,W)`, as is typically seen with computer vision models.109 110## Software Integration111 112**Runtime Engine(s):**113* [TAO-6.1] <br>114 115 116**Supported Hardware Microarchitecture Compatibility:** <br>117* NVIDIA Ampere <br>118* NVIDIA Blackwell <br>119* NVIDIA Jetson  <br>120* NVIDIA Hopper <br>121* NVIDIA Lovelace <br>122* NVIDIA Pascal <br>123* NVIDIA Turing <br>124* NVIDIA Volta <br>125 126**[Preferred/Supported] Operating System(s):** <br>127* Linux128* Linux 4 Tegra129* QNX130* Windows131 132The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.133 134This AI model can be embedded as an Application Programming Interface (API) call into the software environment described above.135 136## Model Version(s)137 138* C-RADIOv4-SO400M (400M parameters).139* C-RADIOv4-H (653M parameters).140 141**Links:**142 143* https://huggingface.co/nvidia/C-RADIOv4-SO400M144* https://huggingface.co/nvidia/C-RADIOv4-H145 146# Training and Evaluation Datasets147 148## Training Dataset149 150**NV-CC-Img-Text-Dataset**151 152**Data Modality:** Image <br>153**Image Training Data Size:** 1 Million to 1 Billion Images <br>154**Data Collection Method by dataset:** Automated <br>155**Labeling Method by dataset:** Not Applicable (no labels are needed) <br>156**Properties:** 700 Million Images <br>157 158## Evaluation Datasets159 160**ImageNet**161 162**Link:** [ImageNet](https://www.image-net.org/) <br>163**Data Collection:** Automated <br>164**Labeling Method:** Human <br>165**Training Images:** 1,281,167 <br>166**Validation Images:** 50,000 <br>167**Test Images:** 100,000 <br>168 169To perform the semantic segmentation evaluation, we use training sets from ADE20K and PascalVOC to train a linear layer, and subsequently performed evaluations on the validation set.170See below for further details:171 172**ADE20k**173 174**Link:** [ADE20K](https://ade20k.csail.mit.edu/) <br>175**Data Collection:** Human <br>176**Labeling Method:** Human <br>177**Training Images:** 25,574 <br>178**Validation Images:** 2,000 <br>179 180**Pascal VOC**181 182**Link:** [Pascal VOC](http://host.robots.ox.ac.uk/pascal/VOC/) <br>183**Data Collection:** Human <br>184**Labeling Method:** Human <br>185**Training Images:** 1,464 <br>186**Validation Images:** 1,449 <br>187 188| Benchmark | C-RADIOv3-B | C-RADIOv3-L | C-RADIOv4-SO400M | C-RADIOv3-H | C-RADIOv4-H |189|-----------|-------------|-------------|------------------|-------------|-------------|190| **ImageNet Classification (Top1 accuracy)** |  | |191| Zero-Shot | 71.30       | 79.95       | 82.01            | 82.65       | 83.09       |192| KNN       | 81.22       | 84.33       | 85.75            | 86.23       | 86.68       |193| **ADE20k Semantic Segmentation (mIoU)**     | 49.79 | 51.87 | 55.14 | 52.75 | 55.20 |194| **Pascal VOC Semantic Segmentation (mIoU)** | 84.68 | 86.12 | 87.22 | 86.41 | 87.24 |195 196 197## Inference198 199**Acceleration Engine:** Tensor(RT), Tensor(RT)-LLM <br>200**Engine:** PyTorch <br>201**Test Hardware:** H100 <br>202 203## Ethical Considerations204 205NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications.  When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.206 207Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included.208 209For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards below.210 211Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).212 213### Bias214 215Field                                                                                               |  Response216:---------------------------------------------------------------------------------------------------|:---------------217Participation considerations from adversely impacted groups [protected classes](https://www.senate.ca.gov/content/protected-classes) in model design and testing:  |  None218Measures taken to mitigate against unwanted bias:                                                   |  None219 220 221### Explainability222 223Field                                                                                                  |  Response224:------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------225Intended Task/Domain:                                                                                  |  Visual Feature Extraction226Model Type:                                                                                            |  Vision Transformer227Intended Users:                                                                                        |  Developers of downstream vision applications228Output:                                                                                                |  Image embeddings229Describe how the model works:                                                                          |  The model takes an image as input, processes the image through multiple transformer blocks, and outputs summary and patch embeddings.230Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of:  |  Not Applicable231Technical Limitations:                                                                                 |  This model generates image embeddings that can be used by a downstream model to, for example, classify images. The downstream model must be trained to leverage the visual embeddings. This model is only tested on input resolutions ranging from 256 to 2048, in increments of 16 pixels. This model may fail to surface information about the orientation of objects (e.g. whether a traffic sign points left/right).232Verified to have met prescribed NVIDIA quality standards:  |  Yes233Performance Metrics:                                                                                   |  Image classification accuracy, semantic segmentation mean-over-intersection.234Potential Known Risks:                                                                                 |  This model may not perform well on visual domains that are not represented in the training data. The generated embeddings might fail to disambiguate differences that appear evident to humans (e.g. two images showing different breeds of dogs might in fact produce very similar embeddings). Domain-specific evaluation is required for the target application.235Licensing:                                                                                             |  [NVIDIA Open Model License](https://developer.download.nvidia.com/licenses/nvidia-open-model-license-agreement-june-2024.pdf)236 237 238### Privacy239 240Field                                                                                                                              |  Response241:----------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------242Generatable or reverse engineerable personal data?                                                                               |  No243Personal data used to create this model?                                                                                       |  None Known244How often is dataset reviewed?                                                                                                     |  Before Every Release245Is there provenance for all datasets used in training?                                                                                |  Yes246Does data labeling (annotation, metadata) comply with privacy laws?                                                                |  Yes247Is data compliant with data subject requests for data correction or removal, if such a request was made?                           |  Yes248Was data from user interactions with the AI model (e.g. user input and prompts) used to train the model?                           | No249Applicable Privacy Policy                                                                          | https://www.nvidia.com/en-us/about-nvidia/privacy-policy/250 251### Safety252 253Field                                               |  Response254:---------------------------------------------------|:----------------------------------255Model Application Field(s):                               |  Generation of visual embeddings256Describe the life critical impact (if present).   |  Not Applicable257Use Case Restrictions:                              |  Abide by NVIDIA Open Model License Agreement258Model and dataset restrictions:            |  The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development. Restrictions enforce dataset access during training, and dataset license constraints adhered to.