Modelpublic

nvidia/Cosmos-AnomalyGen-PCB-2B

sourceHugging Faceotherupdated 4mo agoView on Hugging Face

4likes71downloads

Model Card

Model Overview

Description:

Cosmos AnomalyGen — PCB (UC1) generates synthetic printed-circuit-board anomaly images by inpainting a user-supplied binary mask onto a clean reference PCB image, conditioned on one of three trained <texture>+<anomaly_type> pairs (IC+bridge, passive_component+excess_solder, passive_component+missing). The release ships only the few-shot-finetuned modules — a set of anomaly-token embeddings and a 2-layer MLP adapter — which plug into the frozen Cosmos-Predict2 2B Text-to-Image diffusion backbone (also using a frozen NV-DINOv2 mask encoder and a frozen T5 text encoder) at inference time. Cosmos AnomalyGen — UC1 v1.0.0 was developed by NVIDIA as part of the Cosmos AnomalyGen pipeline. This model is ready for commercial use.

License/Terms of Use:

Governing Terms: Use of this model is governed by the NVIDIA Open Model Agreement.

Deployment Geography:

Global

Use Case:

Industrial visual-inspection teams responsible for PCB QA who have only a small number of real anomaly examples (≤62 per defect type). The model produces large-scale synthetic anomaly datasets (clean PCB + binary mask → realistic bridge / excesssolder / missing-component image) for training downstream defect-detection or segmentation models, including downstream TAO toolkit consumers via the DAFT v3.0 export path. Unlike UC2 and UC3, UC1 spans two PCB texture categories (`IC` and `passivecomponent`), so a single checkpoint can cover defects whose appearance depends on which board region (IC area vs. passive-component area) they occur in.

Release Date:

Github 06/02/2026 via https://github.com/NVIDIA/paidf-anomalygen

References(s):

—Anomaly Diffusion (AAAI 2024) — paper: https://arxiv.org/abs/2312.05767, code: https://github.com/sjtuplayer/anomalydiffusion
—Cosmos-Predict2 — https://github.com/nvidia-cosmos/cosmos-predict2
—NV-DINOv2 classification model — https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/nvdinov2classification_model

Model Architecture:

Architecture Type: Transformer (diffusion DiT backbone with learnable conditioning modules)

Network Architecture:

—anomaly_embedding (trainable, included in this release): token embeddings (256 tokens per <texture>+<anomaly_type> pair) — three pairs trained for UC1: IC+bridge, passive_component+excess_solder, passive_component+missing.
—adapter (trainable, included in this release): 2-layer MLP with GELU activations (input / output hidden size = 1024), projecting the mask encoder output into the diffusion DiT conditioning space.
—mask_encoder (frozen, not redistributed in this release): NV-DINOv2 (ViT-L) backbone with adaptive pool (kernel = 7); weights are loaded from the separately downloaded NV-DINOv2 classification checkpoint at inference time.
—text_encoder (frozen, not redistributed in this release): google-t5/t5-large.
—These modules condition the frozen Cosmos-Predict2 2B T2I DiT denoiser at inference time.

This model was developed based on Cosmos-Predict2-2B-Text2Image.

Number of model parameters: Approximately 2.9×10^6 (2.9 million) trainable parameters in the released modules — anomaly_embedding ≈ 0.79M (256 tokens × 1024 hidden × 3 <texture>+<anomaly_type> pairs) plus the 2-layer MLP adapter ≈ 2.1M (1024→1024 with GELU). The trainable modules are distributed as the model/iter_000014000.pt checkpoint file. The frozen Cosmos-Predict2 2B base contributes ~2.0×10^9 (2 billion) parameters used at inference time but not redistributed in this release.

Input(s):

Input Type(s): Image, Binary Mask, Text

Input Format(s):

—Image: PNG / JPG, Red, Green, Blue (RGB)
—Binary Mask: PNG / JPG, single-channel binary (0 = background, 255 = anomaly region; binarized at threshold 127)
—Text: anomaly-type string in the form <texture>+<anomaly_type> (one of IC+bridge, passive_component+excess_solder, passive_component+missing)

Input Parameters:

—Image: Two-Dimensional (2D)
—Mask: Two-Dimensional (2D)
—Text: One-Dimensional (1D)

Other Properties Related to Input: Input clean image and paired mask must have the same dimensions; the model was trained at 512×512 and inference is run at the same resolution. anomaly_type must exactly match one of the three pairs trained for this UC1 checkpoint — passing an unsupported defect string is rejected by scripts/anomaly_gen/sdg-inference/validate_jsonl.py against this checkpoint's ag_config.yaml → dataloader_train.dataset.anomaly_types. Because UC1 spans two textures, the chosen texture (IC vs. passive_component) must match the board region from which the clean reference image was cropped, otherwise the generated defect may look misplaced. The optional Automatic Mask Placement (AMP) tool can constrain mask placement to legal ROIs (e.g., only on IC pads, only on passive-component pads).

Output(s)

Output Type(s): Image

Output Format(s): PNG; Red, Green, Blue (RGB)

Output Parameters: Two-Dimensional (2D)

Other Properties Related to Output: 512×512 RGB synthetic anomaly image. Anomaly content is generated inside the user-supplied mask region; in the default crop_and_paste=True flow the inpainted patch is pasted back onto the clean reference image so non-masked pixels remain identical to the input. Optionally Poisson blending can be enabled. Generation metadata (per-sample guidance, cropratio, seed, etc.) is written to `SDGresult.csv` alongside the images.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Software Integration:

Runtime Engine(s):

—PyTorch (via the Cosmos-Predict2 2B T2I pipeline)
—Cosmos AnomalyGen scripts (scripts.anomaly_gen.synthetic_dataset_generation, torchrun-based)
—NVIDIA TAO Toolkit — interop via DAFT v3.0 export (scripts.anomaly_gen.convert_to_daft_format)

Supported Hardware Microarchitecture Compatibility:

—NVIDIA Ampere (A100)
—NVIDIA Hopper (H100)
—NVIDIA RTX 6000

Supported Operating System(s):

—Linux

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

Model Version(s):

v1.0.0 — uc1-pcb-2b-512-iter14000 (trained 14,000 iterations; released artifact is the model/iter_000014000.pt checkpoint file containing finetuned modules only).

Training, Testing, and Evaluation Datasets:

Dataset Overview

—Total Number of Datasets: 1 (Cosmos AnomalyGen PCB reference dataset, pre-packaged on NVIDIA NGC — no local preparation required)
—Total Size: 86 anomaly RGB images + 86 paired binary masks + clean PCB reference images (clean images are used at inference time, not as supervised training targets)
—Per <texture>+<anomaly_type> pair: IC+bridge = 8 anomaly images, passive_component+excess_solder = 16 anomaly images, passive_component+missing = 62 anomaly images
—Dataset partition: 100% Training (few-shot fine-tuning regime; no held-out validation or test split)

Public Datasets

—nv-metropolis-dev/metropolis-sdg/cosmos-anomalygen-pcb-dataset:1.0 — NVIDIA NGC — anomaly images, paired binary masks, clean reference images, and defect_spec.jsonl.
—nv-metropolis-dev/metropolis-sdg/cosmos-anomalygen-pcb-assets:1.0 — NVIDIA NGC — supporting assets for the UC1 pipeline.

Training Dataset:

Data Modality:

—Image

Training Data Size:

—Less than a Million Images (86 anomaly images + 86 paired masks across three <texture>+<anomaly_type> pairs)

** Data Collection Method by dataset

—Hybrid: Human, Automatic/Sensors

** Labeling Method by dataset

—Human

Properties: 86 RGB images of printed circuit boards spanning two textures (IC, passive_component) and three defect categories — IC+bridge (8 images), passive_component+excess_solder (16 images), passive_component+missing (62 images) — each paired with a per-defect binary mask. Clean (no-defect) PCB images from the same NGC dataset are used as inpainting references at inference time. The class imbalance across the three pairs (e.g., 62× missing vs. 8× bridge) is intentional and reflects what was available; the diffusion fine-tuning is robust to per-class sample counts as low as a handful. No personal data, copyrighted human-subject content, or human subjects are present in the dataset.

Testing Dataset:

Not Applicable — the model is qualitatively evaluated via the log_image validation callback (inpainted samples logged every N training steps) rather than against a held-out test split.

Evaluation Dataset:

Not Applicable — no held-out evaluation dataset. The model is evaluated qualitatively during training via the log_image validation callback, plus FID and nearest-neighbour metrics (nn_score, mnn_score) logged at each validation step. See the Explainability subcard for performance metric details.

Inference:

Acceleration Engine: PyTorch (native, FP32). Multi-GPU rank-sharded inference is supported via torchrun --nproc_per_node=<N> with the predict2_anomaly_gen_fsdp_2b experiment.

Test Hardware:

—NVIDIA A100
—NVIDIA H100
—NVIDIA RTX 6000

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please make sure you have proper rights and permissions for all input image and video content.

Users are responsible for model inputs and outputs. Users are responsible for ensuring safe integration of this model, including implementing guardrails as well as other safety mechanisms, prior to deployment.

For more detailed information on ethical considerations for this model, please see the Bias, Explainability, Privacy, and Safety & Security subcards.

Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.

Bias

Field	Response
Participation considerations from adversely impacted groups protected classes in model design and testing:	Not Applicable — the model operates on images of printed circuit boards (inanimate objects) and does not target or classify human subjects.
Measures taken to mitigate against unwanted bias:	Not Applicable — the model is a synthetic anomaly image generator for industrial defect inspection and does not produce classifications or predictions on protected attributes.
Bias Metric (If Measured):	Not Measured.

Explainability

Field	Response
Intended Task/Domain:	Industrial visual inspection — synthetic anomaly image generation (image-to-image inpainting) for printed-circuit-board defect detection.
Model Type:	Diffusion model — Transformer-based DiT denoiser (Cosmos-Predict2 2B, frozen) conditioned by learned anomaly-token embeddings, a frozen NV-DINOv2 mask encoder, and a trained 2-layer MLP adapter that projects mask-encoder features into the DiT conditioning space.
Intended Users:	ML engineers and computer-vision practitioners building defect-detection or segmentation models for printed-circuit-board QA.
Output:	Image (single 512×512 RGB anomaly image per inference call; multiple per testcase line when `num_generated_images > 1`).
Describe how the model works:	A user-supplied binary mask indicating where the defect should appear is encoded by a frozen NV-DINOv2 backbone and projected through a trained 2-layer MLP adapter into the diffusion DiT conditioning space. The selected `<texture>+<anomaly_type>` token (`IC+bridge`, `passive_component+excess_solder`, or `passive_component+missing`) retrieves a learned 256-token embedding which further conditions the frozen Cosmos-Predict2 2B T2I DiT denoiser. The denoiser then produces an inpainting-style sample where pixels inside the mask are generated as a defect that matches the trained type, while pixels outside the mask are preserved from the clean reference image (with optional crop-and-paste / Poisson blending). The two-texture design (`IC` vs. `passive_component`) lets a single checkpoint cover defects that appear differently depending on board region.
Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of:	Not Applicable
Technical Limitations & Mitigation:	Trained on a small number of examples per defect pair (8, 16, and 62 anomaly images for `IC+bridge`, `passive_component+excess_solder`, and `passive_component+missing` respectively). Out-of-distribution board layouts, mask placements that deviate substantially from the trained distribution, mismatched texture choice (e.g., using `IC+bridge` on a passive-component region of the board), or any defect not in the three trained pairs may yield low-fidelity outputs. The `IC+bridge` pair has the fewest training examples and is the most sensitive to mask-placement quality. Mitigation: (a) use the Automatic Mask Placement (AMP) tool, which automatically restricts the user-supplied mask to legal regions of interest for each `<texture>+<anomaly_type>` pair (e.g., only on IC pads for `IC+bridge`, only on passive-component pads for `passive_component+*`), preventing the model from generating defects in implausible board locations; (b) run `scripts/anomaly_gen/filter.py` on the generated dataset — the filter scores each output with a Generative Image Quality Assessment (G-IQA) model and discards samples whose quality/realism score falls below a configurable threshold, so unrealistic synthetic images do not pollute the downstream training set; (c) validate any downstream detector trained on synthetic samples against real defect imagery before deployment.
Verified to have met prescribed NVIDIA quality standards:	Yes.
Performance Metrics:	FID (logged during training validation), nearest-neighbor metrics (`nn_score`, `mnn_score`), and visual inspection of `log_image` callback outputs.
Potential Known Risks:	Generated synthetic defects may not perfectly cover all real-world defect variations. Downstream detection models trained primarily on these synthetic samples should be validated against real defect data before deployment in production QA lines.<br><br>This model can generate synthetic images and may produce content that is offensive, unsafe, misleading, indecent, or unsuitable for a target deployment. Users should implement robust safety guardrails — including content filtering, abuse monitoring, and access controls — to reduce the risk of harmful outputs. Users are responsible for ensuring that their use of the model complies with all applicable laws and regulations, and for regularly reviewing and updating their guardrails as risks evolve.
Licensing:	Governing Terms: Use of this model is governed by the NVIDIA Open Model Agreement.<br>

Privacy

Field	Response
Generatable or reverse engineerable personal data?	No
Personal data used to create this model?	No
Was consent obtained for any personal data used?	Not Applicable
How often is dataset reviewed?	Before Release
Was data from user interactions with the AI model (e.g. user input and prompts) used to train the model?	No
Is there provenance for all datasets used in training?	Yes
Does data labeling (annotation, metadata) comply with privacy laws?	Yes
Is data compliant with data subject requests for data correction or removal, if such a request was made?	Not Applicable
Applicable Privacy Policy	https://www.nvidia.com/en-us/about-nvidia/privacy-policy/

Safety

Field	Response
Model Application Field(s):	Industrial/Machinery and Robotics — specifically synthetic data generation for manufacturing QA / visual inspection of printed circuit boards.
Describe the life critical impact (if present).	Not Applicable.
Use Case Restrictions:	Governing Terms: Use of this model is governed by the NVIDIA Open Model Agreement.<br>
Model and dataset restrictions:	The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development. Restrictions enforce dataset access during training, and dataset license constraints adhered to.