shubh303/open-world-dense-object-detection

Open World Dense Object Detection Dataset

A large-scale dataset for open-world object detection containing 8,001 images with 344,079 bounding box annotations across 1,106 categories.

Important: Open-World vs Closed-Set Detection

This dataset is designed for open-world/open-vocabulary object detection models like [Grounding DINO](https://github.com/IDEA-Research/GroundingDINO), [OWL-ViT](https://huggingface.co/google/owlvit-base-patch32), and similar text-conditioned detectors.

Why NOT for YOLO, DETR, or Traditional Detectors?

Traditional closed-set object detection models (YOLO, DETR, Faster R-CNN, etc.) assume:

• —All objects of all classes are labeled in every image
• —The model learns to predict a fixed set of categories
• —Unlabeled objects are treated as "background" (negative examples)

This dataset does NOT follow that assumption. Each image only labels specific objects of interest, not all possible objects. For example:

• —An image might label only "apples" even if there are cups, tables, or people visible
• —Another image might label "coins" but not the surface they're sitting on

Suitable Models

How to Use with Grounding DINO

from groundingdino.util.inference import load_model, predict

model = load_model("groundingdino_swint_ogc.pth")

# Use category names as text prompts
sample = dataset['train'][0]
categories_in_image = set(sample['objects']['category'])
text_prompt = " . ".join(categories_in_image) + " ."

# Run detection
boxes, logits, phrases = predict(
    model=model,
    image=sample['image'],
    caption=text_prompt,
    box_threshold=0.35,
    text_threshold=0.25
)

Dataset Description

This dataset is designed for training and evaluating open-world object detection models in dense object scenarios where multiple objects of various categories appear in a single image. The dataset covers a wide variety of everyday objects, food items, natural elements, and more.

Dataset Statistics

Unlabeled Images

This dataset also includes 4,000+ unlabeled images in unlabeled_images.zip. These images:

• —Have been deduplicated using DINOv3 embeddings (similarity threshold 0.95)
• —Do not overlap with the labeled training images
• —Can be used for semi-supervised learning, self-training, or pseudo-labeling

# Download and extract unlabeled images
from huggingface_hub import hf_hub_download
import zipfile

zip_path = hf_hub_download(
    repo_id="shubh303/open-world-dense-object-detection",
    filename="unlabeled_images.zip",
    repo_type="dataset"
)

with zipfile.ZipFile(zip_path, 'r') as zip_ref:
    zip_ref.extractall("./unlabeled_images")

Example Categories

The dataset includes diverse categories such as:

• —Food & Kitchen: cookie, bread, donut, cupcake, apple, banana, tomato, egg, etc.
• —Objects: bottle, cup, bowl, plate, box, coin, key, pen, book, etc.
• —Nature: bird, flower, tree, bee, butterfly, fish, etc.
• —Vehicles: car, bicycle, bus, boat, airplane, etc.
• —Household Items: furniture, appliances, decorations, etc.

Dataset Structure

Each sample contains:

• —image: The image (PIL Image)
• —image_id: Unique identifier for the image
• —file_name: Original filename
• —width: Image width in pixels
• —height: Image height in pixels
• —objects: Dictionary containing:
• —bbox: List of bounding boxes in COCO format [x, y, width, height]
• —category_id: List of category IDs
• —category: List of category names (use these as text prompts!)
• —area: List of bounding box areas

Usage

Loading the Dataset

from datasets import load_dataset

# Load the dataset
dataset = load_dataset("shubh303/open-world-dense-object-detection")

# Access a sample
sample = dataset['train'][0]
print(f"Image size: {sample['width']}x{sample['height']}")
print(f"Number of objects: {len(sample['objects']['bbox'])}")
print(f"Categories: {set(sample['objects']['category'])}")

Visualizing Annotations

import cv2
import numpy as np
from PIL import Image

def visualize_sample(sample):
    # Convert PIL Image to numpy array
    img = np.array(sample['image'])
    img = cv2.cvtColor(img, cv2.COLOR_RGB2BGR)

    # Draw bounding boxes
    for bbox, category in zip(sample['objects']['bbox'], sample['objects']['category']):
        x, y, w, h = map(int, bbox)
        cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 2)
        cv2.putText(img, category, (x, y - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 1)

    return img

# Visualize first sample
img = visualize_sample(dataset['train'][0])
cv2.imshow("Sample", img)
cv2.waitKey(0)

Example: Using with OWL-ViT

from transformers import OwlViTProcessor, OwlViTForObjectDetection
import torch

processor = OwlViTProcessor.from_pretrained("google/owlvit-base-patch32")
model = OwlViTForObjectDetection.from_pretrained("google/owlvit-base-patch32")

sample = dataset['train'][0]
# Use category names as text queries
text_queries = [list(set(sample['objects']['category']))]

inputs = processor(text=text_queries, images=sample['image'], return_tensors="pt")
outputs = model(**inputs)

Annotation Format

Bounding boxes are in COCO format: [x_min, y_min, width, height]

Where:

• —x_min: X coordinate of the top-left corner
• —y_min: Y coordinate of the top-left corner
• —width: Width of the bounding box
• —height: Height of the bounding box

Data Sources

This dataset is a combination of multiple object detection datasets, including:

• —Custom annotated images for dense object scenarios
• —Curated samples from various open-source datasets

Intended Use

This dataset is intended for:

• —Training open-world/open-vocabulary object detection models
• —Fine-tuning Grounding DINO, OWL-ViT, and similar models
• —Benchmarking text-conditioned object detection
• —Research in zero-shot and few-shot object detection

Limitations

• —Not suitable for closed-set detectors (YOLO, DETR, etc.) - images do not label all objects
• —Some categories may have limited samples
• —Annotation quality may vary across different source datasets
• —Some category names are descriptive phrases rather than single words

Citation

If you use this dataset in your research, please cite:

@dataset{open_world_dense_object_detection,
  author = {shubh303},
  title = {Open World Dense Object Detection Dataset},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/datasets/shubh303/open-world-dense-object-detection}
}

License

This dataset is released under the MIT License.