Datasetpublic

dcher95/multi-species-benchmark

multi-species benchmark Photographs where 2+ species appear in the same frame. Designed to evaluate multi-label species identification and steering capabilities of biological vision-language models. Two sources, unified into one parquet schema. Sources inat21_multilabel (299 rows, 147 images) In-distribution: drawn from iNat21 validation images that already carry an iNat-supplied primary label. We use InternVL3-AWQ to surface images that also… See the full description on the dataset page: https://huggingface.co/datasets/dcher95/multi-species-benchmark.

sourceHugging Faceotherupdated 4mo agoView on Hugging Face

0likes21downloads

Dataset Card

multi-species benchmark

Photographs where 2+ species appear in the same frame. Designed to evaluate multi-label species identification and steering capabilities of biological vision-language models. Two sources, unified into one parquet schema.

Sources

`inat21_multilabel` (299 rows, 147 images)

In-distribution: drawn from iNat21 validation images that already carry an iNat-supplied primary label. We use InternVL3-AWQ to surface images that also contain a secondary species, then BioCLIP2 to propose zero-shot candidates, then two human reviewers (ariana, dan) to accept/reject and confirm the secondary species. Each accepted image contributes a row for the primary organism (with iNat ground-truth taxonomy) and a row per secondary organism (with reviewer-confirmed taxonomy).

`lila_camera_trap` (884 rows, 442 images)

Out-of-distribution: trail-camera images from LILA BC where the source labels (image-level, multiple species in the same frame) have been validated by the LILA community. Filtered to image-level multi-species cases with fully species- resolved taxa, humans + domestic animals excluded.

Schema

field	type	notes
`image_id`	str	unique key
`source_benchmark`	str	`inat21_multilabel` or `lila_camera_trap`
`source_dataset`	str	finer-grained source label (e.g. `iNat-2021-val`, `Nkhotakota Camera Traps`)
`source_url`	str	original URL when available (LILA); null for iNat
`image`	struct (bytes, path)	JPEG bytes inline
`n_species_in_image`	int32	count of distinct species in this image
`species_idx`	int32	0 for primary, 1..N for secondaries
`role`	str	`primary` / `secondary` for iNat21; null for camera trap
`bbox_px`	list<int32>	pixel coords [x1,y1,x2,y2] of the organism; null for camera trap
`kingdom` ... `species`	str	7 taxonomic ranks (iNat convention: capitalized ranks, lowercase species epithet)
`common_name`	str	when available
`taxonomy_source`	str	`inat21_ground_truth` / `bioclip_top_k_index_N` / `free_text` / `lila_camera_trap`
`reviewer`	str	reviewer name for iNat rows; null otherwise

Statistics

`inat21` — 299 rows, 147 images

Species-per-image distribution: 2: 144, 3: 2, 5: 1

Top source datasets:

—iNat-2021-val: 299 rows

Top species (by row count):

—purpuratus: 8 rows
—canadensis: 3 rows
—pluchei: 3 rows
—muricata: 3 rows
—vatia: 3 rows
—herbacea: 2 rows
—iguana: 2 rows
—ravilla: 2 rows

`camera_trap` — 884 rows, 442 images

Species-per-image distribution: 2: 442

Top source datasets:

—Nkhotakota Camera Traps: 668 rows
—ENA24: 158 rows
—Orinoquia Camera Traps: 44 rows
—Caltech Camera Traps: 14 rows

Top species (by row count):

—sylvaticus: 235 rows
—cynocephalus: 233 rows
—floridanus: 76 rows
—brachyrhynchos: 68 rows
—strepsiceros: 41 rows
—grimmia: 34 rows
—pygerythrus: 30 rows
—africanus: 20 rows

Citation

If you use this benchmark, please cite the underlying sources:

—iNat21: <https://github.com/visipedia/inat_comp/tree/master/2021>
—LILA BC: <https://lila.science/lila-bc/>
—InternVL3: <https://huggingface.co/OpenGVLab/InternVL3-38B-AWQ>
—BioCLIP2: <https://huggingface.co/imageomics/bioclip-2>

License

iNat21 photos: per-observation licenses (typically CC BY-NC). LILA BC: varies by dataset; see <https://lila.science/datasets/>. Use of this benchmark for research / evaluation is intended; review the upstream licenses before any redistribution.