birder-project/TreeOfLife-10M-WEBP

Dataset Card for TreeOfLife-10M-WEBP

Dataset Description

This is an optimized version of the TreeOfLife-10M dataset, containing over 10 million images covering 454 thousand taxa in the tree of life. This version has been processed to improve usability and reduce storage requirements while maintaining full compatibility with the original dataset structure.

Dataset Summary

This version modifies the original dataset as follows:

• —Corrupted files were repaired.
• —Very large images (some >40K pixels in width) were resized so that the total number of pixels < 1,048,576 (=1024×1024), preserving aspect ratio.
• —All images were re-encoded to WEBP format.
• —The dataset was repacked in the same shard structure as the original to remain fully compatible.

The result is a significantly reduced dataset size (~500GB vs. ~2TB), with lower I/O overhead and fewer extreme image cases that can slow down training pipelines.

For dataset details, licensing information, taxonomy information and annotation process, please see the original dataset card.

Limitations

• —Maintains all original dataset limitations regarding taxonomic coverage and class imbalance
• —Some images have been resized, which may affect fine-grained visual analysis of extremely high-resolution specimens

Licensing

The TreeOfLife-10M dataset compilation is released under CC0 1.0. This designation applies to the compilation and does not override the licenses of individual images or text records. The source dataset contains assets under a variety of licenses, primarily from the Creative Commons family; EOL image licenses range from CC0 to CC BY-NC-SA.

Repairing, resizing, re-encoding, and repackaging the images does not change those underlying terms. This dataset preserves the original treeoflife_id identifiers, allowing each asset to be linked to the upstream `metadata/licenses.csv`, which records its license, source, copyright owner, and license URL. Users are responsible for complying with the applicable license and attribution requirements for each asset. See the original licensing information for details.

Citation

@dataset{treeoflife_10m,
  author = {Samuel Stevens and Jiaman Wu and Matthew J Thompson and Elizabeth G Campolongo and Chan Hee Song and David Edward Carlyn and Li Dong and Wasila M Dahdul and Charles Stewart and Tanya Berger-Wolf and Wei-Lun Chao and Yu Su},
  title = {TreeOfLife-10M},
  year = {2023},
  url = {https://huggingface.co/datasets/imageomics/TreeOfLife-10M},
  doi = {10.57967/hf/1972},
  publisher = {Hugging Face}
}

@inproceedings{stevens2024bioclip,
  title = {{B}io{CLIP}: A Vision Foundation Model for the Tree of Life},
  author = {Samuel Stevens and Jiaman Wu and Matthew J Thompson and Elizabeth G Campolongo and Chan Hee Song and David Edward Carlyn and Li Dong and Wasila M Dahdul and Charles Stewart and Tanya Berger-Wolf and Wei-Lun Chao and Yu Su},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2024},
  pages = {19412-19424}
}

Acknowledgments

This optimization builds upon the outstanding work of the original TreeOfLife-10M creators. All credit for data curation, taxonomic labeling, and scientific contributions belongs to the original team at the Imageomics Institute.