Datasetpublic

HuggingFaceBio/carbon-pretraining-corpus

🧬 Carbon Pretraining Corpus Description 173M DNA & RNA sequences · 1.1 trillion nucleotides — the DNA pretraining mixture used to train Carbon, a genomic foundation model. This dataset is a collection of data sources intended for training genomic foundation models, such as Carbon. It contains DNA and RNA sequences spanning eukaryote and prokaryote species. Across the four main configs it totals 1.1 T DNA base pairs (180B tokens with Carbon's 6-mer tokenizer). A… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceBio/carbon-pretraining-corpus.

sourceHugging Faceotherupdated 4mo agoView on Hugging Face

31likes2.2kdownloads

50 commits on main

8541ba44mo ago

Update README.md

loubnabnl

cb4c13a5mo ago

Update README.md

loubnabnl

c329f755mo ago

Merge eukaryote_generator_10B_subset: replace 456 small files with 14 chunks

loubnabnl

474d07b5mo ago

README: drop ~ prefix (was rendering as strikethrough on Hub)

loubnabnl

915a50a5mo ago

README: add eukaryote_generator_10B_subset, post-filter stats, fix weights/typos

loubnabnl

cb4a9515mo ago

Add eukaryote_generator_10B_subset (~60 Gbp, natural species distribution)

loubnabnl

c6046275mo ago

Filter eukaryote_generator: drop sequences > 100000 nt

loubnabnl

120943b5mo ago

Update README.md

loubnabnl

a51270f5mo ago

Update README.md

loubnabnl

665ef155mo ago

Update README.md

loubnabnl

6d5c4275mo ago

Update README.md

loubnabnl

ec38d885mo ago

Update README.md

loubnabnl

520bb205mo ago

Update README.md

loubnabnl

15f60635mo ago

Update README.md

loubnabnl

c3b70e35mo ago

Update README.md

loubnabnl

3e85e095mo ago

Update README.md

loubnabnl

b3d9c475mo ago

Update README.md

loubnabnl

6e7eb155mo ago

Update README.md

loubnabnl

9f2f4265mo ago

Add README with subset configs

loubnabnl

1716cbd5mo ago

Scrub metadata column in prokaryote_evo2/004_data_00001.parquet

loubnabnl

87be98e5mo ago

Scrub metadata column in prokaryote_evo2/004_data_00000.parquet

loubnabnl

8b6447d5mo ago

Scrub metadata column in prokaryote_evo2/003_data_00007.parquet

loubnabnl

a79f30e5mo ago

Scrub metadata column in prokaryote_evo2/003_data_00006.parquet

loubnabnl

cba15f25mo ago

Scrub metadata column in prokaryote_evo2/003_data_00005.parquet

loubnabnl

209d90b5mo ago

Scrub metadata column in prokaryote_evo2/003_data_00004.parquet

loubnabnl

c48a8d15mo ago

Scrub metadata column in prokaryote_evo2/003_data_00003.parquet

loubnabnl

42e93c15mo ago

Scrub metadata column in prokaryote_evo2/003_data_00002.parquet

loubnabnl

829fe165mo ago

Scrub metadata column in prokaryote_evo2/003_data_00001.parquet

loubnabnl

f9a85745mo ago

Scrub metadata column in prokaryote_evo2/003_data_00000.parquet

loubnabnl

26423995mo ago

Scrub metadata column in prokaryote_evo2/002_data_00007.parquet

loubnabnl

f0d95c85mo ago

Scrub metadata column in prokaryote_evo2/002_data_00006.parquet

loubnabnl

b976f8e5mo ago

Scrub metadata column in prokaryote_evo2/002_data_00005.parquet

loubnabnl

d582f9f5mo ago

Scrub metadata column in prokaryote_evo2/002_data_00004.parquet

loubnabnl

2f997725mo ago

Scrub metadata column in prokaryote_evo2/002_data_00003.parquet

loubnabnl

d5c09b95mo ago

Scrub metadata column in prokaryote_evo2/002_data_00002.parquet

loubnabnl

d2fd7675mo ago

Scrub metadata column in prokaryote_evo2/002_data_00001.parquet

loubnabnl

a71adf55mo ago

Scrub metadata column in mrna_splice_evo2/002_data_00002.parquet

loubnabnl

1d05cf65mo ago

Scrub metadata column in prokaryote_evo2/002_data_00000.parquet

loubnabnl

fc359965mo ago

Scrub metadata column in mrna_splice_evo2/002_data_00001.parquet

loubnabnl

b575d575mo ago

Scrub metadata column in mrna_splice_evo2/002_data_00000.parquet

loubnabnl

375ebe55mo ago

Scrub metadata column in prokaryote_evo2/001_data_00007.parquet

loubnabnl

2c4beb65mo ago

Scrub metadata column in mrna_splice_evo2/001_data_00007.parquet

loubnabnl

2f72df25mo ago

Scrub metadata column in prokaryote_evo2/001_data_00006.parquet

loubnabnl

b814bbe5mo ago

Scrub metadata column in mrna_splice_evo2/001_data_00006.parquet

loubnabnl

739e80d5mo ago

Scrub metadata column in prokaryote_evo2/001_data_00005.parquet

loubnabnl

a9c575e5mo ago

Scrub metadata column in mrna_splice_evo2/001_data_00005.parquet

loubnabnl

411c6315mo ago

Scrub metadata column in prokaryote_evo2/001_data_00004.parquet

loubnabnl

48cfd3d5mo ago

Scrub metadata column in mrna_splice_evo2/001_data_00004.parquet

loubnabnl

5381aa65mo ago

Scrub metadata column in prokaryote_evo2/001_data_00003.parquet

loubnabnl

cab03165mo ago

Scrub metadata column in mrna_splice_evo2/001_data_00003.parquet

loubnabnl