Datasetpublic

ll922/RedPajama-Data-1T-Sample-Backup

RedPajama Data 1T Sample Backup This dataset is a backup mirror of togethercomputer/RedPajama-Data-1T-Sample. It is provided for easier access when the original dataset is unavailable or difficult to download. Usage Original: from datasets import load_dataset ds = load_dataset( "togethercomputer/RedPajama-Data-1T-Sample", split="train", trust_remote_code=True, ) Backup: from datasets import load_dataset ds = load_dataset(… See the full description on the dataset page: https://huggingface.co/datasets/ll922/RedPajama-Data-1T-Sample-Backup.

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

0likes198downloads

Dataset Card

RedPajama Data 1T Sample Backup

This dataset is a backup mirror of togethercomputer/RedPajama-Data-1T-Sample.

It is provided for easier access when the original dataset is unavailable or difficult to download.

Usage

Original:

python

from datasets import load_dataset

ds = load_dataset(
    "togethercomputer/RedPajama-Data-1T-Sample",
    split="train",
    trust_remote_code=True,
)

Backup:

python

from datasets import load_dataset

ds = load_dataset(
    "ll922/RedPajama-Data-1T-Sample-Backup",
    split="train",
)

Data

The data is stored in Parquet format under the data/ directory.

Source

Original dataset: togethercomputer/RedPajama-Data-1T-Sample