matheusfpinto/test_tokenization

Orpheus PT-BR SNAC 8192 Data Fields input_ids: List[int] length 8192 attention_mask: List[int] length 8192 metadata: Dict[str, Any] containing original dataset, config, split, audio_length, text Usage from datasets import load_dataset ds = load_dataset("matheusfpinto/orpheus-ptbr-snac-8192", split="train", streaming=True) sample = next(iter(ds)) assert len(sample["input_ids"]) == 8192 Citation Please cite the original… See the full description on the dataset page: https://huggingface.co/datasets/matheusfpinto/test_tokenization.