Datasetpublic

bigcode/commitpack-subset-cf

A subset of CommitPack used for pretraining SantaCoderPack from the OctoPack paper. It focuses on data where the code before + special token + code after fits into 8192 tokens and on 6 languages. The data is in commit format (cf): <commit_before>code_before<commit_message>commit_message<commit_after>code_after.

sourceHugging Faceupdated 3y agoView on Hugging Face

2likes2.4kdownloads

rootREADME.md

README.md3 linesDownload Raw Back to root

1A subset of [CommitPack](https://huggingface.co/datasets/bigcode/commitpack) used for pretraining [SantaCoderPack](https://huggingface.co/bigcode/santacoderpack) from the OctoPack paper. 2 3It focuses on data where the code before + special token + code after fits into 8192 tokens and on 6 languages. The data is in commit format (cf): `<commit_before>code_before<commit_message>commit_message<commit_after>code_after`.