Datasetpublic

bigcode/github-commits-diff-dedup-pjjs-april

Deduplicated Commits Deduplicated based on diff: content = '\n'.join(difflib.unified_diff( old_content.splitlines(keepends=True), new_content.splitlines(keepends=True), n=5 )) Parameters: Minimum ngram size: 5 MinHash ngram size: 5 MinHash threshold: 0.8

sourceHugging Faceupdated 3y agoView on Hugging Face

4likes1kdownloads

rootREADME.md

README.md17 linesDownload Raw Back to root

1# Deduplicated Commits2 3Deduplicated based on diff:4 5```6content = '\n'.join(difflib.unified_diff(7    old_content.splitlines(keepends=True), 8    new_content.splitlines(keepends=True), 9    n=510))11```12 13## Parameters:14 15Minimum ngram size: 516MinHash ngram size: 517MinHash threshold: 0.8