Team Ai
Apppublic

HuggingFaceFW-Dev/lang-word-tokenizers

sourceHugging Faceupdated 2y agoView on Hugging Face
4likes
12 commits on main
eab52a42y ago

Update index.html

guipenedo
9a7091f2y ago

georgian tokenizer and south azerbeijani

guipenedo
baa687b2y ago

added khmer, tibetan and lao

guipenedo
49dc1e72y ago

do not propagate to the root

guipenedo
bd410492y ago

macrolanguages fix

guipenedo
0741edf2y ago

updated

guipenedo
f2b528f2y ago

fixes

guipenedo
79ddb0e2y ago

cleaned up unusued parent toks

guipenedo
3a9bfe92y ago

changed default option

guipenedo
6a750902y ago

improved viz to include scripts

guipenedo
d43fb2c2y ago

added data

guipenedo
08bcb432y ago

initial commit

guipenedo