NeuML/language-id-quantized

Language Detection with StaticVectors

This model is an export of this FastText Language Identification model for `staticvectors`. staticvectors enables running inference in Python with NumPy. This helps it maintain solid runtime performance.

Language detection is an important task and identification with n-gram models is an efficient and highly accurate way to do it.

This model is a quantized version of the [base language id model](https://hf.co/neuml/language-id). It's using 2x256 Product Quantization like the original quantized model from FastText. This shrinks this model down to 4MB with only a minor hit on accuracy.

Usage with StaticVectors

from staticvectors import StaticVectors

model = StaticVectors("neuml/language-id-quantized")
model.predict(["What language is this text?"])