cstr/got-ocr2-crispembed-GGUF

GGUF conversion of stepfun-ai/GOT-OCR2_0 for use with CrispEmbed.

Architecture

• —Vision: SAM ViT-B (12 layers, 768d, 12 heads, 16×16 patches, 1024×1024 input)
• —Windowed attention (ws=14) with global attention at layers [2, 5, 8, 11]
• —Decomposed relative position encoding
• —Neck: Conv(768→256) → LN2d → Conv(256→256) → LN2d
• —Downsample: Conv(256→512→1024, stride 2) → 256 vision tokens
• —Projector: Linear(1024, 1024)
• —LLM: Qwen2-0.5B (24 layers, 1024d, MHA 16/16, SiLU SwiGLU, RoPE θ=1M)
• —Tokenizer: tiktoken (151860 vocab)
• —Total: ~0.7B parameters

Files

Precision & parity

The Qwen2-0.5B decoder quantizes cleanly to Q4_K and Q8_0 — all three builds above produce identical, correct OCR. Verified against the real HF model (transformers GotOcr2) plus a Python f32 reference:

• —Vision (ViT layers, neck, downsample, projector): cos ≥ 0.998 vs HF.
• —LLM decoder (per-layer, Q8_0 weights vs f32 reference): cos ≥ 0.99996.

Per-token decode speed on an M1 (256 vision tokens spliced into the prompt):

Q4_K is ~2× faster to decode than F16 and 3× smaller, so it is the default.

Note on earlier builds. A prior version of this repo shipped an F16-decoder build and claimed the 0.5B decoder was "catastrophically sensitive to quantization" (llm_layer_0 cos ≈ 0.936 at Q80). That number was a **measurement artifact** of a per-row bug in the diff harness (it used the token count as the row length), not real quant sensitivity. With the corrected harness the Q80/Q4_K decoder matches f32 at cos ≥ 0.99996 and OCR output is identical to F16. See CrispEmbed issue #25.

Usage

crispembed --ocr got-ocr2 image.png

Reproducing the quants

crispembed-quantize got-ocr2-f16.gguf got-ocr2-q4_k.gguf q4_k
crispembed-quantize got-ocr2-f16.gguf got-ocr2-q8_0.gguf q8_0

(The quantizer also has an optional --decoder-f16 flag that keeps the decoder weights at F16; it is not needed for correctness and is retained only for diagnostic / comparison use.)

License

Apache-2.0 (same as upstream model)

Provenance and EU AI Act Art. 53 note

• —Upstream model: stepfun-ai/GOT-OCR2_0 — published by stepfun-ai.
• —Upstream licence: apache-2.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• —What was done here: format conversion and/or quantisation only (GGUF). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• —Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository. No training-content summary was found on the upstream model card at the time of writing; that documentation gap is upstream's and is not filled here.
• —Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.