Team Ai
Modelpublic

xiaomoguhzz/VisionEncoder

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face
0likes
Model Card

VisionEncoder

Hosted artifacts (derived data + trained checkpoints) for the VisionEncoder research project.

Training code + full reproduction guide: https://github.com/xiaomoguhz/VisionEncoder

The repo is organized into three top-level folders.

data/ — current (V9.x) reproduction data (~6.5G)

PathContent
data/vmllm_cached/qwen3vit/S2 cached_dataset arrow (image/video, 10pct + full); fed directly to stage-2
data/ms-swift-data/sampled sharegpt jsonl (10pct + full)
data/llava_video/V9 decode-probed good_manifest for the video path

ckpts/ — ready-made 4B MLLM inference weights

PathContent
ckpts/4b_stock4B stock baseline (raw Qwen3.5 ViT, skips declip), checkpoint-505, 9.5G
ckpts/4b_v9_14B V9.1 (V-JEPA 2.1 video self-distill), checkpoint-505, 9.5G

Download either and feed it straight to evaluation (see the GitHub README, section 4 — MLLM evaluation) to skip declip + S1 + S2.

legacy/ — historical assets (~368G)

Early-line products, not needed to reproduce the current main line: declip_siglip2/spatial_align, kd_mllm, self_refine, video_mllm_swift (old SigLIP2 / image-only S1+S2 ckpts), and old ViT-family arrow caches.

Download

bash
# current dev data
huggingface-cli download xiaomoguhzz/VisionEncoder --include "data/*" --local-dir .
# ready-made 4B MLLM ckpt (eval directly)
huggingface-cli download xiaomoguhzz/VisionEncoder --include "ckpts/4b_v9_1/*" --local-dir .

Related

  • —Code + reproduction guide: https://github.com/xiaomoguhz/VisionEncoder