lance
Datasets
All datasets matching “lance”swe-lancerfineweb-edu
FineWeb-Edu (Lance Format)
A Lance-formatted version of FineWeb-Edu — over 1.5 billion educational web passages with cleaned text, source metadata, language detection signals, and 384-dim text embeddings — available directly from the Hub at hf://datasets/lance-format/fineweb-edu/data/train.lance.
Key features
Cleaned passage text in the text column with the source url and title carried alongside.
Language detection signals (language, language_probability) for filtered… See the full description on the dataset page: https://huggingface.co/datasets/lance-format/fineweb-edu.agibotworld-beta-rgb-lance
AgiBotWorld-Beta (LeRobot lance format)
agibot-world/AgiBotWorld-Beta
converted to the LeRobot lance storage format, uploaded in coordination with the AgiBot team.
160,454 episodes, 286,556,463 frames, 8 RGB cameras (AV1, copied from the source without re-encoding),
state[20] and action[22] at 30 fps.
Read it in place, no download needed (lerobot with the lancedb extra):
from lerobot.datasets import LeRobotDataset
ds = LeRobotDataset("lance-format/agibotworld-beta-rgb-lance")… See the full description on the dataset page: https://huggingface.co/datasets/lance-format/agibotworld-beta-rgb-lance.pretrain_aiworker_bg2_lance
rllab-postech/pretrain_aiworker_bg2_lance
Merged 19D AI Worker/BG2 pretraining dataset in RLLAB published Lance layout.
Tables
Table
Purpose
data/episodes.lance
Published episode table, one row per episode, no video blob columns.
data/train_episodes.lance
Training trajectory table named by manifest.json.primary_training_table; no video blob columns.
data/frames.lance
Frame-level QA/index table with remapped global frame indices.
data/videos.lance… See the full description on the dataset page: https://huggingface.co/datasets/rllab-postech/pretrain_aiworker_bg2_lance.openvid-lance
OpenVid (Lance Format)
A Lance-formatted version of the OpenVid-1M corpus — 937,957 high-quality clips with inline MP4 bytes, 1024-dim video embeddings, captions, and rich per-clip quality signals — available directly from the Hub at hf://datasets/lance-format/openvid-lance/data/train.lance.
Key features
Inline MP4 bytes in the video_blob column, stored in a side blob file and surfaced as lazy BlobFile handles via take_blobs — metadata scans, search, and filtering… See the full description on the dataset page: https://huggingface.co/datasets/lance-format/openvid-lance.Openvid-1M
OpenVid Dataset (Lance Format)
Lance format version of the OpenVid dataset with 937,957 high-quality videos stored with inline video blobs, embeddings, and rich metadata.
Why Lance?
Lance is an open-source format designed for multimodal AI data, offering significant advantages over traditional formats for modern AI workloads.
Blazing Fast Random Access: Optimized for fetching scattered rows, making it ideal for random sampling, real-time ML serving, and interactive… See the full description on the dataset page: https://huggingface.co/datasets/lance-format/Openvid-1M.