Modelpublic

ai4ce/citywalker

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

0likes177downloads

Model Card

CityWalker (2000hr)

HuggingFace port of the CityWalker waypoint-prediction model (CVPR 2025), trained on ~2000 hours of web-scale urban walking and driving videos. This repo contains the converted weights of CityWalker_2000hr.ckpt (originally a PyTorch Lightning checkpoint) re-packaged as a transformers.PreTrainedModel so it can be loaded with AutoModel.from_pretrained directly.

Upstream training dataset: ai4ce/CityWalker. Our port (model wrapper + ckpt converter + benchmark integration) lives in ai4ce/wanderland-benchmark.

Architecture

images (B, 5, 3, H, W)  ─► center_crop(400) + resize(392) + ImageNet norm
                         ─► DINOv2 (vit-b/14)        ─► obs tokens (B, 5, 768)
coords (B, 6, 2)         ─► PolarEmbedding + Linear  ─► goal token  (B, 1, 768)
                                                      ─► concat ─► (B, 6, 768)
                                                      ─► TransformerEncoder (8 heads, 16 layers)
                                                      ─► MLP head
                                                      ─► waypoints (B, 5, 2)
                                                      ─► arrive_logits (B, 1)

—context_size = 5 past RGB frames.
—len_traj_pred = 5 future XY waypoints.
—The 6 coord rows are the 5 past poses + 1 target pose, all expressed in the current-pose-relative frame and divided by the per-video step_scale (so the model consumes dimensionless units, not meters).

Usage

python

from transformers import AutoModel

model = AutoModel.from_pretrained(
    "ai4ce/citywalker",
    trust_remote_code=True,
)
model.eval()

The repo bundles modeling_citywalker.py and configuration_citywalker.py under auto_map, so trust_remote_code=True is all you need — no need to pip-install the wanderland-lab package. The DINOv2 backbone weights are included in model.safetensors.

Inputs / Outputs

Name	Shape	Notes
`images`	`(B, 5, 3, H, W)` float32	RGB in `[0, 1]`; the model applies `center_crop(400) → resize(392) → ImageNet normalize` internally
`coords`	`(B, 6, 2)` float32	5 past poses + 1 target pose in the current-pose-relative frame, scaled by `1 / step_scale`
`waypoints` out	`(B, 5, 2)` float32	Predicted XY waypoints in the current-pose-relative frame, in stepscale units — multiply by `stepscale` to recover meters
`arrive_logits`	`(B, 1)` float32	Pre-sigmoid logit for the "arrived at target" binary classifier

The model predicts 2D XY waypoints only. It does not output a heading or yaw. Downstream controllers that need (vx, vy, yaw_rate) derive yaw from the predicted waypoint direction (e.g. atan2(wp_y, wp_x)).

Policy wrapper

For robot-control use — per-episode position history, step_scale estimation from recent motion, lookahead along a reference path, and conversion of the waypoint to a body-frame velocity command — see CityWalkerPolicy in ai4ce/wanderland-benchmark.

Citation

@inproceedings{liu2025citywalker,
  title={Citywalker: Learning embodied urban navigation from web-scale videos},
  author={Liu, Xinhao and Li, Jintong and Jiang, Yicheng and Sujay, Niranjan and Yang, Zhicheng and Zhang, Juexiao and Abanes, John and Zhang, Jing and Feng, Chen},
  booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
  pages={6875--6885},
  year={2025}
}

License

Apache-2.0, matching the upstream ai4ce/CityWalker repository.