Team Ai
Modelpublic

MirroS-Lab/AgentGarten-renderer

sourceHugging Faceotherupdated 2d agoView on Hugging Face
1likes47downloads
Model Card

AgentGarten renderer

The real-time neural renderer of AgentGarten: Code Worlds for Evolving Agents. A code world (a simulator or a game engine) keeps the state and runs the rules; this model turns the depth and surface normals it exports into the observations an agent sees, block by block.

  • —Blog: https://mirros.ai/blog/worlds-for-evolving-agents
  • —Technical report: https://mirros.ai/report/agent-garten.pdf
  • —Project page: https://mirros-lab.github.io/agent-garten
  • —Code: https://github.com/MirroS-Lab/AgentGarten

What is in this repository

FileContents
student.safetensorsThe few-step autoregressive generation tower (7.0 B parameters, BF16), distilled with Adversarial Forcing
und.safetensorsThe frozen text tower of Cosmos3-Nano, unchanged
text_tokenizer/The Cosmos3-Nano text tokenizer, unchanged
config.yaml, manifest.jsonNetwork configuration and the file list the loader reads

The model starts from Cosmos3-Nano. It was adapted to depth and normal conditions, trained as a block-causal autoregressive model, and distilled into a four-step student. Video is encoded and decoded with the Wan 2.2 VAE, which is not included here.

Use

Install the AgentGarten code, then:

bash
hf download MirroS-Lab/AgentGarten-renderer --local-dir weights/AgentGarten-renderer
hf download Wan-AI/Wan2.2-TI2V-5B Wan2.2_VAE.pth --local-dir weights/Wan2.2-TI2V-5B
python
from wm.inference.cosmos3 import inspect_artifact, load_artifact
from wm.inference.serving import prepare_serving
from wm.networks.cosmos3.streaming import Cosmos3Stream, StreamPolicy

artifact = load_artifact(inspect_artifact("weights/AgentGarten-renderer"))
prepare_serving(artifact.student)
stream = Cosmos3Stream(artifact.student, StreamPolicy())

The repository's README shows the streaming loop: each block of four latent frames is denoised in four steps and then committed to the key-value cache, so the conditions of the next block can depend on what was just generated.

License

The weights derive from Cosmos3-Nano and are released under the same OpenMDW License Agreement, version 1.1. The text tower and the tokenizer are NVIDIA's files, redistributed unchanged under that license.