uv-scripts/transformers-inference

Transformers Continuous Batching Scripts GPU inference scripts using transformers' native continuous batching (CB). No vLLM dependency required. Why transformers CB? Instant new model support - works with any model supported by transformers, including newly released architectures. No waiting for vLLM to add support. No dependency headaches - no vLLM, flashinfer, or custom wheel indexes. Just transformers + accelerate. Simple HF Jobs setup - no Docker image… See the full description on the dataset page: https://huggingface.co/datasets/uv-scripts/transformers-inference.