Team Ai
Apppublic

tokutsu/exllamav2_patch

sourceHugging Facemitupdated 1y agoView on Hugging Face
0likes
App README

Unigram Tokenizer Patch for ExLlamaV2

This repository provides a patch to support Unigram-based tokenizers with ExLlamaV2. By default, using a Unigram tokenizer may result in an error during inference. This patch resolves that issue.

Files

  • —hf.py.patch A patch file that modifies hf.py in the ExLlamaV2 repository to support Unigram tokenizers.
  • —apply_patch.sh A shell script to apply the patch.

Usage

You can apply the patch with the following command:

bash
$ ./apply_patch.sh EXLLAMAV2_DIR

Replace EXLLAMAV2_DIR with the path to your local clone of ExLlamaV2.

Example:

bash
$ ./apply_patch.sh ~/repos/exllamav2

Purpose

The patch is intended to:

  • —Prevent runtime errors when using tokenizers.models.Unigram.
  • —Add fallback handling for missing unk_token attributes.
  • —Improve compatibility with models trained using SentencePiece + Unigram tokenizers.

Notes

  • —This patch is intended to be applied to the hf.py file in the ExLlamaV2 repository.
  • —Tested Versions: v0.2.6 through v0.2.8
  • —If the patch fails, please ensure your copy of hf.py matches the original before applying.