Seeker38/gemma-2-9b-it-abc-notation-bnb-4bit

base_model:

• —Seeker38/gemma-2-9b-it-abc-notation ---

This is a converted weight from gemma-2-9b-it-abc-notation model in unsloth 4-bit dynamic quant using this collab notebook.

About this Conversion

This conversion uses Unsloth to load the model in 4-bit format and force-save it in the same 4-bit format.

How 4-bit Quantization Works

• —The actual 4-bit quantization is handled by BitsAndBytes (bnb), which works under Torch via AutoGPTQ or BitsAndBytes.
• —Unsloth acts as a wrapper, simplifying and optimizing the process for better efficiency.

This allows for reduced memory usage and faster inference while keeping the model compact.