NouRed/quantized-llama2-alpaca

Quantized LLaMA2

quantized-llama2-alpaca is a fine-tuned version of the LLaMA2 (Llama-2-7b-hf) model on the Alpaca dataset using QLoRA.

Training procedure

The following bitsandbytes quantization config was used during training:

• —loadin8bit: False
• —loadin4bit: True
• —llmint8threshold: 6.0
• —llmint8skip_modules: None
• —llmint8enablefp32cpu_offload: False
• —llmint8hasfp16weight: False
• —bnb4bitquant_type: nf4
• —bnb4bitusedoublequant: True
• —bnb4bitcompute_dtype: bfloat16

Framework versions

• —PEFT 0.4.0