Datasetpublic

derekl35/diffusers-quantization-benchmarks

sourceHugging Faceupdated 1y agoView on Hugging Face

1likes20downloads

rootdiffusers_quantization_benchmark.csv

diffusers_quantization_benchmark.csv27 linesDownload Raw Back to root

1Library,Precision,CPU Offload,Memory After Loading (GB),Peak Memory GB,Inference Time (Seconds)2Diffusers,BF16,False,31.447,36.166,123Diffusers,BF16,True,0.000,23.271,204bitsandbytes,4-bit,False,12.584,17.281,125bitsandbytes,4-bit,True,12.383,12.383,176bitsandbytes,8-bit,False,19.273,24.432,277bitsandbytes,8-bit,True,19.182,23.428,278torchao,int4_weight_only,False,10.635,14.654,1099torchao,int4_weight_only,True,,,10torchao,int8_weight_only,False,17.020,21.482,1511torchao,int8_weight_only,True,0.000,12.189,2612torchao,float8_weight_only,False,17.016,21.488,1513torchao,float8_weight_only,True,0.000,12.348,2714quanto,INT4,False,12.254,16.139,10915quanto,INT4,True,,,16quanto,INT8,False,17.330,21.814,1517quanto,INT8,True,0.000,12.166,2518quanto,FP8,False,16.395,20.898,1619quanto,FP8,True,0.000,12.332,2420GGUF,Q4_1,False,16.838,21.326,2321GGUF,Q4_1,True,0.000,9.164,2822GGUF,Q8_0,False,21.502,25.973,1523GGUF,Q8_0,True,0.000,12.965,2224GGUF,Q2_k,False,13.264,17.752,2625GGUF,Q2_k,True,0.000,9.164,3126Diffusers,FP8 (e4m3),False,23.682,28.451,1327Diffusers,FP8 (e4m3),True,9.119,15.621,22