bdbj
/

Llama-2-13b-qpal-hess-msq-2bit

Model card Files Files and versions

Model Card

Base model: meta-llama/Llama-2-13b-hf
Quantization method: Memory constrained MSQ with Q-Palette
Target bit-width: 2
Backend kernel: Q-Palette kernel
Calibration data: RedPajama (Hessian)

How to run

Follow the instruction in https://github.com/snu-mllab/Q-Palette.

References

Model Paper

Downloads last month: 11

Safetensors

Model size

2B params

Tensor type

F16

·

I16

·

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for bdbj/Llama-2-13b-qpal-hess-msq-2bit

Base model

meta-llama/Llama-2-13b-hf

Quantized

(30)

this model

Collection including bdbj/Llama-2-13b-qpal-hess-msq-2bit

Data-aware quantization w/ Q-Palette

4 items • Updated 24 days ago