Hugging Face's logo

google
/
paligemma-3b-pt-224

Image-Text-to-Text
Transformers
Safetensors
paligemma
image-to-text
text-generation-inference
Model card Files
xet
Community
30

Avoiding RuntimeError when loading in 4-bit / 8-bit

#28
by mumeranwaar - opened
base: refs/heads/main
←
from: refs/pr/28
Discussion Files changed
+4
-0
mumeranwaar

Convert to HalfTensor as the model expects it

model_inputs["pixel_values"] = model_inputs["pixel_values"].half()

This avoid this error: RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.cuda.HalfTensor) should be the same

Avoiding RuntimeError when loading in 4-bit / 8-bitf1f76c1d
Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment