Commit
·
831ad12
1
Parent(s):
d6f82aa
Upload README.md
Browse filesFipx some simple mistakes
README.md
CHANGED
|
@@ -16,9 +16,9 @@ pipeline_tag: text-generation
|
|
| 16 |
---
|
| 17 |
|
| 18 |
|
| 19 |
-
# LlaMa 2 7b 4-bit Python Coder 👩💻
|
| 20 |
|
| 21 |
-
**LlaMa-2 7b** fine-tuned on the **
|
| 22 |
|
| 23 |
## Pretrained description
|
| 24 |
|
|
@@ -77,6 +77,7 @@ The following `bitsandbytes` quantization config was used during training:
|
|
| 77 |
- PEFT 0.4.0
|
| 78 |
|
| 79 |
### Training metrics
|
|
|
|
| 80 |
{'loss': 1.044, 'learning_rate': 3.571428571428572e-05, 'epoch': 0.01}
|
| 81 |
{'loss': 0.8413, 'learning_rate': 7.142857142857143e-05, 'epoch': 0.01}
|
| 82 |
{'loss': 0.7299, 'learning_rate': 0.00010714285714285715, 'epoch': 0.02}
|
|
@@ -99,8 +100,10 @@ The following `bitsandbytes` quantization config was used during training:
|
|
| 99 |
{'loss': 0.5659, 'learning_rate': 0.00019687629501847898, 'epoch': 0.11}
|
| 100 |
{'loss': 0.5754, 'learning_rate': 0.00019643007196568606, 'epoch': 0.11}
|
| 101 |
{'loss': 0.5936, 'learning_rate': 0.000195954644355717, 'epoch': 0.12}
|
|
|
|
| 102 |
|
| 103 |
### Example of usage
|
|
|
|
| 104 |
```py
|
| 105 |
import torch
|
| 106 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
@@ -144,7 +147,6 @@ print(f"Generated instruction:\n{tokenizer.batch_decode(outputs.detach().cpu().n
|
|
| 144 |
title = { llama-2-7b-int4-python-coder },
|
| 145 |
year = 2023,
|
| 146 |
url = { https://huggingface.co/edumunozsala/llama-2-7b-int4-python-18k-alpaca },
|
| 147 |
-
doi = { },
|
| 148 |
publisher = { Hugging Face }
|
| 149 |
}
|
| 150 |
```
|
|
|
|
| 16 |
---
|
| 17 |
|
| 18 |
|
| 19 |
+
# LlaMa 2 7b 4-bit Python Coder 👩💻
|
| 20 |
|
| 21 |
+
**LlaMa-2 7b** fine-tuned on the **python_code_instructions_18k_alpaca Code instructions dataset** by using the method **QLoRA** in 4-bit with [PEFT](https://github.com/huggingface/peft) library.
|
| 22 |
|
| 23 |
## Pretrained description
|
| 24 |
|
|
|
|
| 77 |
- PEFT 0.4.0
|
| 78 |
|
| 79 |
### Training metrics
|
| 80 |
+
```
|
| 81 |
{'loss': 1.044, 'learning_rate': 3.571428571428572e-05, 'epoch': 0.01}
|
| 82 |
{'loss': 0.8413, 'learning_rate': 7.142857142857143e-05, 'epoch': 0.01}
|
| 83 |
{'loss': 0.7299, 'learning_rate': 0.00010714285714285715, 'epoch': 0.02}
|
|
|
|
| 100 |
{'loss': 0.5659, 'learning_rate': 0.00019687629501847898, 'epoch': 0.11}
|
| 101 |
{'loss': 0.5754, 'learning_rate': 0.00019643007196568606, 'epoch': 0.11}
|
| 102 |
{'loss': 0.5936, 'learning_rate': 0.000195954644355717, 'epoch': 0.12}
|
| 103 |
+
```
|
| 104 |
|
| 105 |
### Example of usage
|
| 106 |
+
|
| 107 |
```py
|
| 108 |
import torch
|
| 109 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
| 147 |
title = { llama-2-7b-int4-python-coder },
|
| 148 |
year = 2023,
|
| 149 |
url = { https://huggingface.co/edumunozsala/llama-2-7b-int4-python-18k-alpaca },
|
|
|
|
| 150 |
publisher = { Hugging Face }
|
| 151 |
}
|
| 152 |
```
|