---
title: LLM Workshop Hands-on GPT
license: mit
language:
  - ja
datasets:
  - hotchpotch/fineweb-2-edu-japanese
tags:
  - gpt
  - transformer
  - from-scratch
pipeline_tag: text-generation
---

# LLM Workshop Hands-on GPT Model

本モデルはチュートリアル「[Let's develop LLMs from scratch](https://colab.research.google.com/drive/1g-KjfRWoR4OehKr8FY5MSuFO2etCUf1L)」から学習した小型GPTモデルです。

## 特徴
- vocab_size: 50257
- emb_dim: 128
- n_heads: 8
- n_layers: 4
- context_length: 128

## 使い方
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained(
    "itarutomy/llm_workshop_hands_on_gpt-model",
    trust_remote_code=True
)
print(tok.decode(model.generate(**tok("こんにちは", return_tensors="pt"))[0]))
````