gemma-2-2b — 2.6B Parameter Dense LLM
Model Specifications
- Parameters
- 2.6B
- Architecture
- Dense Transformer
- Context Length
- 4K tokens
- Capabilities
- chat
- Release Date
- 2024-04-09
- Provider
- Family
- gemma
VRAM Requirements
| Quantization | BPW | VRAM | Quality |
|---|---|---|---|
| Q4_K_M | 4.89 | 2.1 GB | 94% |
| Q5_K_S | 5.57 | 2.3 GB | 96% |
| Q5_K_M | 5.7 | 2.3 GB | 96% |
| Q6_K | 6.56 | 2.6 GB | 97% |
| Q8_0 | 8.5 | 3.3 GB | 100% |
| FP16 | 16 | 5.7 GB | 100% |
Benchmark Scores
HumanEval27.2
MMLU-PRO18.8
MATH9.1
IFEval57.5
BBH19.7
GPQA7.4
MUSR15.3
MBPP27.9
Arena Elo1159.0
How to Run gemma-2-2b
Run gemma-2-2b locally with Ollama (needs 2.1 GB VRAM at Q4_K_M):
ollama run gemma2:2.6bCompatible GPUs (30)
GPUs that can run gemma-2-2b at Q4_K_M quantization:
NVIDIA Quadro 5000(3GB, 120 GB/s)NVIDIA Quadro 5000 SDI(3GB, 120 GB/s)NVIDIA Tesla C2050(3GB, 144 GB/s)NVIDIA Tesla M2050(3GB, 148 GB/s)NVIDIA Tesla S2050(3GB, 148 GB/s)NVIDIA GeForce GTX 670MX(3GB, 67 GB/s)AMD Radeon HD 7950(3GB, 240 GB/s)AMD Radeon HD 7950 Boost(3GB, 240 GB/s)AMD Radeon HD 7950 Monica BIOS 1(3GB, 240 GB/s)AMD Radeon HD 7950 Monica BIOS 2(3GB, 240 GB/s)AMD Radeon HD 7970(3GB, 264 GB/s)AMD Radeon HD 7970 GHz Edition(3GB, 288 GB/s)AMD Radeon HD 7970 X2(3GB, 264 GB/s)NVIDIA GeForce GTX 770M(3GB, 96 GB/s)NVIDIA GeForce GTX 780(3GB, 288 GB/s)