▸ SPEC SHEET
Yi-1.5 34B — 34.4B Dense.
▸ SPECIFICATIONS
- PARAMETERS
- 34.4B
- ARCHITECTURE
- Dense Transformer
- CONTEXT LENGTH
- 4K tokens
- CAPABILITIES
- chat
- RELEASE DATE
- 2024-05-13
- PROVIDER
- 01.AI
- FAMILY
- yi
▸ VRAM REQUIREMENTS
| QUANT | BPW | VRAM | QUALITY |
|---|---|---|---|
| IQ3_XXS | 3.25 | 14.5 GB | 82% |
| IQ3_XS | 3.5 | 15.5 GB | 84% |
| Q3_K_S | 3.64 | 16.1 GB | 85% |
| IQ3_M | 3.76 | 16.7 GB | 86% |
| Q3_K_M | 4 | 17.7 GB | 88% |
| Q3_K_L | 4.3 | 19.0 GB | 90% |
| IQ4_XS | 4.46 | 19.7 GB | 92% |
| Q4_K_S | 4.67 | 20.6 GB | 93% |
| Q4_K_M | 4.89 | 21.5 GB | 94% |
| Q5_K_S | 5.57 | 24.4 GB | 96% |
| Q5_K_M | 5.7 | 25.0 GB | 96% |
| Q6_K | 6.56 | 28.7 GB | 97% |
| Q8_0 | 8.5 | 37.0 GB | 100% |
| FP16 | 16 | 69.3 GB | 100% |
§ 01BENCHMARK SCORES
HumanEval66.0
MMLU-PRO52.0
MATH77.0
IFEval60.7
BBH44.3
GPQA15.3
MUSR13.1
BigCodeBench33.9
Arena Elo1460.0
§ 02RUN COMMAND
Run Yi-1.5 34B locally with Ollama — needs 21.5 GB VRAM at Q4_K_M:
$
ollama run yi:34b§ 03COMPATIBLE GPUs
30 @ Q4_K_MNVIDIA RTX 4090
24 GB · 1008 GB/s
NVIDIA RTX 3090 Ti
24 GB · 1008 GB/s
NVIDIA RTX 3090
24 GB · 936 GB/s
AMD RX 7900 XTX
24 GB · 960 GB/s
Apple M4 Pro (24GB)
24 GB · 273 GB/s
NVIDIA L4 24GB
24 GB · 300 GB/s
NVIDIA A10 24GB
24 GB · 600 GB/s
Apple M2 (24GB)
24 GB · 100 GB/s
Apple M3 (24GB)
24 GB · 100 GB/s
Apple M4 (24GB)
24 GB · 120 GB/s
NVIDIA Tesla M40 24 GB
24 GB · 288 GB/s
NVIDIA Tesla P10
24 GB · 694 GB/s
NVIDIA Tesla P40
24 GB · 347 GB/s
NVIDIA Quadro RTX 6000
24 GB · 672 GB/s
NVIDIA Quadro RTX 6000 Passive
24 GB · 624 GB/s