DeepSeek R1 Distill Llama 70B — 70.6B Dense.
Run DeepSeek R1 Distill Llama 70B locally with Ollama — needs 43.6 GB VRAM at Q4_K_M:
ollama run deepseek-r1:70b| Quant | Bits | VRAM @ 16K | Quality |
|---|---|---|---|
| IQ2_M | 2.93 | 30.1 GB 26.3 + 3.8 KV | low |
| Q2_K | 3.16 | 32.1 GB 28.4 + 3.8 KV | low |
| IQ3_XXS | 3.25 | 32.9 GB 29.2 + 3.8 KV | low |
| IQ3_XS | 3.5 | 35.1 GB 31.4 + 3.8 KV | low |
| Q3_K_S | 3.64 | 36.4 GB 32.6 + 3.8 KV | low |
| IQ3_M | 3.76 | 37.4 GB 33.7 + 3.8 KV | low |
| Q3_K_M | 4 | 39.5 GB 35.8 + 3.8 KV | low |
| Q3_K_L | 4.3 | 42.2 GB 38.4 + 3.8 KV | moderate |
| IQ4_XS | 4.46 | 43.6 GB 39.8 + 3.8 KV | moderate |
| Q4_K_S | 4.67 | 45.5 GB 41.7 + 3.8 KV | moderate |
| Q4_K_M | 4.89 | 47.4 GB 43.6 + 3.8 KV | good |
| Q5_K_S | 5.57 | 53.4 GB 49.6 + 3.8 KV | good |
| Q5_K_M | 5.7 | 54.5 GB 50.8 + 3.8 KV | good |
| Q6_K | 6.56 | 62.1 GB 58.4 + 3.8 KV | excellent |
| Q8_0 | 8.5 | 79.3 GB 75.5 + 3.8 KV | lossless |
| FP16 | 16 | 145.4 GB 141.7 + 3.8 KV | lossless |
Select your GPU above to see speed estimates and compatibility for each quantization.
Rent a GPU and run DeepSeek R1 Distill Llama 70B now
Spin up an A100 / H100 / 4090 in ~60s. Pay by the second. Cancel anytime.
Detects your GPU, recommends the best model, downloads it, and starts chatting — zero config. Benchmarks your speed and contributes anonymous data to improve predictions.
At Q4_K_M quantization. Sorted by minimum VRAM.
Find the best GPU for DeepSeek R1 Distill Llama 70B
Build Hardware for DeepSeek R1 Distill Llama 70BDeepSeek R1 at 70B — most capable R1 distill. Near-original R1 quality.
Run DeepSeek R1 Distill Llama 70B locally with Ollama — needs 43.6 GB VRAM at Q4_K_M:
ollama run deepseek-r1:70b