Mistral-Large 123B — 123B Dense.
Run Mistral-Large 123B locally with Ollama — needs 75.7 GB VRAM at Q4_K_M:
ollama run mistral-large:123b| Quant | Bits | VRAM @ 16K | Quality |
|---|---|---|---|
| IQ2_XXS | 2.38 | 41.2 GB 37.1 + 4.1 KV | low |
| IQ2_M | 2.93 | 49.7 GB 45.5 + 4.1 KV | low |
| Q2_K | 3.16 | 53.2 GB 49.1 + 4.1 KV | low |
| IQ3_XXS | 3.25 | 54.6 GB 50.5 + 4.1 KV | low |
| IQ3_XS | 3.5 | 58.4 GB 54.3 + 4.1 KV | low |
| Q3_K_S | 3.64 | 60.6 GB 56.5 + 4.1 KV | low |
| IQ3_M | 3.76 | 62.4 GB 58.3 + 4.1 KV | low |
| Q3_K_M | 4 | 66.1 GB 62.0 + 4.1 KV | low |
| Q3_K_L | 4.3 | 70.7 GB 66.6 + 4.1 KV | moderate |
| IQ4_XS | 4.46 | 73.2 GB 69.1 + 4.1 KV | moderate |
| Q4_K_S | 4.67 | 76.4 GB 72.3 + 4.1 KV | moderate |
| Q4_K_M | 4.89 | 79.8 GB 75.7 + 4.1 KV | good |
| Q5_K_S | 5.57 | 90.3 GB 86.1 + 4.1 KV | good |
| Q5_K_M | 5.7 | 92.3 GB 88.1 + 4.1 KV | good |
| Q6_K | 6.56 | 105.5 GB 101.3 + 4.1 KV | excellent |
| Q8_0 | 8.5 | 135.3 GB 131.2 + 4.1 KV | lossless |
| FP16 | 16 | 250.6 GB 246.5 + 4.1 KV | lossless |
Select your GPU above to see speed estimates and compatibility for each quantization.
Rent a GPU and run Mistral-Large 123B now
Spin up an A100 / H100 / 4090 in ~60s. Pay by the second. Cancel anytime.
Detects your GPU, recommends the best model, downloads it, and starts chatting — zero config. Benchmarks your speed and contributes anonymous data to improve predictions.
At Q4_K_M quantization. Sorted by minimum VRAM.
Find the best GPU for Mistral-Large 123B
Build Hardware for Mistral-Large 123BMistral-Large-Instruct-2407 is an advanced dense Large Language Model (LLM) of 123B parameters with state-of-the-art reasoning, knowledge and coding capabilities.
Run Mistral-Large 123B locally with Ollama — needs 75.7 GB VRAM at Q4_K_M:
ollama run mistral-large:123b