Nous-Hermes-2-Mixtral-8x7B-DPO — 46.7B MoE.
Run Nous-Hermes-2-Mixtral-8x7B-DPO locally with Ollama — needs 29.0 GB VRAM at Q4_K_M:
ollama run nous-hermes2-mixtral:8x7b| Quant | Bits | VRAM @ 16K | Quality |
|---|---|---|---|
| IQ3_XXS | 3.25 | 20.0 GB 19.5 + 0.5 KV | low |
| IQ3_XS | 3.5 | 21.4 GB 20.9 + 0.5 KV | low |
| Q3_K_S | 3.64 | 22.2 GB 21.7 + 0.5 KV | low |
| IQ3_M | 3.76 | 22.9 GB 22.4 + 0.5 KV | low |
| Q3_K_M | 4 | 24.3 GB 23.8 + 0.5 KV | low |
| Q3_K_L | 4.3 | 26.1 GB 25.6 + 0.5 KV | moderate |
| IQ4_XS | 4.46 | 27.0 GB 26.5 + 0.5 KV | moderate |
| Q4_K_S | 4.67 | 28.2 GB 27.7 + 0.5 KV | moderate |
| Q4_K_M | 4.89 | 29.5 GB 29.0 + 0.5 KV | good |
| Q5_K_S | 5.57 | 33.5 GB 33.0 + 0.5 KV | good |
| Q5_K_M | 5.7 | 34.3 GB 33.8 + 0.5 KV | good |
| Q6_K | 6.56 | 39.3 GB 38.8 + 0.5 KV | excellent |
| Q8_0 | 8.5 | 50.6 GB 50.1 + 0.5 KV | lossless |
| FP16 | 16 | 94.4 GB 93.9 + 0.5 KV | lossless |
Select your GPU above to see speed estimates and compatibility for each quantization.
Rent a GPU and run Nous-Hermes-2-Mixtral-8x7B-DPO now
Spin up an A100 / H100 / 4090 in ~60s. Pay by the second. Cancel anytime.
Detects your GPU, recommends the best model, downloads it, and starts chatting — zero config. Benchmarks your speed and contributes anonymous data to improve predictions.
At Q4_K_M quantization. Sorted by minimum VRAM.
Find the best GPU for Nous-Hermes-2-Mixtral-8x7B-DPO
Build Hardware for Nous-Hermes-2-Mixtral-8x7B-DPONous Hermes 2 Mixtral — community fine-tune of Mixtral with improved helpfulness.
Run Nous-Hermes-2-Mixtral-8x7B-DPO locally with Ollama — needs 29.0 GB VRAM at Q4_K_M:
ollama run nous-hermes2-mixtral:8x7b