Spec sheet
Mixtral-8x7B — 46.7B MoE.
Benchmark scores
MMLU-PRO29.6
MATH12.2
IFEval59.0
BBH37.1
GPQA9.5
MUSR16.7
GPQA Diamond29.2
LiveCodeBench6.6
MATH-50029.9
HLE4.5
AA Intelligence7.7
SciCode2.8
AIME0.0
Run command
Run Mixtral-8x7B locally with Ollama — needs 29.0 GB VRAM at Q4_K_M:
$
ollama run mixtral:8x7bCompatible GPUs
30 @ Q4_K_MNVIDIA RTX 5090
32 GB · 1792 GB/s
Apple M1 Max (32GB)
32 GB · 400 GB/s
Apple M2 Max (32GB)
32 GB · 400 GB/s
NVIDIA V100 SXM2 32GB
32 GB · 900 GB/s
Apple M2 Pro (32GB)
32 GB · 200 GB/s
Apple M4 (32GB)
32 GB · 120 GB/s
NVIDIA Tesla V100 DGXS 32 GB
32 GB · 897 GB/s
NVIDIA Tesla V100 PCIe 32 GB
32 GB · 897 GB/s
NVIDIA Tesla V100 SXM2 32 GB
32 GB · 898 GB/s
NVIDIA Tesla V100 SXM3 32 GB
32 GB · 981 GB/s
AMD Radeon Instinct MI60
32 GB · 1020 GB/s
NVIDIA Tesla V100S PCIe 32 GB
32 GB · 1130 GB/s
AMD Radeon Instinct MI100
32 GB · 1230 GB/s
NVIDIA RTX 5000 Ada Generation
32 GB · 576 GB/s
NVIDIA GeForce RTX 5090
32 GB · 1790 GB/s