Spec sheet
Mistral-Small-24B — 24B Dense.
Benchmark scores
MMLU-PRO34.4
MATH20.4
IFEval62.8
BBH40.6
GPQA11.1
MUSR10.2
BigCodeBench36.1
Arena Elo1233
LiveCodeBench25.2
AIME4.3
MATH-5004.3
GPQA Diamond46.2
HLE4.1
EQ-Bench Creative Writing v3705
EQ-Bench 3862
Run command
Run Mistral-Small-24B locally with Ollama — needs 15.2 GB VRAM at Q4_K_M:
$
ollama run mistral-small:24bCompatible GPUs
30 @ Q4_K_MNVIDIA RTX 5080
16 GB · 960 GB/s
NVIDIA RTX 5070 Ti
16 GB · 896 GB/s
NVIDIA RTX 4080 SUPER
16 GB · 736 GB/s
NVIDIA RTX 4080
16 GB · 717 GB/s
NVIDIA RTX 4070 Ti SUPER
16 GB · 672 GB/s
NVIDIA RTX 4060 Ti 16GB
16 GB · 288 GB/s
AMD RX 7900 GRE
16 GB · 576 GB/s
AMD RX 7800 XT
16 GB · 624 GB/s
AMD RX 7600 XT
16 GB · 288 GB/s
AMD RX 6950 XT
16 GB · 576 GB/s
AMD RX 6900 XT
16 GB · 512 GB/s
AMD RX 6800 XT
16 GB · 512 GB/s
AMD RX 6800
16 GB · 512 GB/s
Intel Arc A770 16GB
16 GB · 560 GB/s
Apple M1 Pro (16GB)
16 GB · 200 GB/s