Spec sheet
StableLM 2 12B — 12B Dense.
Benchmark scores
HumanEval35.0
MMLU-PRO19.3
MATH5.4
IFEval54.0
BBH25.3
GPQA2.2
MUSR7.7
Run command
Run StableLM 2 12B locally with Ollama — needs 7.8 GB VRAM at Q4_K_M:
$
ollama run stablelm2:12bCompatible GPUs
30 @ Q4_K_MNVIDIA RTX 4060 Ti 8GB
8 GB · 288 GB/s
NVIDIA RTX 4060
8 GB · 272 GB/s
NVIDIA RTX 3070 Ti
8 GB · 608 GB/s
NVIDIA RTX 3070
8 GB · 448 GB/s
NVIDIA RTX 3060 Ti
8 GB · 448 GB/s
NVIDIA RTX 3050 8GB
8 GB · 224 GB/s
AMD RX 7600
8 GB · 288 GB/s
AMD RX 6650 XT
8 GB · 280 GB/s
Intel Arc A750
8 GB · 512 GB/s
Apple M1 (8GB)
8 GB · 68 GB/s
Apple M2 (8GB)
8 GB · 100 GB/s
Apple M3 (8GB)
8 GB · 100 GB/s
NVIDIA RTX 2080 SUPER
8 GB · 496 GB/s
NVIDIA RTX 2080
8 GB · 448 GB/s
NVIDIA RTX 2070 SUPER
8 GB · 448 GB/s