Spec sheet
CodeLlama 70B — 70B Dense.
Benchmark scores
HumanEval50.6
MMLU-PRO48.1
MATH48.3
IFEval90.0
BBH56.6
GPQA10.5
MUSR15.6
Arena Elo1056
Run command
Run CodeLlama 70B locally with Ollama — needs 43.3 GB VRAM at Q4_K_M:
$
ollama run codellama:70bCompatible GPUs
30 @ Q4_K_MApple M3 Max (48GB)
48 GB · 400 GB/s
Apple M4 Pro (48GB)
48 GB · 273 GB/s
Apple M4 Max (48GB)
48 GB · 546 GB/s
NVIDIA L40S 48GB
48 GB · 864 GB/s
NVIDIA L40 48GB
48 GB · 864 GB/s
NVIDIA RTX 6000 Ada 48GB
48 GB · 960 GB/s
NVIDIA A40 48GB
48 GB · 696 GB/s
NVIDIA RTX A6000 48GB
48 GB · 768 GB/s
NVIDIA Quadro RTX 8000
48 GB · 672 GB/s
NVIDIA Quadro RTX 8000 Passive
48 GB · 624 GB/s
NVIDIA A40 PCIe
48 GB · 696 GB/s
NVIDIA RTX 6000 Ada Generation
48 GB · 960 GB/s
NVIDIA L20
48 GB · 864 GB/s
AMD Radeon PRO W7800 48 GB
48 GB · 864 GB/s
AMD Radeon PRO W7900
48 GB · 864 GB/s