Spec sheet
Llama 3 70B — 70.6B Dense.
Benchmark scores
HumanEval72.0
MMLU-PRO55.0
MATH50.4
IFEval77.6
BBH81.0
GPQA39.5
MUSR22.3
MBPP69.0
Arena Elo1222
LiveCodeBench19.8
AIME0.0
GPQA Diamond37.9
HLE4.4
Run command
Run Llama 3 70B locally with Ollama — needs 43.6 GB VRAM at Q4_K_M:
$
ollama run llama3:70bCompatible GPUs
30 @ Q4_K_MApple M3 Max (48GB)
48 GB · 400 GB/s
Apple M4 Pro (48GB)
48 GB · 273 GB/s
Apple M4 Max (48GB)
48 GB · 546 GB/s
NVIDIA L40S 48GB
48 GB · 864 GB/s
NVIDIA L40 48GB
48 GB · 864 GB/s
NVIDIA RTX 6000 Ada 48GB
48 GB · 960 GB/s
NVIDIA A40 48GB
48 GB · 696 GB/s
NVIDIA RTX A6000 48GB
48 GB · 768 GB/s
NVIDIA Quadro RTX 8000
48 GB · 672 GB/s
NVIDIA Quadro RTX 8000 Passive
48 GB · 624 GB/s
NVIDIA A40 PCIe
48 GB · 696 GB/s
NVIDIA RTX 6000 Ada Generation
48 GB · 960 GB/s
NVIDIA L20
48 GB · 864 GB/s
AMD Radeon PRO W7800 48 GB
48 GB · 864 GB/s
AMD Radeon PRO W7900
48 GB · 864 GB/s