FITMYLLM · SEPTEMBER 6, 2026 · OPEN METHODOLOGY · COMMUNITY BENCHMARKS
LIVE · UPDATED DAILY
Quantization Options Select your GPU for speed estimates Context length: 4K 8K 16K 32K 64K 128K
Quant Bits VRAM @ 16K Quality IQ2_XXS 2.38 126.9 GB
121.0 + 5.9 KV
low IQ2_M 2.93 154.7 GB
148.8 + 5.9 KV
low Q2_K 3.16 166.4 GB
160.5 + 5.9 KV
low IQ3_XXS 3.25 170.9 GB
165.0 + 5.9 KV
low IQ3_XS 3.5 183.6 GB
177.7 + 5.9 KV
low Q3_K_S 3.64 190.7 GB
184.8 + 5.9 KV
low IQ3_M 3.76 196.7 GB
190.8 + 5.9 KV
low Q3_K_M 4 208.9 GB
203.0 + 5.9 KV
low Q3_K_L 4.3 224.1 GB
218.2 + 5.9 KV
moderate IQ4_XS 4.46 232.2 GB
226.3 + 5.9 KV
moderate Q4_K_S 4.67 242.8 GB
236.9 + 5.9 KV
moderate Q4_K_M 4.89 254.0 GB
248.0 + 5.9 KV
good Q5_K_S 5.57 288.4 GB
282.5 + 5.9 KV
good Q5_K_M 5.7 295.0 GB
289.1 + 5.9 KV
good Q6_K 6.56 338.5 GB
332.6 + 5.9 KV
excellent Q8_0 8.5 436.7 GB
430.8 + 5.9 KV
lossless FP16 16 816.4 GB
810.5 + 5.9 KV
lossless
Select your GPU above to see speed estimates and compatibility for each quantization.
Too big for a single GPU — plan a multi-GPU deployment
Even the lightest quant needs ~127 GB. Size GPUs, replicas, TCO and scaling for a production setup. Open in Enterprise →
▸ READY TO RUN THIS? RENT BY THE HOUR
RENT A GPU AND RUN LLAMA 3.1 405B NOW
Spin up an A100 / H100 / 4090 in ~60s. Pay by the second. Cancel anytime.
Community Ratings Chat Coding Reasoning Creative Vision Roleplay Agentic
Loading ratings...
Run this model IQ2_XXS — 121.0 GB VRAM IQ2_M — 148.8 GB VRAM Q2_K — 160.5 GB VRAM IQ3_XXS — 165.0 GB VRAM IQ3_XS — 177.7 GB VRAM Q3_K_S — 184.8 GB VRAM IQ3_M — 190.8 GB VRAM Q3_K_M — 203.0 GB VRAM Q3_K_L — 218.2 GB VRAM IQ4_XS — 226.3 GB VRAM Q4_K_S — 236.9 GB VRAM Q4_K_M — 248.0 GB VRAM Q5_K_S — 282.5 GB VRAM Q5_K_M — 289.1 GB VRAM Q6_K — 332.6 GB VRAM Q8_0 — 430.8 GB VRAM FP16 — 810.5 GB VRAM
Ollama llama.cpp vLLM LM Studio KoboldCpp Jan Docker
▸ Easiest way to get started · Beginners
DOCS ↗ curl -fsSL https://ollama.com/install.sh | shCOPY
$ ollama run llama3.1:405b-text-q4_K_MCOPY
Downloads and runs automatically. Add --verbose for speed stats.
▸ SETUP GUIDE >_
Auto-setup with fitmyllm CLI Detects your GPU, recommends the best model, downloads it, and starts chatting — zero config. Benchmarks your speed and contributes anonymous data to improve predictions.
Auto-detect GPU Live tok/s in chat Speed benchmarks 9 inference engines
GPUs that can run this model At Q4_K_M quantization. Sorted by minimum VRAM.
Find the best GPU for Llama 3.1 405B
Build Hardware for Llama 3.1 405B Llama 3.1 405B — largest open-weight model. Rivals GPT-4 on many tasks.
Read full model card ▸ COLOPHON FITMYLLM · INDEPENDENT · DATA-DRIVEN
FITMYLLM · EST. 2025 · © 2026
RECOMMENDATIONS FROM PUBLISHED MATH, CORRECTED BY THE COMMUNITY — 30.
▸ SPEC SHEET
Llama 3.1 405B — 405B Dense. ▸ SPECIFICATIONS
PARAMETERS 405B
ARCHITECTURE Dense Transformer
CONTEXT LENGTH 128K tokens
CAPABILITIES chat, coding, reasoning
RELEASE DATE 2024-07-23
PROVIDER Meta
FAMILY llama ▸ VRAM REQUIREMENTS
QUANT BPW VRAM QUALITY IQ2_XXS 2.38 121.0 GB 65% IQ2_M 2.93 148.8 GB 75% Q2_K 3.16 160.5 GB 78% IQ3_XXS 3.25 165.0 GB 82% IQ3_XS 3.5 177.7 GB 84% Q3_K_S 3.64 184.8 GB 85% IQ3_M 3.76 190.8 GB 86% Q3_K_M 4 203.0 GB 88% Q3_K_L 4.3 218.2 GB 90% IQ4_XS 4.46 226.3 GB 92% Q4_K_S 4.67 236.9 GB 93% Q4_K_M 4.89 248.0 GB 94% Q5_K_S 5.57 282.5 GB 96% Q5_K_M 5.7 289.1 GB 96% Q6_K 6.56 332.6 GB 97% Q8_0 8.5 430.8 GB 100% FP16 16 810.5 GB 100%
§ 01 BENCHMARK SCORES
HumanEval 89.0
MMLU-PRO 73.3
MATH 73.8
IFEval 88.6
BBH 88.5
GPQA 51.1
Arena Elo 1501.0
GPQA Diamond 72.7
LiveCodeBench 68.6
AIME 69.7
HLE 10.3
AA Intelligence 18.6
AA Coding 16.0
AA Math 69.7
aa_ifbench 34.8
aa_terminal_bench 9.8
aa_tau2 26.6
aa_scicode 34.6
aa_lcr 20.0
MATH-500 3.0
§ 02 RUN COMMAND
Run Llama 3.1 405B locally with Ollama — needs 248.0 GB VRAM at Q4_K_M:
$ ollama run llama3.1:405b
§ 03 COMPATIBLE GPUs
5 @ Q4_K_M Feedback 