FITMYLLM·AUGUST 17, 2026·OPEN METHODOLOGY · COMMUNITY BENCHMARKS

LIVE·UPDATED DAILY

Independent benchmarks for self-hosted AI infrastructure.

▸Privacy Shield

▸ NVIDIA· AMPERE

NVIDIA A800 SXM4 80 GB

VRAM

80 GB

FLAGSHIP

BANDWIDTH

2040

GB/S

MODELS Q4

382/449

85%

7B Q4 SPEED

~233

BLAZING

▸ MODEL COVERAGE @ Q485% OF ALL

▸ ESTIMATED SPEED· BY MODEL SIZE @ Q4

Average speeds at Q4 quantization. Actual performance varies by model architecture and context length.

3B

~544

TOK/S

7B

~233

TOK/S

14B

~117

TOK/S

32B

~51

TOK/S

70B

~23

TOK/S

▸ SPECIFICATIONS

VRAM

80 GB

BANDWIDTH

2040 GB/s

FP16 COMPUTE

78 TFLOPS

TDP

400W

MEMORY

HBM2e

ARCHITECTURE

Ampere

CUDA CORES

6,912

TENSOR CORES

432

PCIE

Gen 4 x16

351

FAST MODELS · >30 TOK/S

Real-time chat speed

382

USABLE · >10 TOK/S

Comfortable for all tasks

382

TOTAL COMPATIBLE

Fit in VRAM at Q4

▸ RENT IT IN THE CLOUD

Buying A800 SXM4 80 GB costs $15–$40k and isn’t practical for most teams. Spin one up by the hour instead:

Rent on Vast.ai →Or RunPod →

Spin up in ~60s. Pay by the second. Cancel anytime.

Some links are affiliate links — we may earn a small commission at no extra cost to you. This helps keep FitMyLLM free and independent.

▸ COMPATIBLE MODELS· 382

Falcon-H1R Tiny 90M0.09B

FALCON·256K CTX· CHAT· REASONING

TOK/S · 1% VRAM

SmolLM2 135M0.135B

SMOLLM·2K CTX· CHAT

TOK/S · 1% VRAM

nomic-embed-text-v1.5 100M0.14B

EMBEDDING·8K CTX· CHAT

TOK/S · 1% VRAM

GPT-2 124M0.14B

GPT2·1K CTX· CHAT

TOK/S · 1% VRAM

SmolVLM 256M0.256B

SMOLLM·8K CTX· CHAT· VISION

TOK/S · 1% VRAM

Gemma 3 270M0.27B

GEMMA·32K CTX· CHAT

TOK/S · 1% VRAM

Snowflake Arctic Embed M v2.00.305B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

TOK/S · 1% VRAM

bge-large-en-v1.5 335M0.335B

EMBEDDING·1K CTX· CHAT

TOK/S · 1% VRAM

mxbai-embed-large-v10.335B

EMBEDDING·1K CTX· EMBEDDING

TOK/S · 1% VRAM

Snowflake Arctic Embed L0.335B

EMBEDDING·1K CTX· EMBEDDING

TOK/S · 1% VRAM

LFM·125K CTX· CHAT· TOOL_USE

TOK/S · 1% VRAM

SmolLM2 360M0.36B

SMOLLM·8K CTX· CHAT

TOK/S · 1% VRAM

GPT-2 Medium 345M0.38B

GPT2·1K CTX· CHAT

TOK/S · 1% VRAM

Qwen 2.5 0.5B0.5B

QWEN·32K CTX· CHAT

TOK/S · 1% VRAM

SmolVLM 500M0.5B

SMOLLM·8K CTX· CHAT· VISION

TOK/S · 1% VRAM

Falcon-H1 0.5B0.52B

FALCON·128K CTX· CHAT

TOK/S · 1% VRAM

EMBEDDING·8K CTX· EMBEDDING

TOK/S · 1% VRAM

Snowflake Arctic Embed L v2.00.568B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

TOK/S · 1% VRAM

Qwen3-Embedding 0.6B0.6B

EMBEDDING·32K CTX· EMBEDDING· MULTILINGUAL

TOK/S · 1% VRAM

Falcon-H1R Tiny 0.6B0.6B

FALCON·32K CTX· CHAT· REASONING

TOK/S · 1% VRAM

Falcon Perception 0.6B0.6B

FALCON·4K CTX· VISION

TOK/S · 1% VRAM

Qwen 1.5 0.5B0.62B

QWEN·32K CTX· CHAT

TOK/S · 1% VRAM

LFM·125K CTX· CHAT· TOOL_USE

TOK/S · 1% VRAM

Qwen3 0.6B0.75B

QWEN·32K CTX· CHAT· REASONING

TOK/S · 1% VRAM

GPT-2 Large 774M0.81B

GPT2·1K CTX· CHAT

TOK/S · 1% VRAM

Qwen 3.5 0.8B0.87B

QWEN·256K CTX· CHAT· CODING· MULTILINGUAL

TOK/S · 1% VRAM

InternVL3 1B0.94B

OTHER·32K CTX· CHAT· VISION

TOK/S · 1% VRAM

MiniCPM5 1B1.08B

MINICPM·128K CTX· CHAT· REASONING· MULTILINGUAL

TOK/S · 1% VRAM

TinyLlama 1.1B1.1B

LLAMA·2K CTX· CHAT

TOK/S · 1% VRAM

LFM2.5-1.2B-Thinking1.2B

LFM·122K CTX· CHAT· REASONING· TOOL_USE

TOK/S · 2% VRAM

Llama-3.2-1B1.2B

LLAMA·4K CTX· CHAT

TOK/S · 2% VRAM

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

TOK/S · 2% VRAM

Zamba2 1.2B1.2B

OTHER·4K CTX· CHAT

TOK/S · 2% VRAM

EXAONE-4.0-1.2B1.3B

EXAONE·64K CTX· CHAT

TOK/S · 2% VRAM

OPT·2K CTX· CHAT

TOK/S · 2% VRAM

MiniCPM-V 4.61.3B

OTHER·256K CTX· CHAT· VISION

TOK/S · 2% VRAM

DeepSeek Coder 1.3B1.35B

DEEPSEEK·16K CTX· CODING

TOK/S · 2% VRAM

Phi-1 1.3B1.42B

PHI·2K CTX· CODING

TOK/S · 2% VRAM

Phi-1.5 1.3B1.42B

PHI·2K CTX· CHAT· CODING

TOK/S · 2% VRAM

LFM2 8B A1B8.3BMoE

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

TOK/S · 7% VRAM

▸ NEXT STEP

Get personalized recommendations.

See ranked models with benchmark scores, run commands, and precise speed estimates for your A800 SXM4 80 GB.

▸Find models for A800 SXM4 80 GB Compare with other GPUs

▸ DEVICE UNDER TEST

NVIDIA A800 SXM4 80 GB — 80 GB VRAM.

▸ A800 SXM4 80 GB SPEC

BRAND: NVIDIA
VRAM: 80 GB HBM2e
BANDWIDTH: 2040 GB/s
FP16 COMPUTE: 78 TFLOPS
FP32 COMPUTE: 19.5 TFLOPS
CUDA CORES: 6,912
TENSOR CORES: 432
TDP: 400 W
ARCHITECTURE: Ampere

▸ AI CAPABILITY

382/ 449 models @ Q4

With 80 GB VRAM and 2040 GB/s bandwidth, this GPU handles models up to 111B parameters.

Speed ≈ bandwidth / model_size × efficiency. A 7B model at Q4 runs at ~233 tok/s.

§ 01TOP MODELS FOR A800 SXM4 80 GB

382 FIT · SHOWING 20

MODEL	SIZE	VRAM Q4	TOK/S	AVG
Command A 111B	111B	68.3 GB	15	27.6
GLM 4.5 Air	110B	67.7 GB	136	51.0
Qwen 1.5 110B	110B	67.7 GB	15	33.4
Llama 4 Scout 17B-16E	109B	67.1 GB	96	33.9
Cogito v2 109B MoE	109B	67.1 GB	96	—
Ling 2.6 Flash	107.49B	66.2 GB	221	36.8
Sarvam 105B	105B	64.7 GB	16	48.0
Command-R+ 104B	104B	64.1 GB	16	52.7
Llama-3.2-90B-Vision-Instruct	90B	55.5 GB	18	48.5
Hunyuan A13B	80B	49.4 GB	126	81.1
Qwen3-Coder-Next	80B	49.4 GB	544	43.0
Qwen3-Next 80B A3B	80B	49.4 GB	544	49.0
NVLM-D 72B	79.38B	49.0 GB	21	48.7
InternVL3 78B	78B	48.2 GB	21	80.6
Qwen2.5-72B	72.7B	44.9 GB	22	39.7
Qwen2-VL 72B	72.7B	44.9 GB	22	55.5
Qwen 1.5 72B	72B	44.5 GB	23	49.7
Qwen2 Math 72B	72B	44.5 GB	23	49.7
Molmo 72B	72B	44.5 GB	23	54.1
DeepSeek R1 Distill Llama 70B	70.6B	43.6 GB	23	42.4