FitMyLLM
NVIDIA/ Ampere

NVIDIA A100 PCIe 80 GB

80 GB
VRAM (Flagship)
1940 GB/s
Memory Bandwidth
267 / 290
Models fit at Q4 (92%)
~222 tok/s
7B Q4 speed (Blazing)
Model coverage at Q492% of all models

Specifications

VRAM
80 GB
Bandwidth
1940 GB/s
FP16 Compute
78 TFLOPS
TDP
300W
Memory
HBM2e
Architecture
Ampere
CUDA Cores
6,912
Tensor Cores
432
PCIe
Gen 4 x16
244
Fast models (>30 tok/s)
Real-time chat speed
267
Usable models (>10 tok/s)
Comfortable for all tasks
267
Total compatible
Fit in VRAM at Q4

Compatible Models(267)

S
GPT-2 124M0.124B
gpt21K ctxchat
12516
tok/s
0% VRAM
S
SmolLM2 135M0.135B
smollm2K ctxchat
11496
tok/s
0% VRAM
S
Nomic Embed Text v1.50.137B
embedding8K ctxembedding
11328
tok/s
0% VRAM
S
mxbai-embed-large-v10.335B
embedding1K ctxembedding
4633
tok/s
1% VRAM
S
Snowflake Arctic Embed L0.335B
embedding1K ctxembedding
4633
tok/s
1% VRAM
S
GPT-2 Medium 345M0.345B
gpt21K ctxchat
4499
tok/s
1% VRAM
S
SmolLM2 360M0.36B
smollm8K ctxchat
4311
tok/s
1% VRAM
S
Qwen 1.5 0.5B0.5B
qwen32K ctxchat
3104
tok/s
1% VRAM
S
Qwen 2.5 0.5B0.5B
qwen32K ctxchat
3104
tok/s
1% VRAM
S
BGE-M30.568B
embedding8K ctxembedding
2732
tok/s
1% VRAM
S
Qwen3 0.6B0.6B
qwen32K ctxchatreasoning
2587
tok/s
1% VRAM
S
GPT-2 Large 774M0.774B
gpt21K ctxchat
2005
tok/s
1% VRAM
S
Qwen3-0.6B0.8B
qwen40K ctxchat
1940
tok/s
1% VRAM
S
Qwen3.5-0.8B0.9B
qwen256K ctxchat
1724
tok/s
1% VRAM
S
Falcon3-1B1B
falcon32K ctxchat
1552
tok/s
1% VRAM
S
InternLM2 1B1B
internlm32K ctxchat
1552
tok/s
1% VRAM
S
TinyLlama-1.1B1.1B
llama2K ctxchat
1411
tok/s
1% VRAM
S
TinyLlama 1.1B1.1B
llama2K ctxchat
1411
tok/s
1% VRAM
S
Llama-3.2-1B1.2B
llama4K ctxchat
1293
tok/s
1% VRAM
S
LFM2.5-1.2B-Thinking1.2B
lfm122K ctxchatreasoningtool_use
1293
tok/s
1% VRAM
S
DeepSeek Coder 1.3B1.3B
deepseek16K ctxcoding
1194
tok/s
2% VRAM
S
OPT 1.3B1.3B
opt2K ctxchat
1194
tok/s
1% VRAM
S
Phi-1 1.3B1.3B
phi2K ctxcoding
1194
tok/s
2% VRAM
S
Phi-1.5 1.3B1.3B
phi2K ctxchatcoding
1194
tok/s
2% VRAM
S
GPT-2 XL 1.5B1.5B
gpt21K ctxchat
1035
tok/s
2% VRAM
S
Qwen2.5-Coder-1.5B1.5B
qwen32K ctxcodingchat
1035
tok/s
2% VRAM
S
Qwen2 Math 1.5B1.5B
qwen4K ctxreasoning
1035
tok/s
2% VRAM
S
Qwen 2.5 1.5B1.5B
qwen32K ctxchatcoding
1035
tok/s
2% VRAM
S
Yi Coder 1.5B1.5B
yi125K ctxcoding
1035
tok/s
2% VRAM
S
stablelm-2-1_6b1.6B
stablelm4K ctxchat
970
tok/s
2% VRAM
S
Qwen3 1.7B1.7B
qwen32K ctxchatreasoning
913
tok/s
2% VRAM
S
SmolLM2 1.7B1.71B
smollm8K ctxchat
908
tok/s
2% VRAM
S
Qwen 1.5 1.8B1.8B
qwen32K ctxchat
862
tok/s
2% VRAM
S
Moondream2 1.9B1.9B
other2K ctxvisionchat
817
tok/s
2% VRAM
S
Granite 4.0 Tiny7BMoE
granite125K ctxchatcodingmultilingual
776
tok/s
6% VRAM
S
Gemma 1 2B2B
gemma8K ctxchat
776
tok/s
2% VRAM
S
Granite 3.0 2B2B
granite128K ctxchatcoding
776
tok/s
2% VRAM
S
Granite 3.1 2B2B
granite128K ctxchatcoding
776
tok/s
2% VRAM
S
Qwen2-VL 2B2.21B
qwen32K ctxchatvision
702
tok/s
2% VRAM
S
Qwen3.5-2B2.3B
qwen256K ctxchat
675
tok/s
2% VRAM

Get personalized recommendations

See ranked models with benchmark scores, run commands, and precise speed estimates for your A100 PCIe 80 GB.

Find Models for A100 PCIe 80 GBCompare with other GPUs