FitMyLLM
NVIDIA/ Hopper

NVIDIA H100 SXM5 64 GB

64 GB
VRAM (Flagship)
2020 GB/s
Memory Bandwidth
257 / 290
Models fit at Q4 (89%)
~231 tok/s
7B Q4 speed (Blazing)
Model coverage at Q489% of all models

Specifications

VRAM
64 GB
Bandwidth
2020 GB/s
FP16 Compute
267.6 TFLOPS
TDP
700W
Memory
HBM3
Architecture
Hopper
CUDA Cores
16,896
Tensor Cores
528
PCIe
Gen 5 x16
239
Fast models (>30 tok/s)
Real-time chat speed
257
Usable models (>10 tok/s)
Comfortable for all tasks
257
Total compatible
Fit in VRAM at Q4

Compatible Models(257)

S
GPT-2 124M0.124B
gpt21K ctxchat
13032
tok/s
1% VRAM
S
SmolLM2 135M0.135B
smollm2K ctxchat
11970
tok/s
1% VRAM
S
Nomic Embed Text v1.50.137B
embedding8K ctxembedding
11796
tok/s
0% VRAM
S
mxbai-embed-large-v10.335B
embedding1K ctxembedding
4824
tok/s
1% VRAM
S
Snowflake Arctic Embed L0.335B
embedding1K ctxembedding
4824
tok/s
1% VRAM
S
GPT-2 Medium 345M0.345B
gpt21K ctxchat
4684
tok/s
1% VRAM
S
SmolLM2 360M0.36B
smollm8K ctxchat
4489
tok/s
1% VRAM
S
Qwen 1.5 0.5B0.5B
qwen32K ctxchat
3232
tok/s
1% VRAM
S
Qwen 2.5 0.5B0.5B
qwen32K ctxchat
3232
tok/s
1% VRAM
S
BGE-M30.568B
embedding8K ctxembedding
2845
tok/s
1% VRAM
S
Qwen3 0.6B0.6B
qwen32K ctxchatreasoning
2693
tok/s
1% VRAM
S
GPT-2 Large 774M0.774B
gpt21K ctxchat
2088
tok/s
1% VRAM
S
Qwen3-0.6B0.8B
qwen40K ctxchat
2020
tok/s
1% VRAM
S
Qwen3.5-0.8B0.9B
qwen256K ctxchat
1796
tok/s
2% VRAM
S
Falcon3-1B1B
falcon32K ctxchat
1616
tok/s
2% VRAM
S
InternLM2 1B1B
internlm32K ctxchat
1616
tok/s
2% VRAM
S
TinyLlama-1.1B1.1B
llama2K ctxchat
1469
tok/s
2% VRAM
S
TinyLlama 1.1B1.1B
llama2K ctxchat
1469
tok/s
2% VRAM
S
Llama-3.2-1B1.2B
llama4K ctxchat
1347
tok/s
2% VRAM
S
LFM2.5-1.2B-Thinking1.2B
lfm122K ctxchatreasoningtool_use
1347
tok/s
1% VRAM
S
DeepSeek Coder 1.3B1.3B
deepseek16K ctxcoding
1243
tok/s
2% VRAM
S
OPT 1.3B1.3B
opt2K ctxchat
1243
tok/s
2% VRAM
S
Phi-1 1.3B1.3B
phi2K ctxcoding
1243
tok/s
2% VRAM
S
Phi-1.5 1.3B1.3B
phi2K ctxchatcoding
1243
tok/s
2% VRAM
S
GPT-2 XL 1.5B1.5B
gpt21K ctxchat
1077
tok/s
2% VRAM
S
Qwen2.5-Coder-1.5B1.5B
qwen32K ctxcodingchat
1077
tok/s
2% VRAM
S
Qwen2 Math 1.5B1.5B
qwen4K ctxreasoning
1077
tok/s
2% VRAM
S
Qwen 2.5 1.5B1.5B
qwen32K ctxchatcoding
1077
tok/s
2% VRAM
S
Yi Coder 1.5B1.5B
yi125K ctxcoding
1077
tok/s
2% VRAM
S
stablelm-2-1_6b1.6B
stablelm4K ctxchat
1010
tok/s
2% VRAM
S
Qwen3 1.7B1.7B
qwen32K ctxchatreasoning
951
tok/s
2% VRAM
S
SmolLM2 1.7B1.71B
smollm8K ctxchat
945
tok/s
2% VRAM
S
Qwen 1.5 1.8B1.8B
qwen32K ctxchat
898
tok/s
2% VRAM
S
Moondream2 1.9B1.9B
other2K ctxvisionchat
851
tok/s
2% VRAM
S
Granite 4.0 Tiny7BMoE
granite125K ctxchatcodingmultilingual
808
tok/s
7% VRAM
S
Gemma 1 2B2B
gemma8K ctxchat
808
tok/s
3% VRAM
S
Granite 3.0 2B2B
granite128K ctxchatcoding
808
tok/s
2% VRAM
S
Granite 3.1 2B2B
granite128K ctxchatcoding
808
tok/s
2% VRAM
S
Qwen2-VL 2B2.21B
qwen32K ctxchatvision
731
tok/s
3% VRAM
S
Qwen3.5-2B2.3B
qwen256K ctxchat
703
tok/s
3% VRAM

Get personalized recommendations

See ranked models with benchmark scores, run commands, and precise speed estimates for your H100 SXM5 64 GB.

Find Models for H100 SXM5 64 GBCompare with other GPUs