FitMyLLM
NVIDIA/ Volta

NVIDIA Tesla V100 SXM2 16 GB

16 GB
VRAM (Mid-range)
1130 GB/s
Memory Bandwidth
191 / 290
Models fit at Q4 (66%)
~129 tok/s
7B Q4 speed (Blazing)
Model coverage at Q466% of all models

Specifications

VRAM
16 GB
Bandwidth
1130 GB/s
FP16 Compute
32.7 TFLOPS
TDP
250W
Memory
HBM2
Architecture
Volta
CUDA Cores
5,120
Tensor Cores
640
PCIe
Gen 3 x16
191
Fast models (>30 tok/s)
Real-time chat speed
191
Usable models (>10 tok/s)
Comfortable for all tasks
191
Total compatible
Fit in VRAM at Q4

Compatible Models(191)

S
GPT-2 124M0.124B
gpt21K ctxchat
7290
tok/s
2% VRAM
S
SmolLM2 135M0.135B
smollm2K ctxchat
6696
tok/s
2% VRAM
S
Nomic Embed Text v1.50.137B
embedding8K ctxembedding
6599
tok/s
2% VRAM
S
mxbai-embed-large-v10.335B
embedding1K ctxembedding
2699
tok/s
3% VRAM
S
Snowflake Arctic Embed L0.335B
embedding1K ctxembedding
2699
tok/s
3% VRAM
S
GPT-2 Medium 345M0.345B
gpt21K ctxchat
2620
tok/s
3% VRAM
S
SmolLM2 360M0.36B
smollm8K ctxchat
2511
tok/s
3% VRAM
S
Qwen 1.5 0.5B0.5B
qwen32K ctxchat
1808
tok/s
5% VRAM
S
Qwen 2.5 0.5B0.5B
qwen32K ctxchat
1808
tok/s
5% VRAM
S
BGE-M30.568B
embedding8K ctxembedding
1592
tok/s
4% VRAM
S
Qwen3 0.6B0.6B
qwen32K ctxchatreasoning
1507
tok/s
4% VRAM
S
GPT-2 Large 774M0.774B
gpt21K ctxchat
1168
tok/s
5% VRAM
S
Qwen3-0.6B0.8B
qwen40K ctxchat
1130
tok/s
6% VRAM
S
Qwen3.5-0.8B0.9B
qwen256K ctxchat
1004
tok/s
6% VRAM
S
Falcon3-1B1B
falcon32K ctxchat
904
tok/s
6% VRAM
S
InternLM2 1B1B
internlm32K ctxchat
904
tok/s
6% VRAM
S
TinyLlama-1.1B1.1B
llama2K ctxchat
822
tok/s
7% VRAM
S
TinyLlama 1.1B1.1B
llama2K ctxchat
822
tok/s
7% VRAM
S
Llama-3.2-1B1.2B
llama4K ctxchat
753
tok/s
7% VRAM
S
LFM2.5-1.2B-Thinking1.2B
lfm122K ctxchatreasoningtool_use
753
tok/s
4% VRAM
S
DeepSeek Coder 1.3B1.3B
deepseek16K ctxcoding
695
tok/s
8% VRAM
S
OPT 1.3B1.3B
opt2K ctxchat
695
tok/s
7% VRAM
S
Phi-1 1.3B1.3B
phi2K ctxcoding
695
tok/s
8% VRAM
S
Phi-1.5 1.3B1.3B
phi2K ctxchatcoding
695
tok/s
8% VRAM
S
GPT-2 XL 1.5B1.5B
gpt21K ctxchat
603
tok/s
8% VRAM
S
Qwen2.5-Coder-1.5B1.5B
qwen32K ctxcodingchat
603
tok/s
8% VRAM
S
Qwen2 Math 1.5B1.5B
qwen4K ctxreasoning
603
tok/s
8% VRAM
S
Qwen 2.5 1.5B1.5B
qwen32K ctxchatcoding
603
tok/s
8% VRAM
S
Yi Coder 1.5B1.5B
yi125K ctxcoding
603
tok/s
8% VRAM
S
stablelm-2-1_6b1.6B
stablelm4K ctxchat
565
tok/s
9% VRAM
S
Qwen3 1.7B1.7B
qwen32K ctxchatreasoning
532
tok/s
9% VRAM
S
SmolLM2 1.7B1.71B
smollm8K ctxchat
529
tok/s
9% VRAM
S
Qwen 1.5 1.8B1.8B
qwen32K ctxchat
502
tok/s
9% VRAM
S
Moondream2 1.9B1.9B
other2K ctxvisionchat
476
tok/s
9% VRAM
S
Granite 4.0 Tiny7BMoE
granite125K ctxchatcodingmultilingual
452
tok/s
28% VRAM
S
Gemma 1 2B2B
gemma8K ctxchat
452
tok/s
10% VRAM
S
Granite 3.0 2B2B
granite128K ctxchatcoding
452
tok/s
10% VRAM
S
Granite 3.1 2B2B
granite128K ctxchatcoding
452
tok/s
10% VRAM
S
Qwen2-VL 2B2.21B
qwen32K ctxchatvision
409
tok/s
11% VRAM
S
Qwen3.5-2B2.3B
qwen256K ctxchat
393
tok/s
11% VRAM

Get personalized recommendations

See ranked models with benchmark scores, run commands, and precise speed estimates for your Tesla V100 SXM2 16 GB.

Find Models for Tesla V100 SXM2 16 GBCompare with other GPUs