▸ NVIDIA· VOLTA

NVIDIA Tesla V100 PCIe 16 GB

Running LLMs on the Tesla V100 PCIe 16 GB — the long read: which models fit at which quantisation, and the settings worth changing. · Or what a budget buys

VRAM

16 GB

MID-RANGE

BANDWIDTH

897

GB/S

MODELS Q4

262/449

58%

7B Q4 SPEED

~103

BLAZING

▸ MODEL COVERAGE @ Q458% OF ALL

▸ ESTIMATED SPEED· BY MODEL SIZE @ Q4

Average speeds at Q4 quantization. Actual performance varies by model architecture and context length.

~239

TOK/S

~103

TOK/S

14B

~51

TOK/S

32B

—

18.0GB NEEDED

70B

—

39.4GB NEEDED

▸ MEASURED RIG REPORTS

We rent the machine and time every model on it: decode, VRAM peak, concurrency, watts and cost per million tokens. Yours may already be one of them — and two are free to read in full.

SEE THE REPORTS →

▸ SPECIFICATIONS

VRAM

16 GB

BANDWIDTH

897 GB/s

FP16 COMPUTE

28.3 TFLOPS

TDP

300W

MEMORY

HBM2

ARCHITECTURE

Volta

CUDA CORES

5,120

TENSOR CORES

640

PCIE

Gen 3 x16

262

FAST MODELS · >30 TOK/S

Real-time chat speed

262

USABLE · >10 TOK/S

Comfortable for all tasks

262

TOTAL COMPATIBLE

Fit in VRAM at Q4

▸ RENT IT IN THE CLOUD

Buying Tesla V100 PCIe 16 GB costs $15–$40k and isn’t practical for most teams. Spin one up by the hour instead:

Rent on Vast.ai →Or RunPod →

Spin up in ~60s. Pay by the second. Cancel anytime.

Some links are affiliate links — we may earn a small commission at no extra cost to you. This helps keep FitMyLLM free and independent.

▸ COMPATIBLE MODELS· 262

Falcon-H1R Tiny 90M0.09B

FALCON·256K CTX· CHAT· REASONING

nomic-embed-text-v1.5 100M0.14B

EMBEDDING·8K CTX· CHAT

SMOLLM·8K CTX· CHAT· VISION

Snowflake Arctic Embed M v2.00.305B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

bge-large-en-v1.5 335M0.335B

EMBEDDING·1K CTX· CHAT

mxbai-embed-large-v10.335B

EMBEDDING·1K CTX· EMBEDDING

Snowflake Arctic Embed L0.335B

EMBEDDING·1K CTX· EMBEDDING

LFM·125K CTX· CHAT· TOOL_USE

GPT-2 Medium 345M0.38B

SMOLLM·8K CTX· CHAT· VISION

FALCON·128K CTX· CHAT

EMBEDDING·8K CTX· EMBEDDING

Snowflake Arctic Embed L v2.00.568B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

Qwen3-Embedding 0.6B0.6B

EMBEDDING·32K CTX· EMBEDDING· MULTILINGUAL

Falcon-H1R Tiny 0.6B0.6B

FALCON·32K CTX· CHAT· REASONING

Falcon Perception 0.6B0.6B

FALCON·4K CTX· VISION

LFM·125K CTX· CHAT· TOOL_USE

QWEN·32K CTX· CHAT· REASONING

GPT-2 Large 774M0.81B

QWEN·256K CTX· CHAT· CODING· MULTILINGUAL

OTHER·32K CTX· CHAT· VISION

MINICPM·128K CTX· CHAT· REASONING· MULTILINGUAL

LFM2.5-1.2B-Thinking1.2B

LFM·122K CTX· CHAT· REASONING· TOOL_USE

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

OTHER·256K CTX· CHAT· VISION

DeepSeek Coder 1.3B1.35B

DEEPSEEK·16K CTX· CODING

PHI·2K CTX· CHAT· CODING

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

▸ NEXT STEP

Get personalized recommendations.

See ranked models with benchmark scores, run commands, and precise speed estimates for your Tesla V100 PCIe 16 GB.

▸Find models for Tesla V100 PCIe 16 GB Compare with other GPUs

WHAT THIS CARD IS WORTH

Tesla V100 PCIe 16 GB holds 262 of the models in our catalogue and is, in practice, a IQ4_XS card — the largest it takes is ERNIE 4.5 21B A3B at IQ4_XS.

TOKENS/SEC PER $100

—

8B at Q4_K_M, so cards compare like for like

VRAM PER $100

—

what memory costs on this card

THE BIGGEST IT TAKES

ERNIE 4.5 21B A3B21.95B · IQ4_XS13.9 GB235 tok/sEST

GPT-OSS 20B21B · Q4_K_S13.8 GB187 tok/sEST

Reka Flash 321B · Q4_K_S14.2 GB32 tok/sEST

Reka Flash 3.121B · Q4_K_S14.2 GB32 tok/sEST

InternLM2 20B19.8B · Q4_K_M14.3 GB32 tok/sEST

InternLM2.5 20B19.8B · Q4_K_M14.3 GB32 tok/sEST

Ling-lite 16.8B16.8B · Q5_K_M13.6 GB230 tok/sEST

DeepSeek V2 Lite 16B16B · Q5_K_M12.9 GB230 tok/sEST

AGAINST CARDS OF SIMILAR MEMORY

M1 Max (32GB)21 GB$1,4993.0 tok/s per $100

M2 Max (32GB)21 GB$1,7992.5 tok/s per $100

M2 Pro (32GB)21 GB$1,4991.9 tok/s per $100

M4 (32GB)21 GB$1,1991.7 tok/s per $100

Neighbours in memory rather than in price: memory decides whether a card can do the job at all, so two cards of the same size at different prices is the comparison you are making. Ordered by memory, then bandwidth — not by the value column, which is worked out from bandwidth and price alone and therefore rewards a cheap card whatever its software stack does to that bandwidth in practice. Read it as one input, not as a ranking.

▸ DEVICE UNDER TEST

NVIDIA Tesla V100 PCIe 16 GB — 16 GB VRAM.

▸ TESLA V100 PCIE 16 GB SPEC

BRAND: NVIDIA
VRAM: 16 GB HBM2
BANDWIDTH: 897 GB/s
FP16 COMPUTE: 28.3 TFLOPS
FP32 COMPUTE: 14.1 TFLOPS
CUDA CORES: 5,120
TENSOR CORES: 640
TDP: 300 W
ARCHITECTURE: Volta

▸ AI CAPABILITY

262/ 449 models @ Q4

With 16 GB VRAM and 897 GB/s bandwidth, this GPU handles models up to 19.8B parameters.

Speed ≈ bandwidth / model_size × efficiency. A 7B model at Q4 runs at ~103 tok/s.

RENT IT — LIVE PRICES

Checking what the clouds are charging…

§ 01TOP MODELS FOR TESLA V100 PCIE 16 GB

262 FIT · SHOWING 20

MODEL	SIZE	VRAM Q4	TOK/S	AVG
InternLM2 20B	19.8B	12.6 GB	36	45.1
InternLM2.5 20B	19.8B	12.6 GB	36	50.9
Ling-lite 16.8B	16.8B	10.8 GB	299	—
DeepSeek V2 Lite 16B	16B	10.3 GB	299	38.0
StarCoder2 15B	15.96B	10.2 GB	45	26.5
DeepSeek-Coder-V2-Lite 15.7B	15.7B	10.1 GB	299	43.0
DeepSeek-VL2 Small 16B	15.7B	10.1 GB	299	43.1
StarCoder 15B	15.5B	10.0 GB	46	21.0
InternVL3 14B	15.12B	9.7 GB	47	38.1
Phi-4-reasoning-vision 15B	15B	9.7 GB	48	42.8
DeepSeek R1 Distill Qwen 14B	14.8B	9.5 GB	48	43.9
DeepCoder 14B	14.8B	9.5 GB	48	38.7
Qwen2.5-Coder-14B	14.8B	9.5 GB	48	41.3
Qwen2.5-14B	14.8B	9.5 GB	48	41.3
Qwen3 14B	14.8B	9.5 GB	48	45.7
phi-4 14B	14.66B	9.4 GB	49	33.7
Phi-4-reasoning 14B	14.66B	9.4 GB	49	33.7
Phi-4-reasoning-plus 14B	14.66B	9.4 GB	49	75.5
Ministral 3 14B	14B	9.0 GB	51	25.9
Phi-3-medium-14b	14B	9.0 GB	51	33.7