▸ NVIDIA· BLACKWELL 2.0

NVIDIA RTX PRO 4000 Blackwell SFF

Running LLMs on the RTX PRO 4000 Blackwell SFF — the long read: which models fit at which quantisation, and the settings worth changing. · Or what else $2,249 buys

VRAM

24 GB

HIGH-END

BANDWIDTH

432

GB/S

MODELS Q4

293/449

65%

7B Q4 SPEED

~49

FAST

▸ MODEL COVERAGE @ Q465% OF ALL

▸ ESTIMATED SPEED· BY MODEL SIZE @ Q4

Average speeds at Q4 quantization. Actual performance varies by model architecture and context length.

~115

TOK/S

~49

TOK/S

14B

~25

TOK/S

32B

~11

TOK/S

70B

—

39.4GB NEEDED

▸ MEASURED RIG REPORTS

We rent the machine and time every model on it: decode, VRAM peak, concurrency, watts and cost per million tokens. Yours may already be one of them — and two are free to read in full.

SEE THE REPORTS →

▸ SPECIFICATIONS

VRAM

24 GB

BANDWIDTH

432 GB/s

FP16 COMPUTE

25.7 TFLOPS

TDP

70W

MEMORY

GDDR7

ARCHITECTURE

Blackwell 2.0

CUDA CORES

8,960

TENSOR CORES

280

PCIE

Gen 5 x8

MSRP

$2,249

236

FAST MODELS · >30 TOK/S

Real-time chat speed

293

USABLE · >10 TOK/S

Comfortable for all tasks

293

TOTAL COMPATIBLE

Fit in VRAM at Q4

▸ RENT IT IN THE CLOUD

Buying RTX PRO 4000 Blackwell SFF costs $15–$40k and isn’t practical for most teams. Spin one up by the hour instead:

Rent on Vast.ai →Or RunPod →

Spin up in ~60s. Pay by the second. Cancel anytime.

Some links are affiliate links — we may earn a small commission at no extra cost to you. This helps keep FitMyLLM free and independent.

▸ COMPATIBLE MODELS· 293

Falcon-H1R Tiny 90M0.09B

FALCON·256K CTX· CHAT· REASONING

nomic-embed-text-v1.5 100M0.14B

EMBEDDING·8K CTX· CHAT

SMOLLM·8K CTX· CHAT· VISION

Snowflake Arctic Embed M v2.00.305B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

bge-large-en-v1.5 335M0.335B

EMBEDDING·1K CTX· CHAT

mxbai-embed-large-v10.335B

EMBEDDING·1K CTX· EMBEDDING

Snowflake Arctic Embed L0.335B

EMBEDDING·1K CTX· EMBEDDING

LFM·125K CTX· CHAT· TOOL_USE

GPT-2 Medium 345M0.38B

SMOLLM·8K CTX· CHAT· VISION

FALCON·128K CTX· CHAT

EMBEDDING·8K CTX· EMBEDDING

Snowflake Arctic Embed L v2.00.568B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

Qwen3-Embedding 0.6B0.6B

EMBEDDING·32K CTX· EMBEDDING· MULTILINGUAL

Falcon-H1R Tiny 0.6B0.6B

FALCON·32K CTX· CHAT· REASONING

Falcon Perception 0.6B0.6B

FALCON·4K CTX· VISION

LFM·125K CTX· CHAT· TOOL_USE

QWEN·32K CTX· CHAT· REASONING

GPT-2 Large 774M0.81B

QWEN·256K CTX· CHAT· CODING· MULTILINGUAL

OTHER·32K CTX· CHAT· VISION

MINICPM·128K CTX· CHAT· REASONING· MULTILINGUAL

LFM2.5-1.2B-Thinking1.2B

LFM·122K CTX· CHAT· REASONING· TOOL_USE

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

OTHER·256K CTX· CHAT· VISION

DeepSeek Coder 1.3B1.35B

DEEPSEEK·16K CTX· CODING

PHI·2K CTX· CHAT· CODING

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

▸ NEXT STEP

Get personalized recommendations.

See ranked models with benchmark scores, run commands, and precise speed estimates for your RTX PRO 4000 Blackwell SFF.

▸Find models for RTX PRO 4000 Blackwell SFF Compare with other GPUs

WHAT THIS CARD IS WORTH

RTX PRO 4000 Blackwell SFF holds 293 of the models in our catalogue and is, in practice, a IQ4_XS card — the largest it takes is DeepSeek Coder 33B at IQ4_XS.

TOKENS/SEC PER $100

2.1

8B at Q4_K_M, so cards compare like for like

VRAM PER $100

1.07 GB

what memory costs on this card

THE BIGGEST IT TAKES

DeepSeek Coder 33B33B · IQ4_XS21.6 GB12 tok/sEST

DeepSeek-R1-Distill-Qwen-32B32.8B · IQ4_XS21.5 GB12 tok/sEST

Qwen3 32B32.8B · IQ4_XS21.5 GB12 tok/sEST

Qwen2.5-32B32.5B · IQ4_XS21.3 GB12 tok/sEST

Qwen 2.5 Coder 32B32.5B · IQ4_XS21.3 GB12 tok/sEST

QwQ-32B32.5B · IQ4_XS21.3 GB12 tok/sEST

Granite Switch 4.1 30B Preview32.24B · IQ4_XS21.1 GB12 tok/sEST

OLMo-2-0325-32B32.2B · IQ4_XS21.1 GB12 tok/sEST

AGAINST CARDS OF SIMILAR MEMORY

M3 Max (36GB)27 GB$2,4991.5 tok/s per $100

M3 Pro (36GB)27 GB$1,9991.2 tok/s per $100

RTX 509032 GB$1,9997.9 tok/s per $100

RTX 5090 D32 GB$1,9997.9 tok/s per $100

RTX PRO 4000 Blackwell SFF24 GB$2,2492.1 tok/s per $100

Neighbours in memory rather than in price: memory decides whether a card can do the job at all, so two cards of the same size at different prices is the comparison you are making. Ordered by memory, then bandwidth — not by the value column, which is worked out from bandwidth and price alone and therefore rewards a cheap card whatever its software stack does to that bandwidth in practice. Read it as one input, not as a ranking.

▸ DEVICE UNDER TEST

NVIDIA RTX PRO 4000 Blackwell SFF — 24 GB VRAM.

▸ RTX PRO 4000 BLACKWELL SFF SPEC

BRAND: NVIDIA
VRAM: 24 GB GDDR7
BANDWIDTH: 432 GB/s
FP16 COMPUTE: 25.7 TFLOPS
FP32 COMPUTE: 25.7 TFLOPS
CUDA CORES: 8,960
TENSOR CORES: 280
TDP: 70 W
ARCHITECTURE: Blackwell 2.0
MSRP: $2249

▸ AI CAPABILITY

293/ 449 models @ Q4

With 24 GB VRAM and 432 GB/s bandwidth, this GPU handles models up to 30.5B parameters.

Speed ≈ bandwidth / model_size × efficiency. A 7B model at Q4 runs at ~49 tok/s.

RENT IT — LIVE PRICES

Checking what the clouds are charging…

§ 01TOP MODELS FOR RTX PRO 4000 BLACKWELL SFF

293 FIT · SHOWING 20

MODEL	SIZE	VRAM Q4	TOK/S	AVG
Qwen3 30B A3B	30.5B	19.1 GB	115	46.7
Qwen3-Coder 30B-A3B	30.5B	19.1 GB	105	36.9
Qwen3-30B-A3B Instruct 2507	30.5B	19.1 GB	105	43.0
MPT-30B	30B	18.8 GB	12	26.8
OPT 30B	30B	18.8 GB	12	6.3
Qwen3-Omni 30B-A3B	30B	18.8 GB	115	40.6
Granite 4.1 30B	30B	18.8 GB	12	24.7
TranslateGemma 27B	28.84B	18.1 GB	12	38.6
PaliGemma 2 28B	28B	17.6 GB	12	38.6
ERNIE 4.5 VL 28B A3B Thinking	28B	17.6 GB	115	—
Qwen3.5-27B	27.8B	17.5 GB	12	59.4
Qwen 3.8 27B	27.78B	17.5 GB	12	64.6
gemma-3-27b	27.4B	17.2 GB	13	27.2
gemma-2-27b	27.2B	17.1 GB	13	34.6
Qwen 3.6 27B	27B	17.0 GB	13	41.1
Gemma 4 26B A4B	26B	16.4 GB	86	47.9
Aria 25B A3.9B	25.3B	16.0 GB	89	64.8
Mistral-Small-24B	24B	15.2 GB	14	25.0
Mistral-Small-3.1-24B	24B	15.2 GB	14	28.8
Magistral Small 24B	24B	15.2 GB	14	47.0