▸ NVIDIA· BLACKWELL 2.0

NVIDIA RTX PRO 6000 Blackwell Server

Running LLMs on the RTX PRO 6000 Blackwell Server — the long read: which models fit at which quantisation, and the settings worth changing. · Or what else $9,999 buys

VRAM

96 GB

FLAGSHIP

BANDWIDTH

1790

GB/S

MODELS Q4

392/449

87%

7B Q4 SPEED

~205

BLAZING

▸ MODEL COVERAGE @ Q487% OF ALL

▸ ESTIMATED SPEED· BY MODEL SIZE @ Q4

Average speeds at Q4 quantization. Actual performance varies by model architecture and context length.

~477

TOK/S

~205

TOK/S

14B

~102

TOK/S

32B

~45

TOK/S

70B

~20

TOK/S

▸ MEASURED RIG REPORTS

We rent the machine and time every model on it: decode, VRAM peak, concurrency, watts and cost per million tokens. Yours may already be one of them — and two are free to read in full.

SEE THE REPORTS →

▸ SPECIFICATIONS

VRAM

96 GB

BANDWIDTH

1790 GB/s

FP16 COMPUTE

126 TFLOPS

TDP

600W

MEMORY

GDDR7

ARCHITECTURE

Blackwell 2.0

CUDA CORES

24,064

TENSOR CORES

752

PCIE

Gen 5 x16

MSRP

$9,999

357

FAST MODELS · >30 TOK/S

Real-time chat speed

392

USABLE · >10 TOK/S

Comfortable for all tasks

392

TOTAL COMPATIBLE

Fit in VRAM at Q4

▸ RENT IT IN THE CLOUD

Buying RTX PRO 6000 Blackwell Server costs $15–$40k and isn’t practical for most teams. Spin one up by the hour instead:

Rent on Vast.ai →Or RunPod →

Spin up in ~60s. Pay by the second. Cancel anytime.

Some links are affiliate links — we may earn a small commission at no extra cost to you. This helps keep FitMyLLM free and independent.

▸ COMPATIBLE MODELS· 392

Falcon-H1R Tiny 90M0.09B

FALCON·256K CTX· CHAT· REASONING

nomic-embed-text-v1.5 100M0.14B

EMBEDDING·8K CTX· CHAT

SMOLLM·8K CTX· CHAT· VISION

Snowflake Arctic Embed M v2.00.305B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

bge-large-en-v1.5 335M0.335B

EMBEDDING·1K CTX· CHAT

mxbai-embed-large-v10.335B

EMBEDDING·1K CTX· EMBEDDING

Snowflake Arctic Embed L0.335B

EMBEDDING·1K CTX· EMBEDDING

LFM·125K CTX· CHAT· TOOL_USE

GPT-2 Medium 345M0.38B

SMOLLM·8K CTX· CHAT· VISION

FALCON·128K CTX· CHAT

EMBEDDING·8K CTX· EMBEDDING

Snowflake Arctic Embed L v2.00.568B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

Qwen3-Embedding 0.6B0.6B

EMBEDDING·32K CTX· EMBEDDING· MULTILINGUAL

Falcon-H1R Tiny 0.6B0.6B

FALCON·32K CTX· CHAT· REASONING

Falcon Perception 0.6B0.6B

FALCON·4K CTX· VISION

LFM·125K CTX· CHAT· TOOL_USE

QWEN·32K CTX· CHAT· REASONING

GPT-2 Large 774M0.81B

QWEN·256K CTX· CHAT· CODING· MULTILINGUAL

OTHER·32K CTX· CHAT· VISION

MINICPM·128K CTX· CHAT· REASONING· MULTILINGUAL

LFM2.5-1.2B-Thinking1.2B

LFM·122K CTX· CHAT· REASONING· TOOL_USE

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

OTHER·256K CTX· CHAT· VISION

DeepSeek Coder 1.3B1.35B

DEEPSEEK·16K CTX· CODING

PHI·2K CTX· CHAT· CODING

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

▸ NEXT STEP

Get personalized recommendations.

See ranked models with benchmark scores, run commands, and precise speed estimates for your RTX PRO 6000 Blackwell Server.

▸Find models for RTX PRO 6000 Blackwell Server Compare with other GPUs

WHAT THIS CARD IS WORTH

RTX PRO 6000 Blackwell Server holds 392 of the models in our catalogue and is, in practice, a IQ4_XS card — the largest it takes is dots.llm1.inst 142.8B at IQ4_XS.

TOKENS/SEC PER $100

1.6

8B at Q4_K_M, so cards compare like for like

VRAM PER $100

0.96 GB

what memory costs on this card

THE BIGGEST IT TAKES

dots.llm1.inst 142.8B142.8B · IQ4_XS86.0 GB9 tok/sEST

WizardLM 2 8x22B141B · Q4_K_S85.7 GB31 tok/sEST

Mixtral-8x22B140.6B · Q4_K_S85.5 GB31 tok/sEST

DBRX 132B132B · Q4_K_M83.8 GB32 tok/sEST

Mistral Medium 3.5128B · Q4_K_M82.1 GB9 tok/sEST

Pixtral Large 124B124B · Q4_K_M79.7 GB9 tok/sEST

Nemotron 3 Super 120B-A12B123.61B · Q4_K_M78.4 GB96 tok/sEST

Mistral-Large 123B123B · Q4_K_M79.1 GB9 tok/sEST

AGAINST CARDS OF SIMILAR MEMORY

Radeon Instinct MI300128 GB$12,0005.8 tok/s per $100

Radeon Instinct MI250128 GB$12,0002.9 tok/s per $100

Instinct MI250X128 GB$10,0003.5 tok/s per $100

M1 Ultra (128GB)96 GB$4,9991.4 tok/s per $100

RTX PRO 6000 Blackwell Server96 GB$9,9991.6 tok/s per $100

Neighbours in memory rather than in price: memory decides whether a card can do the job at all, so two cards of the same size at different prices is the comparison you are making. Ordered by memory, then bandwidth — not by the value column, which is worked out from bandwidth and price alone and therefore rewards a cheap card whatever its software stack does to that bandwidth in practice. Read it as one input, not as a ranking.

▸ DEVICE UNDER TEST

NVIDIA RTX PRO 6000 Blackwell Server — 96 GB VRAM.

▸ RTX PRO 6000 BLACKWELL SERVER SPEC

BRAND: NVIDIA
VRAM: 96 GB GDDR7
BANDWIDTH: 1790 GB/s
FP16 COMPUTE: 126 TFLOPS
FP32 COMPUTE: 126 TFLOPS
CUDA CORES: 24,064
TENSOR CORES: 752
TDP: 600 W
ARCHITECTURE: Blackwell 2.0
MSRP: $9999

▸ AI CAPABILITY

392/ 449 models @ Q4

With 96 GB VRAM and 1790 GB/s bandwidth, this GPU handles models up to 132B parameters.

Speed ≈ bandwidth / model_size × efficiency. A 7B model at Q4 runs at ~205 tok/s.

RENT IT — LIVE PRICES

Checking what the clouds are charging…

§ 01TOP MODELS FOR RTX PRO 6000 BLACKWELL SERVER

392 FIT · SHOWING 20

MODEL	SIZE	VRAM Q4	TOK/S	AVG
DBRX 132B	132B	81.2 GB	40	46.3
Mistral Medium 3.5	128B	78.7 GB	11	46.6
Pixtral Large 124B	124B	76.3 GB	12	39.3
Nemotron 3 Super 120B-A12B	123.61B	76.0 GB	119	53.2
Mistral-Large 123B	123B	75.7 GB	12	33.5
Devstral 2 123B	123B	75.7 GB	12	38.1
Qwen 3.5 122B A10B	122B	75.1 GB	143	56.8
Nemotron 3 Super 120B	120B	73.8 GB	119	57.3
Mistral Small 4 119B	119B	73.2 GB	220	50.2
GPT-OSS 120B	117B	72.0 GB	281	54.1
Command A 111B	111B	68.3 GB	13	27.6
GLM 4.5 Air	110B	67.7 GB	119	51.0
Qwen 1.5 110B	110B	67.7 GB	13	33.4
Llama 4 Scout 17B-16E	109B	67.1 GB	84	33.9
Cogito v2 109B MoE	109B	67.1 GB	84	—
Ling 2.6 Flash	107.49B	66.2 GB	194	36.8
Sarvam 105B	105B	64.7 GB	14	48.0
Command-R+ 104B	104B	64.1 GB	14	52.7
Llama-3.2-90B-Vision-Instruct	90B	55.5 GB	16	48.5
Hunyuan A13B	80B	49.4 GB	110	81.1