▸ NVIDIA· ADA LOVELACE

NVIDIA RTX 6000 Ada Generation

VRAM

48 GB

FLAGSHIP

BANDWIDTH

960

GB/S

MODELS Q4

333/428

78%

7B Q4 SPEED

~110

BLAZING

▸ MODEL COVERAGE @ Q478% OF ALL

▸ ESTIMATED SPEED· BY MODEL SIZE @ Q4

Average speeds at Q4 quantization. Actual performance varies by model architecture and context length.

3B

~256

TOK/S

7B

~110

TOK/S

14B

~55

TOK/S

32B

~24

TOK/S

70B

~11

TOK/S

▸ SPECIFICATIONS

VRAM

48 GB

BANDWIDTH

960 GB/s

FP16 COMPUTE

91.1 TFLOPS

TDP

300W

MEMORY

GDDR6

ARCHITECTURE

Ada Lovelace

CUDA CORES

18,176

TENSOR CORES

568

PCIE

Gen 4 x16

MSRP

$6,800

284

FAST MODELS · >30 TOK/S

Real-time chat speed

333

USABLE · >10 TOK/S

Comfortable for all tasks

333

TOTAL COMPATIBLE

Fit in VRAM at Q4

▸ RENT IT IN THE CLOUD

Buying RTX 6000 Ada Generation costs $15–$40k and isn’t practical for most teams. Spin one up by the hour instead:

Rent on RunPod →Or Vast.ai →

Spin up in ~60s. Pay by the second. Cancel anytime.

Some links are affiliate links — we may earn a small commission at no extra cost to you. This helps keep FitMyLLM free and independent.

▸ COMPATIBLE MODELS· 333

Falcon-H1R Tiny 90M0.09B

FALCON·256K CTX· CHAT· REASONING

TOK/S · 1% VRAM

nomic-embed-text-v1.5 100M0.1B

EMBEDDING·8K CTX· CHAT

TOK/S · 1% VRAM

GPT-2 124M0.124B

GPT2·1K CTX· CHAT

TOK/S · 1% VRAM

SmolLM2 135M0.135B

SMOLLM·2K CTX· CHAT

TOK/S · 1% VRAM

SmolVLM 256M0.256B

SMOLLM·8K CTX· CHAT· VISION

TOK/S · 1% VRAM

Gemma 3 270M0.27B

GEMMA·32K CTX· CHAT

TOK/S · 1% VRAM

Snowflake Arctic Embed M v2.00.305B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

TOK/S · 1% VRAM

bge-large-en-v1.5 335M0.335B

EMBEDDING·1K CTX· CHAT

TOK/S · 1% VRAM

mxbai-embed-large-v10.335B

EMBEDDING·1K CTX· EMBEDDING

TOK/S · 1% VRAM

Snowflake Arctic Embed L0.335B

EMBEDDING·1K CTX· EMBEDDING

TOK/S · 1% VRAM

GPT-2 Medium 345M0.345B

GPT2·1K CTX· CHAT

TOK/S · 1% VRAM

LFM·125K CTX· CHAT· TOOL_USE

TOK/S · 1% VRAM

SmolLM2 360M0.36B

SMOLLM·8K CTX· CHAT

TOK/S · 1% VRAM

Falcon-H1 0.5B0.5B

FALCON·128K CTX· CHAT

TOK/S · 2% VRAM

Qwen 1.5 0.5B0.5B

QWEN·32K CTX· CHAT

TOK/S · 2% VRAM

Qwen 2.5 0.5B0.5B

QWEN·32K CTX· CHAT

TOK/S · 2% VRAM

SmolVLM 500M0.5B

SMOLLM·8K CTX· CHAT· VISION

TOK/S · 2% VRAM

EMBEDDING·8K CTX· EMBEDDING

TOK/S · 2% VRAM

Snowflake Arctic Embed L v2.00.568B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

TOK/S · 2% VRAM

QWEN·32K CTX· CHAT· REASONING

TOK/S · 2% VRAM

Qwen3-Embedding 0.6B0.6B

EMBEDDING·32K CTX· EMBEDDING· MULTILINGUAL

TOK/S · 2% VRAM

Falcon-H1R Tiny 0.6B0.6B

FALCON·32K CTX· CHAT· REASONING

TOK/S · 2% VRAM

Falcon Perception 0.6B0.6B

FALCON·4K CTX· VISION

TOK/S · 2% VRAM

LFM·125K CTX· CHAT· TOOL_USE

TOK/S · 2% VRAM

GPT-2 Large 774M0.774B

GPT2·1K CTX· CHAT

TOK/S · 2% VRAM

Qwen 3.5 0.8B0.8B

QWEN·256K CTX· CHAT· CODING· MULTILINGUAL

TOK/S · 2% VRAM

Qwen3.5-0.8B0.9B

QWEN·256K CTX· CHAT

TOK/S · 2% VRAM

FALCON·32K CTX· CHAT

TOK/S · 2% VRAM

INTERNLM·32K CTX· CHAT

TOK/S · 2% VRAM

OTHER·32K CTX· CHAT· VISION

TOK/S · 2% VRAM

TinyLlama 1.1B1.1B

LLAMA·2K CTX· CHAT

TOK/S · 2% VRAM

LFM2.5-1.2B-Thinking1.2B

LFM·122K CTX· CHAT· REASONING· TOOL_USE

TOK/S · 3% VRAM

Llama-3.2-1B1.2B

LLAMA·4K CTX· CHAT

TOK/S · 3% VRAM

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

TOK/S · 3% VRAM

Zamba2 1.2B1.2B

OTHER·4K CTX· CHAT

TOK/S · 3% VRAM

DeepSeek Coder 1.3B1.3B

DEEPSEEK·16K CTX· CODING

TOK/S · 3% VRAM

EXAONE-4.0-1.2B1.3B

EXAONE·64K CTX· CHAT

TOK/S · 3% VRAM

OPT·2K CTX· CHAT

TOK/S · 3% VRAM

PHI·2K CTX· CODING

TOK/S · 3% VRAM

Phi-1.5 1.3B1.3B

PHI·2K CTX· CHAT· CODING

TOK/S · 3% VRAM

▸ NEXT STEP

Get personalized recommendations.

See ranked models with benchmark scores, run commands, and precise speed estimates for your RTX 6000 Ada Generation.

▸Find models for RTX 6000 Ada Generation Compare with other GPUs

▸ DEVICE UNDER TEST

NVIDIA RTX 6000 Ada Generation — 48 GB VRAM.

▸ RTX 6000 ADA GENERATION SPEC

BRAND: NVIDIA
VRAM: 48 GB GDDR6
BANDWIDTH: 960 GB/s
FP16 COMPUTE: 91.1 TFLOPS
FP32 COMPUTE: 91.1 TFLOPS
CUDA CORES: 18,176
TENSOR CORES: 568
TDP: 300 W
ARCHITECTURE: Ada Lovelace
MSRP: $6800

▸ AI CAPABILITY

333/ 428 models @ Q4

With 48 GB VRAM and 960 GB/s bandwidth, this GPU handles models up to 66B parameters.

Speed ≈ bandwidth / model_size × efficiency. A 7B model at Q4 runs at ~110 tok/s.

§ 01TOP MODELS FOR RTX 6000 ADA GENERATION

333 FIT · SHOWING 20

MODEL	SIZE	VRAM Q4	TOK/S	AVG
OPT 66B	66B	40.8 GB	12	—
LLaMA 1 65B	65.2B	40.3 GB	12	42.6
Jamba 2 Mini	52B	32.3 GB	64	9.2
Jamba 1.5 Mini 52B	51.6B	32.0 GB	64	24.2
Kimi-Linear-48B-A3B	48B	29.8 GB	256	26.6
Nemotron-H 47B	47B	29.2 GB	16	84.6
Mixtral-8x7B	46.7B	29.0 GB	59	18.8
Nous-Hermes-2-Mixtral-8x7B-DPO	46.7B	29.0 GB	59	27.4
Dolphin 2.6 Mixtral 8x7B	46.7B	29.0 GB	59	23.8
Phi-3.5 MoE 42B	41.9B	26.1 GB	116	56.7
Falcon 40B	40B	24.9 GB	19	20.9
InternVL3 38B	38B	23.7 GB	20	78.9
Qwen3.5-35B-A3B	36B	22.5 GB	21	48.5
Seed-OSS 36B Instruct	36B	22.5 GB	21	54.4
c4ai-command-r-v01 35B	35B	21.9 GB	22	27.5
Qwen 3.5 35B A3B	35B	21.9 GB	256	53.3
Qwen 3.6 35B A3B	35B	21.9 GB	256	53.9
Nous Capybara 34B	34.4B	21.5 GB	22	42.0
Yi-1.5 34B	34.4B	21.5 GB	22	45.3
Falcon-H1 34B	34B	21.3 GB	23	66.1