FitMyLLM
NVIDIA/ Ada Lovelace

NVIDIA RTX 4000 SFF Ada Generation

20 GB
VRAM (Mid-range)
280 GB/s
Memory Bandwidth
198 / 290
Models fit at Q4 (68%)
~32 tok/s
7B Q4 speed (Fast)
Model coverage at Q468% of all models

Specifications

VRAM
20 GB
Bandwidth
280 GB/s
FP16 Compute
19.2 TFLOPS
TDP
70W
Memory
GDDR6
Architecture
Ada Lovelace
CUDA Cores
6,144
Tensor Cores
192
PCIe
Gen 4 x16
115
Fast models (>30 tok/s)
Real-time chat speed
190
Usable models (>10 tok/s)
Comfortable for all tasks
198
Total compatible
Fit in VRAM at Q4

Compatible Models(198)

S
GPT-2 124M0.124B
gpt21K ctxchat
1806
tok/s
2% VRAM
S
SmolLM2 135M0.135B
smollm2K ctxchat
1659
tok/s
2% VRAM
S
Nomic Embed Text v1.50.137B
embedding8K ctxembedding
1635
tok/s
1% VRAM
S
mxbai-embed-large-v10.335B
embedding1K ctxembedding
669
tok/s
2% VRAM
S
Snowflake Arctic Embed L0.335B
embedding1K ctxembedding
669
tok/s
2% VRAM
S
GPT-2 Medium 345M0.345B
gpt21K ctxchat
649
tok/s
2% VRAM
S
SmolLM2 360M0.36B
smollm8K ctxchat
622
tok/s
2% VRAM
S
Qwen 1.5 0.5B0.5B
qwen32K ctxchat
448
tok/s
4% VRAM
S
Qwen 2.5 0.5B0.5B
qwen32K ctxchat
448
tok/s
4% VRAM
S
BGE-M30.568B
embedding8K ctxembedding
394
tok/s
3% VRAM
S
Qwen3 0.6B0.6B
qwen32K ctxchatreasoning
373
tok/s
3% VRAM
S
GPT-2 Large 774M0.774B
gpt21K ctxchat
289
tok/s
4% VRAM
S
Qwen3-0.6B0.8B
qwen40K ctxchat
280
tok/s
5% VRAM
S
Qwen3.5-0.8B0.9B
qwen256K ctxchat
249
tok/s
5% VRAM
S
Falcon3-1B1B
falcon32K ctxchat
224
tok/s
5% VRAM
S
InternLM2 1B1B
internlm32K ctxchat
224
tok/s
5% VRAM
S
TinyLlama-1.1B1.1B
llama2K ctxchat
204
tok/s
6% VRAM
S
TinyLlama 1.1B1.1B
llama2K ctxchat
204
tok/s
6% VRAM
S
Llama-3.2-1B1.2B
llama4K ctxchat
187
tok/s
6% VRAM
S
LFM2.5-1.2B-Thinking1.2B
lfm122K ctxchatreasoningtool_use
187
tok/s
4% VRAM
S
DeepSeek Coder 1.3B1.3B
deepseek16K ctxcoding
172
tok/s
6% VRAM
S
OPT 1.3B1.3B
opt2K ctxchat
172
tok/s
6% VRAM
S
Phi-1 1.3B1.3B
phi2K ctxcoding
172
tok/s
6% VRAM
S
Phi-1.5 1.3B1.3B
phi2K ctxchatcoding
172
tok/s
6% VRAM
S
GPT-2 XL 1.5B1.5B
gpt21K ctxchat
149
tok/s
6% VRAM
S
Qwen2.5-Coder-1.5B1.5B
qwen32K ctxcodingchat
149
tok/s
7% VRAM
S
Qwen2 Math 1.5B1.5B
qwen4K ctxreasoning
149
tok/s
7% VRAM
S
Qwen 2.5 1.5B1.5B
qwen32K ctxchatcoding
149
tok/s
7% VRAM
S
Yi Coder 1.5B1.5B
yi125K ctxcoding
149
tok/s
7% VRAM
S
stablelm-2-1_6b1.6B
stablelm4K ctxchat
140
tok/s
7% VRAM
S
Qwen3 1.7B1.7B
qwen32K ctxchatreasoning
132
tok/s
7% VRAM
S
SmolLM2 1.7B1.71B
smollm8K ctxchat
131
tok/s
7% VRAM
S
Qwen 1.5 1.8B1.8B
qwen32K ctxchat
124
tok/s
8% VRAM
S
Moondream2 1.9B1.9B
other2K ctxvisionchat
118
tok/s
7% VRAM
S
Granite 4.0 Tiny7BMoE
granite125K ctxchatcodingmultilingual
112
tok/s
22% VRAM
S
Gemma 1 2B2B
gemma8K ctxchat
112
tok/s
8% VRAM
S
Granite 3.0 2B2B
granite128K ctxchatcoding
112
tok/s
8% VRAM
S
Granite 3.1 2B2B
granite128K ctxchatcoding
112
tok/s
8% VRAM
S
Qwen2-VL 2B2.21B
qwen32K ctxchatvision
101
tok/s
8% VRAM
S
Qwen3.5-2B2.3B
qwen256K ctxchat
97
tok/s
9% VRAM

Get personalized recommendations

See ranked models with benchmark scores, run commands, and precise speed estimates for your RTX 4000 SFF Ada Generation.

Find Models for RTX 4000 SFF Ada GenerationCompare with other GPUs