FitMyLLM
NVIDIA/ Ada Lovelace

NVIDIA RTX 4000 Ada Generation

20 GB
VRAM (Mid-range)
360 GB/s
Memory Bandwidth
198 / 290
Models fit at Q4 (68%)
~41 tok/s
7B Q4 speed (Fast)
Model coverage at Q468% of all models

Specifications

VRAM
20 GB
Bandwidth
360 GB/s
FP16 Compute
26.7 TFLOPS
TDP
130W
Memory
GDDR6
Architecture
Ada Lovelace
CUDA Cores
6,144
Tensor Cores
192
PCIe
Gen 4 x16
147
Fast models (>30 tok/s)
Real-time chat speed
198
Usable models (>10 tok/s)
Comfortable for all tasks
198
Total compatible
Fit in VRAM at Q4

Compatible Models(198)

S
GPT-2 124M0.124B
gpt21K ctxchat
2323
tok/s
2% VRAM
S
SmolLM2 135M0.135B
smollm2K ctxchat
2133
tok/s
2% VRAM
S
Nomic Embed Text v1.50.137B
embedding8K ctxembedding
2102
tok/s
1% VRAM
S
mxbai-embed-large-v10.335B
embedding1K ctxembedding
860
tok/s
2% VRAM
S
Snowflake Arctic Embed L0.335B
embedding1K ctxembedding
860
tok/s
2% VRAM
S
GPT-2 Medium 345M0.345B
gpt21K ctxchat
835
tok/s
2% VRAM
S
SmolLM2 360M0.36B
smollm8K ctxchat
800
tok/s
2% VRAM
S
Qwen 1.5 0.5B0.5B
qwen32K ctxchat
576
tok/s
4% VRAM
S
Qwen 2.5 0.5B0.5B
qwen32K ctxchat
576
tok/s
4% VRAM
S
BGE-M30.568B
embedding8K ctxembedding
507
tok/s
3% VRAM
S
Qwen3 0.6B0.6B
qwen32K ctxchatreasoning
480
tok/s
3% VRAM
S
GPT-2 Large 774M0.774B
gpt21K ctxchat
372
tok/s
4% VRAM
S
Qwen3-0.6B0.8B
qwen40K ctxchat
360
tok/s
5% VRAM
S
Qwen3.5-0.8B0.9B
qwen256K ctxchat
320
tok/s
5% VRAM
S
Falcon3-1B1B
falcon32K ctxchat
288
tok/s
5% VRAM
S
InternLM2 1B1B
internlm32K ctxchat
288
tok/s
5% VRAM
S
TinyLlama-1.1B1.1B
llama2K ctxchat
262
tok/s
6% VRAM
S
TinyLlama 1.1B1.1B
llama2K ctxchat
262
tok/s
6% VRAM
S
Llama-3.2-1B1.2B
llama4K ctxchat
240
tok/s
6% VRAM
S
LFM2.5-1.2B-Thinking1.2B
lfm122K ctxchatreasoningtool_use
240
tok/s
4% VRAM
S
DeepSeek Coder 1.3B1.3B
deepseek16K ctxcoding
222
tok/s
6% VRAM
S
OPT 1.3B1.3B
opt2K ctxchat
222
tok/s
6% VRAM
S
Phi-1 1.3B1.3B
phi2K ctxcoding
222
tok/s
6% VRAM
S
Phi-1.5 1.3B1.3B
phi2K ctxchatcoding
222
tok/s
6% VRAM
S
GPT-2 XL 1.5B1.5B
gpt21K ctxchat
192
tok/s
6% VRAM
S
Qwen2.5-Coder-1.5B1.5B
qwen32K ctxcodingchat
192
tok/s
7% VRAM
S
Qwen2 Math 1.5B1.5B
qwen4K ctxreasoning
192
tok/s
7% VRAM
S
Qwen 2.5 1.5B1.5B
qwen32K ctxchatcoding
192
tok/s
7% VRAM
S
Yi Coder 1.5B1.5B
yi125K ctxcoding
192
tok/s
7% VRAM
S
stablelm-2-1_6b1.6B
stablelm4K ctxchat
180
tok/s
7% VRAM
S
Qwen3 1.7B1.7B
qwen32K ctxchatreasoning
169
tok/s
7% VRAM
S
SmolLM2 1.7B1.71B
smollm8K ctxchat
168
tok/s
7% VRAM
S
Qwen 1.5 1.8B1.8B
qwen32K ctxchat
160
tok/s
8% VRAM
S
Moondream2 1.9B1.9B
other2K ctxvisionchat
152
tok/s
7% VRAM
S
Granite 4.0 Tiny7BMoE
granite125K ctxchatcodingmultilingual
144
tok/s
22% VRAM
S
Gemma 1 2B2B
gemma8K ctxchat
144
tok/s
8% VRAM
S
Granite 3.0 2B2B
granite128K ctxchatcoding
144
tok/s
8% VRAM
S
Granite 3.1 2B2B
granite128K ctxchatcoding
144
tok/s
8% VRAM
S
Qwen2-VL 2B2.21B
qwen32K ctxchatvision
130
tok/s
8% VRAM
S
Qwen3.5-2B2.3B
qwen256K ctxchat
125
tok/s
9% VRAM

Get personalized recommendations

See ranked models with benchmark scores, run commands, and precise speed estimates for your RTX 4000 Ada Generation.

Find Models for RTX 4000 Ada GenerationCompare with other GPUs