▸ NVIDIA· FERMI 2.0

NVIDIA GeForce GTX 560M

Running LLMs on the GeForce GTX 560M — the long read: which models fit at which quantisation, and the settings worth changing. · Or what a budget buys

VRAM

2 GB

BUDGET

BANDWIDTH

GB/S

MODELS Q4

25/449

7B Q4 SPEED

SLOW

▸ MODEL COVERAGE @ Q46% OF ALL

▸ ESTIMATED SPEED· BY MODEL SIZE @ Q4

Average speeds at Q4 quantization. Actual performance varies by model architecture and context length.

—

1.7GB NEEDED

—

3.9GB NEEDED

14B

—

7.9GB NEEDED

32B

—

18.0GB NEEDED

70B

—

39.4GB NEEDED

▸ MEASURED RIG REPORTS

We rent the machine and time every model on it: decode, VRAM peak, concurrency, watts and cost per million tokens. Yours may already be one of them — and two are free to read in full.

SEE THE REPORTS →

▸ SPECIFICATIONS

VRAM

2 GB

BANDWIDTH

60 GB/s

FP16 COMPUTE

0.6 TFLOPS

TDP

75W

MEMORY

GDDR5

ARCHITECTURE

Fermi 2.0

CUDA CORES

192

FAST MODELS · >30 TOK/S

Real-time chat speed

USABLE · >10 TOK/S

Comfortable for all tasks

TOTAL COMPATIBLE

Fit in VRAM at Q4

▸ DON’T WANT TO BUY?

Test GeForce GTX 560M (or anything bigger) without committing. Pay by the second, cancel anytime.

Rent on Vast.ai →Or RunPod →

Spin up in ~60s. Pay by the second. Cancel anytime.

Some links are affiliate links — we may earn a small commission at no extra cost to you. This helps keep FitMyLLM free and independent.

▸ COMPATIBLE MODELS· 25

Falcon-H1R Tiny 90M0.09B

FALCON·256K CTX· CHAT· REASONING

nomic-embed-text-v1.5 100M0.14B

EMBEDDING·8K CTX· CHAT

SMOLLM·8K CTX· CHAT· VISION

Snowflake Arctic Embed M v2.00.305B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

bge-large-en-v1.5 335M0.335B

EMBEDDING·1K CTX· CHAT

mxbai-embed-large-v10.335B

EMBEDDING·1K CTX· EMBEDDING

Snowflake Arctic Embed L0.335B

EMBEDDING·1K CTX· EMBEDDING

LFM·125K CTX· CHAT· TOOL_USE

GPT-2 Medium 345M0.38B

SMOLLM·8K CTX· CHAT· VISION

FALCON·128K CTX· CHAT

EMBEDDING·8K CTX· EMBEDDING

Snowflake Arctic Embed L v2.00.568B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

Qwen3-Embedding 0.6B0.6B

EMBEDDING·32K CTX· EMBEDDING· MULTILINGUAL

Falcon-H1R Tiny 0.6B0.6B

FALCON·32K CTX· CHAT· REASONING

Falcon Perception 0.6B0.6B

FALCON·4K CTX· VISION

LFM·125K CTX· CHAT· TOOL_USE

QWEN·32K CTX· CHAT· REASONING

GPT-2 Large 774M0.81B

▸ NEXT STEP

Get personalized recommendations.

See ranked models with benchmark scores, run commands, and precise speed estimates for your GeForce GTX 560M.

▸Find models for GeForce GTX 560M Compare with other GPUs

WHAT THIS CARD IS WORTH

GTX 560M holds 25 of the models in our catalogue and is, in practice, a Q4_K_M card — the largest it takes is Falcon-H1 1.5B at Q4_K_M.

TOKENS/SEC PER $100

—

8B at Q4_K_M, so cards compare like for like

VRAM PER $100

—

what memory costs on this card

THE BIGGEST IT TAKES

Falcon-H1 1.5B1.55B · Q4_K_M1.3 GB47 tok/sEST

Qwen2.5-Coder-1.5B1.5B · Q4_K_M1.3 GB49 tok/sEST

Qwen2 Math 1.5B1.5B · Q4_K_M1.3 GB49 tok/sEST

Qwen 2.5 1.5B1.5B · Q4_K_M1.3 GB49 tok/sEST

Stella en 1.5B v51.5B · Q4_K_M1.3 GB49 tok/sEST

EXAONE-4.0-1.2B1.3B · Q4_K_M1.3 GB57 tok/sEST

MiniCPM-V 4.61.3B · Q5_K_M1.3 GB49 tok/sEST

LFM2.5-1.2B-Thinking1.2B · Q5_K_M1.3 GB53 tok/sEST

AGAINST CARDS OF SIMILAR MEMORY

M2 (8GB)5 GB$5992.8 tok/s per $100

M3 (8GB)5 GB$5992.8 tok/s per $100

M1 (8GB)5 GB$4992.2 tok/s per $100

Radeon RX 6600S4 GB$175—

GTX 560M2 GB——

Neighbours in memory rather than in price: memory decides whether a card can do the job at all, so two cards of the same size at different prices is the comparison you are making. Ordered by memory, then bandwidth — not by the value column, which is worked out from bandwidth and price alone and therefore rewards a cheap card whatever its software stack does to that bandwidth in practice. Read it as one input, not as a ranking.

▸ DEVICE UNDER TEST

NVIDIA GeForce GTX 560M — 2 GB VRAM.

▸ GEFORCE GTX 560M SPEC

BRAND: NVIDIA
VRAM: 2 GB GDDR5
BANDWIDTH: 60 GB/s
FP16 COMPUTE: 0.6 TFLOPS
FP32 COMPUTE: 0.6 TFLOPS
CUDA CORES: 192
TDP: 75 W
ARCHITECTURE: Fermi 2.0

▸ AI CAPABILITY

25/ 449 models @ Q4

With 2 GB VRAM and 60 GB/s bandwidth, this GPU handles models up to 0.81B parameters.

Speed ≈ bandwidth / model_size × efficiency. A 7B model at Q4 runs at ~7 tok/s.

§ 01TOP MODELS FOR GEFORCE GTX 560M

25 FIT · SHOWING 20

MODEL	SIZE	VRAM Q4	TOK/S	AVG
GPT-2 Large 774M	0.81B	1.0 GB	59	5.6
Qwen3 0.6B	0.75B	0.9 GB	64	19.1
LFM2 700M	0.74B	0.9 GB	65	50.4
Qwen 1.5 0.5B	0.62B	0.9 GB	77	9.9
Qwen3-Embedding 0.6B	0.6B	0.9 GB	80	—
Falcon-H1R Tiny 0.6B	0.6B	0.9 GB	80	—
Falcon Perception 0.6B	0.6B	0.9 GB	80	—
BGE-M3	0.568B	0.8 GB	85	63.0
Snowflake Arctic Embed L v2.0	0.568B	0.8 GB	85	—
Falcon-H1 0.5B	0.52B	0.8 GB	92	41.7
Qwen 2.5 0.5B	0.5B	0.8 GB	96	19.4
SmolVLM 500M	0.5B	0.8 GB	96	—
GPT-2 Medium 345M	0.38B	0.7 GB	126	5.9
SmolLM2 360M	0.36B	0.7 GB	133	8.2
LFM2 350M	0.35B	0.7 GB	137	46.3
bge-large-en-v1.5 335M	0.335B	0.7 GB	143	62.3
mxbai-embed-large-v1	0.335B	0.7 GB	143	64.7
Snowflake Arctic Embed L	0.335B	0.7 GB	143	56.0
Snowflake Arctic Embed M v2.0	0.305B	0.7 GB	157	—
Gemma 3 270M	0.27B	0.7 GB	178	12.6