▸ APPLE· M2

Apple M2 (24GB)

Running LLMs on the Apple M2 (24GB) — the long read: which models fit at which quantisation, and the settings worth changing. · Or what else $999 buys

USABLE MEMORY

16 GB

MID-RANGE

BANDWIDTH

100

GB/S

MODELS Q4

262/449

58%

7B Q4 SPEED

~13

USABLE

▸ MODEL COVERAGE @ Q458% OF ALL

▸ ESTIMATED SPEED· BY MODEL SIZE @ Q4

Average speeds at Q4 quantization. Actual performance varies by model architecture and context length.

~30

TOK/S

~13

TOK/S

14B

TOK/S

32B

—

18.0GB NEEDED

70B

—

39.4GB NEEDED

▸ MEASURED RIG REPORTS

We rent the machine and time every model on it: decode, VRAM peak, concurrency, watts and cost per million tokens. Yours may already be one of them — and two are free to read in full.

SEE THE REPORTS →

▸ SPECIFICATIONS

USABLE MEMORY

16 GB of 24

BANDWIDTH

100 GB/s

FP16 COMPUTE

7.2 TFLOPS

TDP

22W

MEMORY

LPDDR5

ARCHITECTURE

MSRP

$999

FAST MODELS · >30 TOK/S

Real-time chat speed

216

USABLE · >10 TOK/S

Comfortable for all tasks

262

TOTAL COMPATIBLE

Fit in usable memory at Q4

▸ DON’T WANT TO BUY?

Test M2 (24GB) (or anything bigger) without committing. Pay by the second, cancel anytime.

Rent on Vast.ai →Or RunPod →

Spin up in ~60s. Pay by the second. Cancel anytime.

Some links are affiliate links — we may earn a small commission at no extra cost to you. This helps keep FitMyLLM free and independent.

▸ COMPATIBLE MODELS· 262

Falcon-H1R Tiny 90M0.09B

FALCON·256K CTX· CHAT· REASONING

nomic-embed-text-v1.5 100M0.14B

EMBEDDING·8K CTX· CHAT

SMOLLM·8K CTX· CHAT· VISION

Snowflake Arctic Embed M v2.00.305B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

bge-large-en-v1.5 335M0.335B

EMBEDDING·1K CTX· CHAT

mxbai-embed-large-v10.335B

EMBEDDING·1K CTX· EMBEDDING

Snowflake Arctic Embed L0.335B

EMBEDDING·1K CTX· EMBEDDING

LFM·125K CTX· CHAT· TOOL_USE

GPT-2 Medium 345M0.38B

SMOLLM·8K CTX· CHAT· VISION

FALCON·128K CTX· CHAT

EMBEDDING·8K CTX· EMBEDDING

Snowflake Arctic Embed L v2.00.568B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

Qwen3-Embedding 0.6B0.6B

EMBEDDING·32K CTX· EMBEDDING· MULTILINGUAL

Falcon-H1R Tiny 0.6B0.6B

FALCON·32K CTX· CHAT· REASONING

Falcon Perception 0.6B0.6B

FALCON·4K CTX· VISION

LFM·125K CTX· CHAT· TOOL_USE

QWEN·32K CTX· CHAT· REASONING

GPT-2 Large 774M0.81B

QWEN·256K CTX· CHAT· CODING· MULTILINGUAL

OTHER·32K CTX· CHAT· VISION

MINICPM·128K CTX· CHAT· REASONING· MULTILINGUAL

LFM2.5-1.2B-Thinking1.2B

LFM·122K CTX· CHAT· REASONING· TOOL_USE

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

OTHER·256K CTX· CHAT· VISION

DeepSeek Coder 1.3B1.35B

DEEPSEEK·16K CTX· CODING

PHI·2K CTX· CHAT· CODING

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

▸ NEXT STEP

Get personalized recommendations.

See ranked models with benchmark scores, run commands, and precise speed estimates for your Apple M2 (24GB).

▸Find models for Apple M2 (24GB)Compare with other GPUs

WHAT THIS CARD IS WORTH

M2 (24GB) holds 262 of the models in our catalogue and is, in practice, a IQ4_XS card — the largest it takes is ERNIE 4.5 21B A3B at IQ4_XS.

TOKENS/SEC PER $100

1.7

8B at Q4_K_M, so cards compare like for like

USABLE MEMORY PER $100

1.60 GB

what memory costs on this card

THE BIGGEST IT TAKES

ERNIE 4.5 21B A3B21.95B · IQ4_XS13.9 GB45 tok/sEST

GPT-OSS 20B21B · Q4_K_S13.8 GB36 tok/sEST

Reka Flash 321B · Q4_K_S14.2 GB6 tok/sEST

Reka Flash 3.121B · Q4_K_S14.2 GB6 tok/sEST

InternLM2 20B19.8B · Q4_K_M14.3 GB6 tok/sEST

InternLM2.5 20B19.8B · Q4_K_M14.3 GB6 tok/sEST

Ling-lite 16.8B16.8B · Q5_K_M13.6 GB44 tok/sEST

DeepSeek V2 Lite 16B16B · Q5_K_M12.9 GB44 tok/sEST

AGAINST CARDS OF SIMILAR MEMORY

M1 Max (32GB)21 GB$1,4993.0 tok/s per $100

M2 Max (32GB)21 GB$1,7992.5 tok/s per $100

M2 Pro (32GB)21 GB$1,4991.9 tok/s per $100

M4 (32GB)21 GB$1,1991.7 tok/s per $100

M2 (24GB)16 GB$9991.7 tok/s per $100

Neighbours in memory rather than in price: memory decides whether a card can do the job at all, so two cards of the same size at different prices is the comparison you are making. Ordered by memory, then bandwidth — not by the value column, which is worked out from bandwidth and price alone and therefore rewards a cheap card whatever its software stack does to that bandwidth in practice. Read it as one input, not as a ranking.

▸ DEVICE UNDER TEST

Apple M2 (24GB) — 24 GB unified, 16 GB usable.

▸ APPLE M2 (24GB) SPEC

BRAND: Apple
UNIFIED MEMORY: 24 GB
USABLE BY A MODEL: 16 GB
BANDWIDTH: 100 GB/s
FP16 COMPUTE: 7.2 TFLOPS
FP32 COMPUTE: 3.6 TFLOPS
TDP: 22 W
ARCHITECTURE: M2
MSRP: $999

▸ AI CAPABILITY

262/ 449 models @ Q4

With 16 GB of its 24 GB reaching a model and 100 GB/s bandwidth, this machine handles models up to 19.8B parameters.

Speed ≈ bandwidth / model_size × efficiency. A 7B model at Q4 runs at ~11 tok/s.

RENT IT — LIVE PRICES

Checking what the clouds are charging…

§ 01TOP MODELS FOR APPLE M2 (24GB)

262 FIT · SHOWING 20

MODEL	SIZE	VRAM Q4	TOK/S	AVG
InternLM2 20B	19.8B	12.6 GB	4	45.1
InternLM2.5 20B	19.8B	12.6 GB	4	50.9
Ling-lite 16.8B	16.8B	10.8 GB	37	—
DeepSeek V2 Lite 16B	16B	10.3 GB	37	38.0
StarCoder2 15B	15.96B	10.2 GB	6	26.5
DeepSeek-Coder-V2-Lite 15.7B	15.7B	10.1 GB	37	43.0
DeepSeek-VL2 Small 16B	15.7B	10.1 GB	37	43.1
StarCoder 15B	15.5B	10.0 GB	6	21.0
InternVL3 14B	15.12B	9.7 GB	6	38.1
Phi-4-reasoning-vision 15B	15B	9.7 GB	6	42.8
DeepSeek R1 Distill Qwen 14B	14.8B	9.5 GB	6	43.9
DeepCoder 14B	14.8B	9.5 GB	6	38.7
Qwen2.5-Coder-14B	14.8B	9.5 GB	6	41.3
Qwen2.5-14B	14.8B	9.5 GB	6	41.3
Qwen3 14B	14.8B	9.5 GB	6	45.7
phi-4 14B	14.66B	9.4 GB	6	33.7
Phi-4-reasoning 14B	14.66B	9.4 GB	6	33.7
Phi-4-reasoning-plus 14B	14.66B	9.4 GB	6	75.5
Ministral 3 14B	14B	9.0 GB	6	25.9
Phi-3-medium-14b	14B	9.0 GB	6	33.7