FitMyLLM
▸ APPLE· M5 ULTRA

Apple M5 Ultra (384GB)

Running LLMs on the Apple M5 Ultra (384GB) — the long read: which models fit at which quantisation, and the settings worth changing. · Or what a budget buys

USABLE MEMORY
288 GB
FLAGSHIP
BANDWIDTH
1228
GB/S
MODELS Q4
425/449
95%
7B Q4 SPEED
~156
BLAZING
▸ MODEL COVERAGE @ Q495% OF ALL
▸ ESTIMATED SPEED· BY MODEL SIZE @ Q4

Average speeds at Q4 quantization. Actual performance varies by model architecture and context length.

3B
~364
TOK/S
7B
~156
TOK/S
14B
~78
TOK/S
32B
~34
TOK/S
70B
~16
TOK/S
▸ MEASURED RIG REPORTS

We rent the machine and time every model on it: decode, VRAM peak, concurrency, watts and cost per million tokens. Yours may already be one of them — and two are free to read in full.

SEE THE REPORTS →
▸ SPECIFICATIONS
USABLE MEMORY
288 GB of 384
BANDWIDTH
1228 GB/s
FP16 COMPUTE
36 TFLOPS
TDP
60W
MEMORY
Unified
ARCHITECTURE
M5 Ultra
374
FAST MODELS · >30 TOK/S
Real-time chat speed
413
USABLE · >10 TOK/S
Comfortable for all tasks
425
TOTAL COMPATIBLE
Fit in usable memory at Q4
▸ DON’T WANT TO BUY?

Test M5 Ultra (384GB) (or anything bigger) without committing. Pay by the second, cancel anytime.

Rent on Vast.ai →Or RunPod →

Spin up in ~60s. Pay by the second. Cancel anytime.

Some links are affiliate links — we may earn a small commission at no extra cost to you. This helps keep FitMyLLM free and independent.

▸ COMPATIBLE MODELS· 425
S
Falcon-H1R Tiny 90M0.09B
FALCON·256K CTX· CHAT· REASONING
12128
TOK/S · 0% MEM
›
S
SmolLM2 135M0.135B
SMOLLM·2K CTX· CHAT
8086
TOK/S · 0% MEM
›
S
nomic-embed-text-v1.5 100M0.14B
EMBEDDING·8K CTX· CHAT
7797
TOK/S · 0% MEM
›
S
GPT-2 124M0.14B
GPT2·1K CTX· CHAT
7797
TOK/S · 0% MEM
›
S
SmolVLM 256M0.256B
SMOLLM·8K CTX· CHAT· VISION
4264
TOK/S · 0% MEM
›
S
Gemma 3 270M0.27B
GEMMA·32K CTX· CHAT
4043
TOK/S · 0% MEM
›
S
Snowflake Arctic Embed M v2.00.305B
EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL
3579
TOK/S · 0% MEM
›
S
bge-large-en-v1.5 335M0.335B
EMBEDDING·1K CTX· CHAT
3258
TOK/S · 0% MEM
›
S
mxbai-embed-large-v10.335B
EMBEDDING·1K CTX· EMBEDDING
3258
TOK/S · 0% MEM
›
S
Snowflake Arctic Embed L0.335B
EMBEDDING·1K CTX· EMBEDDING
3258
TOK/S · 0% MEM
›
S
LFM2 350M0.35B
LFM·125K CTX· CHAT· TOOL_USE
3119
TOK/S · 0% MEM
›
S
SmolLM2 360M0.36B
SMOLLM·8K CTX· CHAT
3032
TOK/S · 0% MEM
›
S
GPT-2 Medium 345M0.38B
GPT2·1K CTX· CHAT
2873
TOK/S · 0% MEM
›
S
Qwen 2.5 0.5B0.5B
QWEN·32K CTX· CHAT
2183
TOK/S · 0% MEM
›
S
SmolVLM 500M0.5B
SMOLLM·8K CTX· CHAT· VISION
2183
TOK/S · 0% MEM
›
S
Falcon-H1 0.5B0.52B
FALCON·128K CTX· CHAT
2099
TOK/S · 0% MEM
›
S
BGE-M30.568B
EMBEDDING·8K CTX· EMBEDDING
1922
TOK/S · 0% MEM
›
S
Snowflake Arctic Embed L v2.00.568B
EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL
1922
TOK/S · 0% MEM
›
S
Qwen3-Embedding 0.6B0.6B
EMBEDDING·32K CTX· EMBEDDING· MULTILINGUAL
1819
TOK/S · 0% MEM
›
S
Falcon-H1R Tiny 0.6B0.6B
FALCON·32K CTX· CHAT· REASONING
1819
TOK/S · 0% MEM
›
S
Falcon Perception 0.6B0.6B
FALCON·4K CTX· VISION
1819
TOK/S · 0% MEM
›
S
Qwen 1.5 0.5B0.62B
QWEN·32K CTX· CHAT
1761
TOK/S · 0% MEM
›
S
LFM2 700M0.74B
LFM·125K CTX· CHAT· TOOL_USE
1475
TOK/S · 0% MEM
›
S
Qwen3 0.6B0.75B
QWEN·32K CTX· CHAT· REASONING
1455
TOK/S · 0% MEM
›
S
GPT-2 Large 774M0.81B
GPT2·1K CTX· CHAT
1348
TOK/S · 0% MEM
›
S
Qwen 3.5 0.8B0.87B
QWEN·256K CTX· CHAT· CODING· MULTILINGUAL
1255
TOK/S · 0% MEM
›
S
InternVL3 1B0.94B
OTHER·32K CTX· CHAT· VISION
1161
TOK/S · 0% MEM
›
S
MiniCPM5 1B1.08B
MINICPM·128K CTX· CHAT· REASONING· MULTILINGUAL
1011
TOK/S · 0% MEM
›
S
TinyLlama 1.1B1.1B
LLAMA·2K CTX· CHAT
992
TOK/S · 0% MEM
›
S
LFM2.5-1.2B-Thinking1.2B
LFM·122K CTX· CHAT· REASONING· TOOL_USE
910
TOK/S · 0% MEM
›
S
Llama-3.2-1B1.2B
LLAMA·4K CTX· CHAT
910
TOK/S · 0% MEM
›
S
LFM2 1.2B1.2B
LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL
910
TOK/S · 0% MEM
›
S
Zamba2 1.2B1.2B
OTHER·4K CTX· CHAT
910
TOK/S · 0% MEM
›
S
EXAONE-4.0-1.2B1.3B
EXAONE·64K CTX· CHAT
840
TOK/S · 0% MEM
›
S
OPT 1.3B1.3B
OPT·2K CTX· CHAT
840
TOK/S · 0% MEM
›
S
MiniCPM-V 4.61.3B
OTHER·256K CTX· CHAT· VISION
840
TOK/S · 0% MEM
›
S
DeepSeek Coder 1.3B1.35B
DEEPSEEK·16K CTX· CODING
809
TOK/S · 0% MEM
›
S
Phi-1 1.3B1.42B
PHI·2K CTX· CODING
769
TOK/S · 0% MEM
›
S
Phi-1.5 1.3B1.42B
PHI·2K CTX· CHAT· CODING
769
TOK/S · 0% MEM
›
S
LFM2 8B A1B8.3BMoE
LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL
728
TOK/S · 2% MEM
›
▸ NEXT STEP

Get personalized recommendations.

See ranked models with benchmark scores, run commands, and precise speed estimates for your Apple M5 Ultra (384GB).

▸Find models for Apple M5 Ultra (384GB)Compare with other GPUs
WHAT THIS CARD IS WORTH

M5 Ultra (384GB) holds 425 of the models in our catalogue and is, in practice, a Q4_K_S card — the largest it takes is MiniMax-M3 at Q4_K_S.

TOKENS/SEC PER $100
—
8B at Q4_K_M, so cards compare like for like
USABLE MEMORY PER $100
—
what memory costs on this card
THE BIGGEST IT TAKES
MiniMax-M3427.04B · Q4_K_S252.3 GB32 tok/sEST
Llama 3.1 405B405B · Q4_K_M252.0 GB2 tok/sEST
Hermes 4 405B405B · Q4_K_M252.0 GB2 tok/sEST
Cogito v2 405B405B · Q4_K_M252.0 GB2 tok/sEST
Llama-4-Maverick-17B-128E400B · Q4_K_M247.7 GB42 tok/sEST
Jamba Large 1.6398B · Q4_K_M246.9 GB8 tok/sEST
Qwen3.5-397B-A17B397B · Q4_K_M245.3 GB42 tok/sEST
Hunyuan-Large389B · Q4_K_M241.3 GB14 tok/sEST
AGAINST CARDS OF SIMILAR MEMORY
M5 Ultra (384GB)288 GB——
M4 Ultra (384GB)288 GB$9,9990.9 tok/s per $100
Radeon Instinct MI325X288 GB$20,0005.5 tok/s per $100
Radeon Instinct MI350X288 GB$25,0003.5 tok/s per $100
Radeon Instinct MI355X288 GB$30,0002.9 tok/s per $100

Neighbours in memory rather than in price: memory decides whether a card can do the job at all, so two cards of the same size at different prices is the comparison you are making. Ordered by memory, then bandwidth — not by the value column, which is worked out from bandwidth and price alone and therefore rewards a cheap card whatever its software stack does to that bandwidth in practice. Read it as one input, not as a ranking.

▸ DEVICE UNDER TEST

Apple M5 Ultra (384GB) — 384 GB unified, 288 GB usable.

▸ APPLE M5 ULTRA (384GB) SPEC
BRAND
Apple
UNIFIED MEMORY
384 GB
USABLE BY A MODEL
288 GB
BANDWIDTH
1228 GB/s
FP16 COMPUTE
36 TFLOPS
TDP
60 W
ARCHITECTURE
M5 Ultra
▸ AI CAPABILITY
425/ 449 models @ Q4

With 288 GB of its 384 GB reaching a model and 1228 GB/s bandwidth, this machine handles models up to 405B parameters.

Speed ≈ bandwidth / model_size × efficiency. A 7B model at Q4 runs at ~140 tok/s.

RENT IT — LIVE PRICES
Checking what the clouds are charging…
§ 01TOP MODELS FOR APPLE M5 ULTRA (384GB)
425 FIT · SHOWING 20
MODELSIZEVRAM Q4TOK/SAVG
Llama 3.1 405B405B248.0 GB348.4
Hermes 4 405B405B248.0 GB333.4
Cogito v2 405B405B248.0 GB3—
Llama-4-Maverick-17B-128E400B245.0 GB6446.1
Jamba Large 1.6398B243.8 GB1211.0
Qwen3.5-397B-A17B397B243.2 GB6468.4
Hunyuan-Large389B238.3 GB2173.9
GLM 4.7358B219.3 GB3464.4
GLM 4.6357B218.7 GB9141.4
GLM 4.5355B217.5 GB9147.5
Nemotron-4 340B340B208.3 GB365.4
MiMo V2.5310.78B190.5 GB7357.5
MiMo V2 Flash 309B309B189.4 GB7372.4
DeepSeek V4 Flash 0731304.18B186.4 GB8462.4
ERNIE-4.5-300B-A47B-Paddle300.5B184.2 GB437.4
ERNIE 4.5 300B-A47B300B183.9 GB2349.8
Hunyuan Hy3-preview295B180.8 GB5262.7
DeepSeek V4 Flash290.94B178.3 GB8459.3
DeepSeek V2 236B236B144.7 GB5240.4
DeepSeek-V2.5 236B236B144.7 GB5264.0