▸ APPLE· M5 ULTRA

Apple M5 Ultra (192GB)

Running LLMs on the Apple M5 Ultra (192GB) — the long read: which models fit at which quantisation, and the settings worth changing. · Or what a budget buys

USABLE MEMORY

144 GB

FLAGSHIP

BANDWIDTH

1228

GB/S

MODELS Q4

398/449

89%

7B Q4 SPEED

~156

BLAZING

▸ MODEL COVERAGE @ Q489% OF ALL

▸ ESTIMATED SPEED· BY MODEL SIZE @ Q4

Average speeds at Q4 quantization. Actual performance varies by model architecture and context length.

~364

TOK/S

~156

TOK/S

14B

~78

TOK/S

32B

~34

TOK/S

70B

~16

TOK/S

▸ MEASURED RIG REPORTS

We rent the machine and time every model on it: decode, VRAM peak, concurrency, watts and cost per million tokens. Yours may already be one of them — and two are free to read in full.

SEE THE REPORTS →

▸ SPECIFICATIONS

USABLE MEMORY

144 GB of 192

BANDWIDTH

1228 GB/s

FP16 COMPUTE

36 TFLOPS

TDP

60W

MEMORY

Unified

ARCHITECTURE

M5 Ultra

355

FAST MODELS · >30 TOK/S

Real-time chat speed

391

USABLE · >10 TOK/S

Comfortable for all tasks

398

TOTAL COMPATIBLE

Fit in usable memory at Q4

▸ DON’T WANT TO BUY?

Test M5 Ultra (192GB) (or anything bigger) without committing. Pay by the second, cancel anytime.

Rent on Vast.ai →Or RunPod →

Spin up in ~60s. Pay by the second. Cancel anytime.

Some links are affiliate links — we may earn a small commission at no extra cost to you. This helps keep FitMyLLM free and independent.

▸ COMPATIBLE MODELS· 398

Falcon-H1R Tiny 90M0.09B

FALCON·256K CTX· CHAT· REASONING

nomic-embed-text-v1.5 100M0.14B

EMBEDDING·8K CTX· CHAT

SMOLLM·8K CTX· CHAT· VISION

Snowflake Arctic Embed M v2.00.305B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

bge-large-en-v1.5 335M0.335B

EMBEDDING·1K CTX· CHAT

mxbai-embed-large-v10.335B

EMBEDDING·1K CTX· EMBEDDING

Snowflake Arctic Embed L0.335B

EMBEDDING·1K CTX· EMBEDDING

LFM·125K CTX· CHAT· TOOL_USE

GPT-2 Medium 345M0.38B

SMOLLM·8K CTX· CHAT· VISION

FALCON·128K CTX· CHAT

EMBEDDING·8K CTX· EMBEDDING

Snowflake Arctic Embed L v2.00.568B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

Qwen3-Embedding 0.6B0.6B

EMBEDDING·32K CTX· EMBEDDING· MULTILINGUAL

Falcon-H1R Tiny 0.6B0.6B

FALCON·32K CTX· CHAT· REASONING

Falcon Perception 0.6B0.6B

FALCON·4K CTX· VISION

LFM·125K CTX· CHAT· TOOL_USE

QWEN·32K CTX· CHAT· REASONING

GPT-2 Large 774M0.81B

QWEN·256K CTX· CHAT· CODING· MULTILINGUAL

OTHER·32K CTX· CHAT· VISION

MINICPM·128K CTX· CHAT· REASONING· MULTILINGUAL

LFM2.5-1.2B-Thinking1.2B

LFM·122K CTX· CHAT· REASONING· TOOL_USE

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

OTHER·256K CTX· CHAT· VISION

DeepSeek Coder 1.3B1.35B

DEEPSEEK·16K CTX· CODING

PHI·2K CTX· CHAT· CODING

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

▸ NEXT STEP

Get personalized recommendations.

See ranked models with benchmark scores, run commands, and precise speed estimates for your Apple M5 Ultra (192GB).

▸Find models for Apple M5 Ultra (192GB)Compare with other GPUs

WHAT THIS CARD IS WORTH

M5 Ultra (192GB) holds 398 of the models in our catalogue and is, in practice, a IQ4_XS card — the largest it takes is Command A+ at IQ4_XS.

TOKENS/SEC PER $100

—

8B at Q4_K_M, so cards compare like for like

USABLE MEMORY PER $100

—

what memory costs on this card

THE BIGGEST IT TAKES

Command A+218B · IQ4_XS124.5 GB31 tok/sEST

Step 3.7 Flash201.37B · Q4_K_M125.8 GB65 tok/sEST

Falcon 180B180B · Q5_K_S129.1 GB3 tok/sEST

bloom 176.2B176.2B · Q4_K_M125.5 GB4 tok/sEST

dots.llm1.inst 142.8B142.8B · Q6_K123.5 GB4 tok/sEST

WizardLM 2 8x22B141B · Q6_K119.0 GB14 tok/sEST

Mixtral-8x22B140.6B · Q6_K118.7 GB14 tok/sEST

DBRX 132B132B · Q6_K111.4 GB15 tok/sEST

AGAINST CARDS OF SIMILAR MEMORY

Radeon Instinct MI300A192 GB$12,0009.2 tok/s per $100

Radeon Instinct MI308X192 GB$12,0009.2 tok/s per $100

Instinct MI300X192 GB$15,0003.8 tok/s per $100

M5 Ultra (192GB)144 GB——

M4 Ultra (192GB)144 GB$7,4991.2 tok/s per $100

Neighbours in memory rather than in price: memory decides whether a card can do the job at all, so two cards of the same size at different prices is the comparison you are making. Ordered by memory, then bandwidth — not by the value column, which is worked out from bandwidth and price alone and therefore rewards a cheap card whatever its software stack does to that bandwidth in practice. Read it as one input, not as a ranking.

▸ DEVICE UNDER TEST

Apple M5 Ultra (192GB) — 192 GB unified, 144 GB usable.

▸ APPLE M5 ULTRA (192GB) SPEC

BRAND: Apple
UNIFIED MEMORY: 192 GB
USABLE BY A MODEL: 144 GB
BANDWIDTH: 1228 GB/s
FP16 COMPUTE: 36 TFLOPS
TDP: 60 W
ARCHITECTURE: M5 Ultra

▸ AI CAPABILITY

398/ 449 models @ Q4

With 144 GB of its 192 GB reaching a model and 1228 GB/s bandwidth, this machine handles models up to 201.37B parameters.

Speed ≈ bandwidth / model_size × efficiency. A 7B model at Q4 runs at ~140 tok/s.

RENT IT — LIVE PRICES

Checking what the clouds are charging…

§ 01TOP MODELS FOR APPLE M5 ULTRA (192GB)

398 FIT · SHOWING 20

MODEL	SIZE	VRAM Q4	TOK/S	AVG
Step 3.7 Flash	201.37B	123.6 GB	99	53.8
Falcon 180B	180B	110.5 GB	6	51.3
bloom 176.2B	176.2B	108.2 GB	6	15.0
dots.llm1.inst 142.8B	142.8B	87.8 GB	8	—
WizardLM 2 8x22B	141B	86.7 GB	28	42.4
Mixtral-8x22B	140.6B	86.4 GB	28	31.9
DBRX 132B	132B	81.2 GB	30	46.3
Mistral Medium 3.5	128B	78.7 GB	9	46.6
Pixtral Large 124B	124B	76.3 GB	9	39.3
Nemotron 3 Super 120B-A12B	123.61B	76.0 GB	91	53.2
Mistral-Large 123B	123B	75.7 GB	9	33.5
Devstral 2 123B	123B	75.7 GB	9	38.1
Qwen 3.5 122B A10B	122B	75.1 GB	109	56.8
Nemotron 3 Super 120B	120B	73.8 GB	91	57.3
Mistral Small 4 119B	119B	73.2 GB	168	50.2
GPT-OSS 120B	117B	72.0 GB	214	54.1
Command A 111B	111B	68.3 GB	10	27.6
GLM 4.5 Air	110B	67.7 GB	91	51.0
Qwen 1.5 110B	110B	67.7 GB	10	33.4
Llama 4 Scout 17B-16E	109B	67.1 GB	64	33.9