▸ AMD· GCN 5.1

AMD Radeon Instinct MI60

Running LLMs on the Radeon Instinct MI60 — the long read: which models fit at which quantisation, and the settings worth changing. · Or what a budget buys

VRAM

32 GB

HIGH-END

BANDWIDTH

1020

GB/S

MODELS Q4

337/449

75%

7B Q4 SPEED

~130

BLAZING

▸ MODEL COVERAGE @ Q475% OF ALL

▸ ESTIMATED SPEED· BY MODEL SIZE @ Q4

Average speeds at Q4 quantization. Actual performance varies by model architecture and context length.

~302

TOK/S

~130

TOK/S

14B

~65

TOK/S

32B

~28

TOK/S

70B

—

39.4GB NEEDED

▸ MEASURED RIG REPORTS

We rent the machine and time every model on it: decode, VRAM peak, concurrency, watts and cost per million tokens. Yours may already be one of them — and two are free to read in full.

SEE THE REPORTS →

▸ SPECIFICATIONS

VRAM

32 GB

BANDWIDTH

1020 GB/s

FP16 COMPUTE

29.5 TFLOPS

TDP

300W

MEMORY

HBM2

ARCHITECTURE

GCN 5.1

STREAM PROCESSORS

4,096

COMPUTE UNITS

PCIE

Gen 4 x16

301

FAST MODELS · >30 TOK/S

Real-time chat speed

337

USABLE · >10 TOK/S

Comfortable for all tasks

337

TOTAL COMPATIBLE

Fit in VRAM at Q4

▸ RENT IT IN THE CLOUD

Buying Radeon Instinct MI60 costs $15–$40k and isn’t practical for most teams. Spin one up by the hour instead:

Rent on Vast.ai →Or RunPod →

Spin up in ~60s. Pay by the second. Cancel anytime.

Some links are affiliate links — we may earn a small commission at no extra cost to you. This helps keep FitMyLLM free and independent.

▸ COMPATIBLE MODELS· 337

Falcon-H1R Tiny 90M0.09B

FALCON·256K CTX· CHAT· REASONING

nomic-embed-text-v1.5 100M0.14B

EMBEDDING·8K CTX· CHAT

SMOLLM·8K CTX· CHAT· VISION

Snowflake Arctic Embed M v2.00.305B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

bge-large-en-v1.5 335M0.335B

EMBEDDING·1K CTX· CHAT

mxbai-embed-large-v10.335B

EMBEDDING·1K CTX· EMBEDDING

Snowflake Arctic Embed L0.335B

EMBEDDING·1K CTX· EMBEDDING

LFM·125K CTX· CHAT· TOOL_USE

GPT-2 Medium 345M0.38B

SMOLLM·8K CTX· CHAT· VISION

FALCON·128K CTX· CHAT

EMBEDDING·8K CTX· EMBEDDING

Snowflake Arctic Embed L v2.00.568B

EMBEDDING·8K CTX· EMBEDDING· MULTILINGUAL

Qwen3-Embedding 0.6B0.6B

EMBEDDING·32K CTX· EMBEDDING· MULTILINGUAL

Falcon-H1R Tiny 0.6B0.6B

FALCON·32K CTX· CHAT· REASONING

Falcon Perception 0.6B0.6B

FALCON·4K CTX· VISION

LFM·125K CTX· CHAT· TOOL_USE

QWEN·32K CTX· CHAT· REASONING

GPT-2 Large 774M0.81B

QWEN·256K CTX· CHAT· CODING· MULTILINGUAL

OTHER·32K CTX· CHAT· VISION

MINICPM·128K CTX· CHAT· REASONING· MULTILINGUAL

LFM2.5-1.2B-Thinking1.2B

LFM·122K CTX· CHAT· REASONING· TOOL_USE

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

OTHER·256K CTX· CHAT· VISION

DeepSeek Coder 1.3B1.35B

DEEPSEEK·16K CTX· CODING

PHI·2K CTX· CHAT· CODING

LFM·125K CTX· CHAT· TOOL_USE· MULTILINGUAL

▸ NEXT STEP

Get personalized recommendations.

See ranked models with benchmark scores, run commands, and precise speed estimates for your Radeon Instinct MI60.

▸Find models for Radeon Instinct MI60 Compare with other GPUs

WHAT THIS CARD IS WORTH

Radeon Instinct MI60 holds 337 of the models in our catalogue and is, in practice, a Q4_K_M card — the largest it takes is Phi-3.5 MoE 42B at Q4_K_M.

TOKENS/SEC PER $100

—

8B at Q4_K_M, so cards compare like for like

VRAM PER $100

—

what memory costs on this card

THE BIGGEST IT TAKES

Phi-3.5 MoE 42B41.9B · Q4_K_M28.6 GB118 tok/sEST

Falcon 40B40B · Q4_K_M27.4 GB27 tok/sEST

InternVL3 38B38B · Q4_K_M26.7 GB28 tok/sEST

Seed-OSS 36B Instruct36B · Q5_K_S28.6 GB26 tok/sEST

c4ai-command-r-v01 35B35B · Q5_K_M28.1 GB27 tok/sEST

Qwen 3.5 35B A3B35B · Q5_K_M27.5 GB192 tok/sEST

Qwen 3.6 35B A3B35B · Q5_K_M27.5 GB192 tok/sEST

Nous Capybara 34B34.4B · Q5_K_M27.9 GB27 tok/sEST

AGAINST CARDS OF SIMILAR MEMORY

M4 Max (48GB)36 GB$2,4992.2 tok/s per $100

M3 Max (48GB)36 GB$2,8991.6 tok/s per $100

M4 Pro (48GB)36 GB$1,7991.9 tok/s per $100

A100 SXM4 40 GB40 GB$10,0001.4 tok/s per $100

Radeon Instinct MI6032 GB——

Neighbours in memory rather than in price: memory decides whether a card can do the job at all, so two cards of the same size at different prices is the comparison you are making. Ordered by memory, then bandwidth — not by the value column, which is worked out from bandwidth and price alone and therefore rewards a cheap card whatever its software stack does to that bandwidth in practice. Read it as one input, not as a ranking.

▸ DEVICE UNDER TEST

AMD Radeon Instinct MI60 — 32 GB VRAM.

▸ RADEON INSTINCT MI60 SPEC

BRAND: AMD
VRAM: 32 GB HBM2
BANDWIDTH: 1020 GB/s
FP16 COMPUTE: 29.5 TFLOPS
FP32 COMPUTE: 14.8 TFLOPS
STREAM PROCESSORS: 4,096
TDP: 300 W
ARCHITECTURE: GCN 5.1

▸ AI CAPABILITY

337/ 449 models @ Q4

With 32 GB VRAM and 1020 GB/s bandwidth, this GPU handles models up to 41.9B parameters.

Speed ≈ bandwidth / model_size × efficiency. A 7B model at Q4 runs at ~117 tok/s.

RENT IT — LIVE PRICES

Checking what the clouds are charging…

§ 01TOP MODELS FOR RADEON INSTINCT MI60

337 FIT · SHOWING 20

MODEL	SIZE	VRAM Q4	TOK/S	AVG
Phi-3.5 MoE 42B	41.9B	26.1 GB	137	56.7
Falcon 40B	40B	24.9 GB	23	20.9
InternVL3 38B	38B	23.7 GB	24	78.9
Seed-OSS 36B Instruct	36B	22.5 GB	25	54.4
c4ai-command-r-v01 35B	35B	21.9 GB	26	27.5
Qwen 3.5 35B A3B	35B	21.9 GB	302	53.3
Qwen 3.6 35B A3B	35B	21.9 GB	302	53.9
Nous Capybara 34B	34.4B	21.5 GB	26	42.0
Yi-1.5 34B	34.4B	21.5 GB	26	45.3
Falcon-H1 34B	34B	21.3 GB	27	66.1
CodeLlama 34B	34B	21.3 GB	27	25.4
Nous Hermes 2 34B	34B	21.3 GB	27	47.0
Phind CodeLlama 34B	34B	21.3 GB	27	68.1
LLaVA-1.6 Yi 34B	34B	21.3 GB	27	47.4
WizardCoder Python 34B	34B	21.3 GB	27	73.2
Yi 34B	34B	21.3 GB	27	33.4
Qwen3-VL 32B Instruct	33.36B	20.9 GB	27	44.6
DeepSeek Coder 33B	33B	20.7 GB	27	26.0
Vicuna 33B	33B	20.7 GB	27	17.2
LLaMA 1 30B	33B	20.7 GB	27	17.8