FitMyLLM
Alibaba/Dense

AlibabaQwen 2.5 Coder 32B

Qwen 2.5 Coder 32B — best open coding model. Matches GPT-4 on many code benchmarks.

codingchatreasoning
32.5B
Parameters
128K
Context length
14
Benchmarks
14
Quantizations
5.0M
HF downloads
Architecture
Dense
Released
2024-11-12
Layers
64
KV Heads
8
Head Dim
128
Family
qwen

Quantization Options

Context length:
QuantBitsVRAM @ 16KQuality
IQ3_XXS3.25
16.7 GB
13.7 + 3.0 KV
low
IQ3_XS3.5
17.7 GB
14.7 + 3.0 KV
low
Q3_K_S3.64
18.3 GB
15.3 + 3.0 KV
low
IQ3_M3.76
18.8 GB
15.8 + 3.0 KV
low
Q3_K_M4
19.7 GB
16.7 + 3.0 KV
low
Q3_K_L4.3
21.0 GB
18.0 + 3.0 KV
moderate
IQ4_XS4.46
21.6 GB
18.6 + 3.0 KV
moderate
Q4_K_S4.67
22.5 GB
19.5 + 3.0 KV
moderate
Q4_K_M4.89
23.4 GB
20.4 + 3.0 KV
good
Q5_K_S5.57
26.1 GB
23.1 + 3.0 KV
good
Q5_K_M5.7
26.6 GB
23.6 + 3.0 KV
good
Q6_K6.56
30.1 GB
27.1 + 3.0 KV
excellent
Q8_08.5
38.0 GB
35.0 + 3.0 KV
lossless
FP1616
68.5 GB
65.5 + 3.0 KV
lossless

Select your GPU above to see speed estimates and compatibility for each quantization.

Deploying for a team or in production? Size GPUs, cost & scaling in Enterprise →
▸ READY TO RUN THIS?RENT BY THE HOUR

RENT A GPU AND RUN QWEN 2.5 CODER 32B NOW

Rent on Vast.ai →Or RunPod →

Spin up an A100 / H100 / 4090 in ~60s. Pay by the second. Cancel anytime.

Community Ratings

Loading ratings...

Benchmarks (14)

Arena Elo1232
HumanEval92.7
MATH83.9
IFEval81.5
MBPP77.0
BBH56.4
BigCodeBench53.2
MMLU-PRO50.7
GPQA Diamond41.7
LiveCodeBench29.5
GPQA22.7
MUSR18.5
AIME12.0
HLE3.8

Run this model

▸Easiest way to get started·Beginners
DOCS ↗
curl -fsSL https://ollama.com/install.sh | sh
$ollama run qwen2.5-coder:32b-base-q4_K_M

Downloads and runs automatically. Add --verbose for speed stats.

▸ SETUP GUIDE
>_

Auto-setup with fitmyllm CLI

Detects your GPU, recommends the best model, downloads it, and starts chatting — zero config. Benchmarks your speed and contributes anonymous data to improve predictions.

pip install fitmyllmthen run fitmyllmLearn more
Auto-detect GPULive tok/s in chatSpeed benchmarks9 inference engines
HuggingFaceOllama LibraryGGUF DownloadsBuild Hardware

GPUs that can run this model

At Q4_K_M quantization. Sorted by minimum VRAM.

NVIDIA RTX 4090
24 GB VRAM • 1008 GB/s
NVIDIA
$1599
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA RTX 3090 Ti
24 GB VRAM • 1008 GB/s
NVIDIA
$999
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA RTX 3090
24 GB VRAM • 936 GB/s
NVIDIA
$850
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
AMD RX 7900 XTX
24 GB VRAM • 960 GB/s
AMD
$999
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
Apple M4 Pro (24GB)
24 GB VRAM • 273 GB/s
APPLE
$1399
NVIDIA L4 24GB
24 GB VRAM • 300 GB/s
NVIDIA
$2500
NVIDIA A10 24GB
24 GB VRAM • 600 GB/s
NVIDIA
$3500
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
Apple M2 (24GB)
24 GB VRAM • 100 GB/s
APPLE
$999
Apple M3 (24GB)
24 GB VRAM • 100 GB/s
APPLE
$999
Apple M4 (24GB)
24 GB VRAM • 120 GB/s
APPLE
$699
NVIDIA Tesla M40 24 GB
24 GB VRAM • 288 GB/s
NVIDIA
NVIDIA Tesla P10
24 GB VRAM • 694 GB/s
NVIDIA
NVIDIA Tesla P40
24 GB VRAM • 347 GB/s
NVIDIA
NVIDIA Quadro RTX 6000
24 GB VRAM • 672 GB/s
NVIDIA
$4000
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA Quadro RTX 6000 Passive
24 GB VRAM • 624 GB/s
NVIDIA
$4000
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce RTX 3090
24 GB VRAM • 936 GB/s
NVIDIA
$1499
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA A10 PCIe
24 GB VRAM • 600 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA A10G
24 GB VRAM • 600 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA RTX A5000
24 GB VRAM • 768 GB/s
NVIDIA
$2500
NVIDIA GeForce RTX 3090 Ti
24 GB VRAM • 1010 GB/s
NVIDIA
$1999
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce RTX 4090
24 GB VRAM • 1010 GB/s
NVIDIA
$1599
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA L40 CNX
24 GB VRAM • 864 GB/s
NVIDIA
$5000
NVIDIA L40G
24 GB VRAM • 864 GB/s
NVIDIA
$5000
AMD Radeon RX 7900 XTX
24 GB VRAM • 960 GB/s
AMD
$999
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce RTX 4090 D
24 GB VRAM • 1010 GB/s
NVIDIA
$1599
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce RTX 5090 D V2
24 GB VRAM • 1340 GB/s
NVIDIA
$1999
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA TITAN RTX
24 GB VRAM • 672 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA A30 PCIe
24 GB VRAM • 933 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA A30X
24 GB VRAM • 1220 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA PG506-207
24 GB VRAM • 933 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai

Find the best GPU for Qwen 2.5 Coder 32B

Build Hardware for Qwen 2.5 Coder 32B

Model Card

View on HuggingFace

Qwen 2.5 Coder 32B — best open coding model. Matches GPT-4 on many code benchmarks.

▸ SPEC SHEET

Qwen 2.5 Coder 32B — 32.5B Dense.

▸ SPECIFICATIONS
PARAMETERS
32.5B
ARCHITECTURE
Dense Transformer
CONTEXT LENGTH
128K tokens
CAPABILITIES
coding, chat, reasoning
RELEASE DATE
2024-11-12
PROVIDER
Alibaba
FAMILY
qwen
▸ VRAM REQUIREMENTS
QUANTBPWVRAMQUALITY
IQ3_XXS3.2513.7 GB82%
IQ3_XS3.514.7 GB84%
Q3_K_S3.6415.3 GB85%
IQ3_M3.7615.8 GB86%
Q3_K_M416.7 GB88%
Q3_K_L4.318.0 GB90%
IQ4_XS4.4618.6 GB92%
Q4_K_S4.6719.5 GB93%
Q4_K_M4.8920.4 GB94%
Q5_K_S5.5723.1 GB96%
Q5_K_M5.723.6 GB96%
Q6_K6.5627.1 GB97%
Q8_08.535.0 GB100%
FP161665.5 GB100%
§ 01BENCHMARK SCORES
HumanEval92.7
MMLU-PRO50.7
MATH83.9
IFEval81.5
BBH56.4
GPQA22.7
MUSR18.5
MBPP77.0
BigCodeBench53.2
Arena Elo1232.0
LiveCodeBench29.5
AIME12.0
GPQA Diamond41.7
HLE3.8
§ 02RUN COMMAND

Run Qwen 2.5 Coder 32B locally with Ollama — needs 20.4 GB VRAM at Q4_K_M:

$ollama run qwen2.5-coder:32b
§ 03COMPATIBLE GPUs
30 @ Q4_K_M
NVIDIA RTX 4090
24 GB · 1008 GB/s
NVIDIA RTX 3090 Ti
24 GB · 1008 GB/s
NVIDIA RTX 3090
24 GB · 936 GB/s
AMD RX 7900 XTX
24 GB · 960 GB/s
Apple M4 Pro (24GB)
24 GB · 273 GB/s
NVIDIA L4 24GB
24 GB · 300 GB/s
NVIDIA A10 24GB
24 GB · 600 GB/s
Apple M2 (24GB)
24 GB · 100 GB/s
Apple M3 (24GB)
24 GB · 100 GB/s
Apple M4 (24GB)
24 GB · 120 GB/s
NVIDIA Tesla M40 24 GB
24 GB · 288 GB/s
NVIDIA Tesla P10
24 GB · 694 GB/s
NVIDIA Tesla P40
24 GB · 347 GB/s
NVIDIA Quadro RTX 6000
24 GB · 672 GB/s
NVIDIA Quadro RTX 6000 Passive
24 GB · 624 GB/s