Architecture
MoE
Released
2026-08-12
Layers
92
KV Heads
4
Head Dim
256
Family
qwen
Quantization Options
Context length:
| Quant | Bits | VRAM @ 16K | Quality |
|---|---|---|---|
| IQ2_XXS | 2.38 | 729.3 GB 728.2 + 1.1 KV | low |
| IQ2_M | 2.93 | 897.5 GB 896.4 + 1.1 KV | low |
| Q2_K | 3.16 | 967.8 GB 966.7 + 1.1 KV | low |
| IQ3_XXS | 3.25 | 995.3 GB 994.2 + 1.1 KV | low |
| IQ3_XS | 3.5 | 1071.8 GB 1070.7 + 1.1 KV | low |
| Q3_K_S | 3.64 | 1114.6 GB 1113.5 + 1.1 KV | low |
| IQ3_M | 3.76 | 1151.3 GB 1150.2 + 1.1 KV | low |
| Q3_K_M | 4 | 1224.7 GB 1223.6 + 1.1 KV | low |
| Q3_K_L | 4.3 | 1316.4 GB 1315.3 + 1.1 KV | moderate |
| IQ4_XS | 4.46 | 1365.3 GB 1364.2 + 1.1 KV | moderate |
| Q4_K_S | 4.67 | 1429.5 GB 1428.4 + 1.1 KV | moderate |
| Q4_K_M | 4.89 | 1496.8 GB 1495.7 + 1.1 KV | good |
| Q5_K_S | 5.57 | 1704.7 GB 1703.6 + 1.1 KV | good |
| Q5_K_M | 5.7 | 1744.5 GB 1743.4 + 1.1 KV | good |
| Q6_K | 6.56 | 2007.4 GB 2006.4 + 1.1 KV | excellent |
| Q8_0 | 8.5 | 2600.6 GB 2599.6 + 1.1 KV | lossless |
| FP16 | 16 | 4893.9 GB 4892.8 + 1.1 KV | lossless |
Select your GPU above to see speed estimates and compatibility for each quantization.
Too big for a single GPU — plan a multi-GPU deployment
Even the lightest quant needs ~729 GB. Size GPUs, replicas, TCO and scaling for a production setup. Open in Enterprise →
Ready to run this?Rent by the hour
Rent a GPU and run Qwen3.8 2.4T A95B now
Spin up an A100 / H100 / 4090 in ~60s. Pay by the second. Cancel anytime.
Community Ratings
Loading ratings...
Benchmarks (9)
Run this model
>_
Auto-setup with fitmyllm CLI
Detects your GPU, recommends the best model, downloads it, and starts chatting — zero config. Benchmarks your speed and contributes anonymous data to improve predictions.
Auto-detect GPULive tok/s in chatSpeed benchmarks9 inference engines
Find the best GPU for Qwen3.8 2.4T A95B
Build Hardware for Qwen3.8 2.4T A95B