bloom 176.2B — 176.2B Parameter Dense LLM
Model Specifications
- Parameters
- 176.2B
- Architecture
- Dense Transformer
- Context Length
- 4K tokens
- Capabilities
- chat
- Release Date
- 2022-07-06
- Provider
- BigScience
- Family
- bloom
VRAM Requirements
| Quantization | BPW | VRAM | Quality |
|---|---|---|---|
| IQ2_XXS | 2.38 | 52.9 GB | 65% |
| IQ2_M | 2.93 | 65.0 GB | 75% |
| Q2_K | 3.16 | 70.1 GB | 78% |
| IQ3_XXS | 3.25 | 72.1 GB | 82% |
| IQ3_XS | 3.5 | 77.6 GB | 84% |
| Q3_K_S | 3.64 | 80.7 GB | 85% |
| IQ3_M | 3.76 | 83.3 GB | 86% |
| Q3_K_M | 4 | 88.6 GB | 88% |
| Q3_K_L | 4.3 | 95.2 GB | 90% |
| IQ4_XS | 4.46 | 98.7 GB | 92% |
| Q4_K_S | 4.67 | 103.3 GB | 93% |
| Q4_K_M | 4.89 | 108.2 GB | 94% |
| Q5_K_S | 5.57 | 123.2 GB | 96% |
| Q5_K_M | 5.7 | 126.0 GB | 96% |
| Q6_K | 6.56 | 145.0 GB | 97% |
| Q8_0 | 8.5 | 187.7 GB | 100% |
| FP16 | 16 | 352.9 GB | 100% |
Benchmark Scores
HumanEval15.0
How to Run bloom 176.2B
Run bloom 176.2B locally with Ollama (needs 108.2 GB VRAM at Q4_K_M):
ollama run bloom:176bCompatible GPUs (30)
GPUs that can run bloom 176.2B at Q4_K_M quantization:
AMD Instinct MI300A(120GB, 5300 GB/s)Apple M4 Max (128GB)(128GB, 546 GB/s)AMD Instinct MI250X(128GB, 3277 GB/s)Apple M1 Ultra (128GB)(128GB, 800 GB/s)Apple M2 Ultra (128GB)(128GB, 800 GB/s)AMD Radeon Instinct MI250(128GB, 3280 GB/s)AMD Radeon Instinct MI250X(128GB, 3280 GB/s)AMD Radeon Instinct MI300(128GB, 6550 GB/s)Intel Data Center GPU Max 1550(128GB, 3280 GB/s)Intel Data Center GPU Max Subsystem(128GB, 3210 GB/s)NVIDIA GB10(128GB, 273 GB/s)NVIDIA Jetson T5000(128GB, 273 GB/s)Apple M5 Max (128GB)(128GB, 614 GB/s)NVIDIA H200 SXM 141GB(140GB, 4800 GB/s)NVIDIA H200 NVL(141GB, 4890 GB/s)