▸ SPEC SHEET
OPT 30B — 30B Dense.
▸ SPECIFICATIONS
- PARAMETERS
- 30B
- ARCHITECTURE
- Dense Transformer
- CONTEXT LENGTH
- 2K tokens
- CAPABILITIES
- chat
- RELEASE DATE
- 2022-05-03
- PROVIDER
- Meta
- FAMILY
- opt
▸ VRAM REQUIREMENTS
| QUANT | BPW | VRAM | QUALITY |
|---|---|---|---|
| IQ3_XXS | 3.25 | 12.7 GB | 82% |
| IQ3_XS | 3.5 | 13.6 GB | 84% |
| Q3_K_S | 3.64 | 14.1 GB | 85% |
| IQ3_M | 3.76 | 14.6 GB | 86% |
| Q3_K_M | 4 | 15.5 GB | 88% |
| Q3_K_L | 4.3 | 16.6 GB | 90% |
| IQ4_XS | 4.46 | 17.2 GB | 92% |
| Q4_K_S | 4.67 | 18.0 GB | 93% |
| Q4_K_M | 4.89 | 18.8 GB | 94% |
| Q5_K_S | 5.57 | 21.4 GB | 96% |
| Q5_K_M | 5.7 | 21.9 GB | 96% |
| Q6_K | 6.56 | 25.1 GB | 97% |
| Q8_0 | 8.5 | 32.4 GB | 100% |
| FP16 | 16 | 60.5 GB | 100% |
§ 01BENCHMARK SCORES
MMLU-PRO1.8
MATH1.1
IFEval24.5
BBH3.5
GPQA2.6
MUSR4.2
§ 03COMPATIBLE GPUs
30 @ Q4_K_MAMD RX 7900 XT
20 GB · 800 GB/s
NVIDIA RTX 4000 Ada 20GB
20 GB · 432 GB/s
NVIDIA A10M
20 GB · 500 GB/s
NVIDIA GeForce RTX 3080 Ti 20 GB
20 GB · 760 GB/s
AMD Radeon RX 7900 XT
20 GB · 800 GB/s
NVIDIA RTX 4000 Ada Generation
20 GB · 360 GB/s
NVIDIA RTX 4000 SFF Ada Generation
20 GB · 280 GB/s
NVIDIA RTX A4500
20 GB · 640 GB/s
NVIDIA RTX 4090
24 GB · 1008 GB/s
NVIDIA RTX 3090 Ti
24 GB · 1008 GB/s
NVIDIA RTX 3090
24 GB · 936 GB/s
AMD RX 7900 XTX
24 GB · 960 GB/s
Apple M4 Pro (24GB)
24 GB · 273 GB/s
NVIDIA L4 24GB
24 GB · 300 GB/s
NVIDIA A10 24GB
24 GB · 600 GB/s