Spec sheet
OPT 13B — 13B Dense.
Benchmark scores
HumanEval5.5
MMLU-PRO49.1
MATH42.1
IFEval64.0
BBH50.2
GPQA18.9
MUSR21.1
Compatible GPUs
30 @ Q4_K_MNVIDIA RTX 3080 10GB
10 GB · 760 GB/s
Intel Arc B570
10 GB · 456 GB/s
NVIDIA GeForce RTX 3080
10 GB · 760 GB/s
AMD Radeon RX 6700
10 GB · 320 GB/s
AMD Radeon RX 6700M
10 GB · 320 GB/s
AMD Radeon RX 6750 GRE 10 GB
10 GB · 320 GB/s
NVIDIA P102-101
10 GB · 320 GB/s
AMD Xbox Series X GPU
10 GB · 560 GB/s
NVIDIA CMP 170HX 10 GB
10 GB · 1560 GB/s
NVIDIA CMP 50HX
10 GB · 560 GB/s
NVIDIA CMP 90HX
10 GB · 760 GB/s
AMD Xbox Series X 6nm GPU
10 GB · 560 GB/s
NVIDIA RTX 2080 Ti
11 GB · 616 GB/s
NVIDIA GTX 1080 Ti
11 GB · 484 GB/s
NVIDIA GeForce GTX 1080 Ti
11 GB · 484 GB/s