Meta/Dense

LLaMA 1 65B

chat
65.2B
Parameters
2K
Context length
7
Benchmarks
4
Quantizations
80K
HF downloads
Architecture
Dense
Released
2023-02-24
Layers
80
KV Heads
64
Head Dim
128
Family
llama

This contains the weights for the LLaMA-65b model. This model is under a non-commercial license (see the LICENSE file). You should only use this repository if you have been granted access to the model by filling out this form but either lost your copy of the weights or got some trouble converting them to the Transformers format.

Quantizations & VRAM

Q4_K_M4.5 bpw
38.1 GB
VRAM required
94%
Quality
Q6_K6.5 bpw
54.4 GB
VRAM required
97%
Quality
Q8_08 bpw
66.7 GB
VRAM required
100%
Quality
FP1616 bpw
131.9 GB
VRAM required
100%
Quality

Benchmarks (7)

IFEval81.2
BBH54.1
MMLU-PRO48.1
MATH44.1
GPQA24.6
HumanEval23.7
MUSR22.3

GPUs that can run this model

At Q4_K_M quantization. Sorted by minimum VRAM.

NVIDIA A100 PCIe 40GB
40 GB VRAM • 1555 GB/s
NVIDIA
$10000
NVIDIA A100 PCIe 40 GB
40 GB VRAM • 1560 GB/s
NVIDIA
NVIDIA A100 SXM4 40 GB
40 GB VRAM • 1560 GB/s
NVIDIA
NVIDIA A800 PCIe 40 GB
40 GB VRAM • 1560 GB/s
NVIDIA
Apple M3 Max (48GB)
48 GB VRAM • 400 GB/s
APPLE
$2899
Apple M4 Pro (48GB)
48 GB VRAM • 273 GB/s
APPLE
$1799
Apple M4 Max (48GB)
48 GB VRAM • 546 GB/s
APPLE
$2499
NVIDIA L40S 48GB
48 GB VRAM • 864 GB/s
NVIDIA
$7500
NVIDIA L40 48GB
48 GB VRAM • 864 GB/s
NVIDIA
$5500
NVIDIA RTX 6000 Ada 48GB
48 GB VRAM • 960 GB/s
NVIDIA
$6800
NVIDIA A40 48GB
48 GB VRAM • 696 GB/s
NVIDIA
$4650
NVIDIA RTX A6000 48GB
48 GB VRAM • 768 GB/s
NVIDIA
$4650
NVIDIA Quadro RTX 8000
48 GB VRAM • 672 GB/s
NVIDIA
NVIDIA Quadro RTX 8000 Passive
48 GB VRAM • 624 GB/s
NVIDIA
NVIDIA A40 PCIe
48 GB VRAM • 696 GB/s
NVIDIA
NVIDIA RTX 6000 Ada Generation
48 GB VRAM • 960 GB/s
NVIDIA
NVIDIA L20
48 GB VRAM • 864 GB/s
NVIDIA
AMD Radeon PRO W7800 48 GB
48 GB VRAM • 864 GB/s
AMD
AMD Radeon PRO W7900
48 GB VRAM • 864 GB/s
AMD
Intel Data Center GPU Max 1100
48 GB VRAM • 1230 GB/s
INTEL
NVIDIA RTX 5880 Ada Generation
48 GB VRAM • 864 GB/s
NVIDIA
NVIDIA RTX PRO 5000 Blackwell
48 GB VRAM • 1340 GB/s
NVIDIA
AMD Radeon PRO W7900D
48 GB VRAM • 864 GB/s
AMD
Apple M1 Ultra (64GB)
64 GB VRAM • 800 GB/s
APPLE
$2499
Apple M2 Ultra (64GB)
64 GB VRAM • 800 GB/s
APPLE
$2999
Apple M4 Max (64GB)
64 GB VRAM • 546 GB/s
APPLE
$2899
Apple M2 Max (64GB)
64 GB VRAM • 400 GB/s
APPLE
$2299
Apple M3 Max (64GB)
64 GB VRAM • 300 GB/s
APPLE
$2799
Apple M4 Pro (64GB)
64 GB VRAM • 273 GB/s
APPLE
$2599
AMD Radeon Instinct MI200
64 GB VRAM • 1640 GB/s
AMD

Find the best GPU for LLaMA 1 65B

Build Hardware for LLaMA 1 65B