Meta/Dense

LLaMA 1 7B

chat
7B
Parameters
2K
Context length
8
Benchmarks
4
Quantizations
300K
HF downloads
Architecture
Dense
Released
2023-02-24
Layers
32
KV Heads
32
Head Dim
128
Family
llama

This contains the weights for the LLaMA-7b model. This model is under a non-commercial license (see the LICENSE file). You should only use this repository if you have been granted access to the model by filling out this form but either lost your copy of the weights or got some trouble converting them to the Transformers format.

Quantizations & VRAM

Q4_K_M4.5 bpw
4.4 GB
VRAM required
94%
Quality
Q6_K6.5 bpw
6.2 GB
VRAM required
97%
Quality
Q8_08 bpw
7.5 GB
VRAM required
100%
Quality
FP1616 bpw
14.5 GB
VRAM required
100%
Quality

Benchmarks (8)

IFEval81.3
MBPP46.8
MMLU-PRO33.0
BBH32.0
MATH22.0
MUSR12.5
HumanEval10.5
GPQA8.1

GPUs that can run this model

At Q4_K_M quantization. Sorted by minimum VRAM.

NVIDIA Tesla K20c
5 GB VRAM • 208 GB/s
NVIDIA
NVIDIA Tesla K20m
5 GB VRAM • 208 GB/s
NVIDIA
NVIDIA Tesla K20s
5 GB VRAM • 208 GB/s
NVIDIA
NVIDIA GeForce GTX 1060 5 GB
5 GB VRAM • 160 GB/s
NVIDIA
NVIDIA P102-100
5 GB VRAM • 440 GB/s
NVIDIA
NVIDIA RTX 3050 6GB
6 GB VRAM • 168 GB/s
NVIDIA
$169
Intel Arc A380
6 GB VRAM • 186 GB/s
INTEL
$129
NVIDIA RTX 2060 6GB
6 GB VRAM • 336 GB/s
NVIDIA
$150
NVIDIA GTX 1660 SUPER
6 GB VRAM • 336 GB/s
NVIDIA
$150
NVIDIA GTX 1660 Ti
6 GB VRAM • 288 GB/s
NVIDIA
$140
NVIDIA GTX 1060 6GB
6 GB VRAM • 192 GB/s
NVIDIA
$80
NVIDIA Tesla C2070
6 GB VRAM • 143 GB/s
NVIDIA
NVIDIA Tesla C2075
6 GB VRAM • 150 GB/s
NVIDIA
NVIDIA Tesla C2090
6 GB VRAM • 177 GB/s
NVIDIA
NVIDIA Tesla M2070
6 GB VRAM • 150 GB/s
NVIDIA
NVIDIA Tesla M2070-Q
6 GB VRAM • 150 GB/s
NVIDIA
NVIDIA Tesla M2075
6 GB VRAM • 150 GB/s
NVIDIA
NVIDIA Tesla M2090
6 GB VRAM • 177 GB/s
NVIDIA
NVIDIA Tesla X2070
6 GB VRAM • 177 GB/s
NVIDIA
NVIDIA Tesla X2090
6 GB VRAM • 177 GB/s
NVIDIA
NVIDIA Tesla K20X
6 GB VRAM • 250 GB/s
NVIDIA
NVIDIA Tesla K20Xm
6 GB VRAM • 250 GB/s
NVIDIA
NVIDIA GeForce GTX 1060 6 GB
6 GB VRAM • 192 GB/s
NVIDIA
NVIDIA GeForce GTX 1060 6 GB 9Gbps
6 GB VRAM • 217 GB/s
NVIDIA
NVIDIA GeForce GTX 1060 6 GB GDDR5X
6 GB VRAM • 192 GB/s
NVIDIA
NVIDIA GeForce GTX 1060 6 GB GP104
6 GB VRAM • 192 GB/s
NVIDIA
NVIDIA GeForce GTX 1060 6 GB Rev. 2
6 GB VRAM • 192 GB/s
NVIDIA
NVIDIA GeForce GTX 1660
6 GB VRAM • 192 GB/s
NVIDIA
NVIDIA GeForce GTX 1660 SUPER
6 GB VRAM • 336 GB/s
NVIDIA
NVIDIA GeForce GTX 1660 Ti
6 GB VRAM • 288 GB/s
NVIDIA

Find the best GPU for LLaMA 1 7B

Build Hardware for LLaMA 1 7B