Shanghai AI Lab/Dense

InternLM2 5B

chat

4.5B

Parameters

32K

Context length

1

Benchmarks

4

Quantizations

40K

HF downloads

Architecture

Dense

Released

2024-01-17

Layers

32

KV Heads

8

Head Dim

128

Family

internlm

Quantizations & VRAM

Q4_K_M4.5 bpw

3.0 GB

VRAM required

94%

Quality

Q6_K6.5 bpw

4.1 GB

VRAM required

97%

Quality

Q8_08 bpw

5.0 GB

VRAM required

100%

Quality

FP1616 bpw

9.5 GB

VRAM required

100%

Quality

Benchmarks (1)

HumanEval47.6

HuggingFace GGUF Downloads Build Hardware

GPUs that can run this model

At Q4_K_M quantization. Sorted by minimum VRAM.

NVIDIA Tesla C1080

4 GB VRAM • 102 GB/s

NVIDIA Tesla K10

4 GB VRAM • 160 GB/s

NVIDIA Tesla M4

4 GB VRAM • 88 GB/s

NVIDIA GeForce GTX 1050 Ti

4 GB VRAM • 112 GB/s

AMD Radeon Instinct MI8

4 GB VRAM • 512 GB/s

NVIDIA GeForce GTX 1650

4 GB VRAM • 128 GB/s

NVIDIA GeForce GTX 1650 SUPER

4 GB VRAM • 192 GB/s

AMD Radeon RX 5300 XT OEM

4 GB VRAM • 112 GB/s

AMD Radeon RX 5500 OEM

4 GB VRAM • 224 GB/s

AMD Radeon RX 5500 XT

4 GB VRAM • 224 GB/s

AMD Radeon RX 5500M

4 GB VRAM • 224 GB/s

NVIDIA GeForce GTX 1650 GDDR6

4 GB VRAM • 192 GB/s

NVIDIA GeForce GTX 1650 TU106

4 GB VRAM • 192 GB/s

NVIDIA GeForce GTX 1650 TU116

4 GB VRAM • 192 GB/s

NVIDIA GeForce GTX 1630

4 GB VRAM • 96 GB/s

NVIDIA GeForce RTX 3050 4 GB

4 GB VRAM • 192 GB/s

NVIDIA RTX A1000 Embedded

4 GB VRAM • 224 GB/s

NVIDIA RTX A2000 Embedded

4 GB VRAM • 192 GB/s

AMD Radeon RX 6400

4 GB VRAM • 128 GB/s

AMD Radeon RX 6500 XT

4 GB VRAM • 144 GB/s

AMD Radeon RX 6500M

4 GB VRAM • 144 GB/s

AMD Radeon RX 6600S

4 GB VRAM • 224 GB/s

4 GB VRAM • 124 GB/s

4 GB VRAM • 124 GB/s

Intel Arc A350M

4 GB VRAM • 112 GB/s

Intel Arc A370M

4 GB VRAM • 112 GB/s

AMD Radeon RX 6450M

4 GB VRAM • 128 GB/s

AMD Radeon RX 6550M

4 GB VRAM • 144 GB/s

AMD Radeon RX 6550S

4 GB VRAM • 128 GB/s

NVIDIA GeForce RTX 4010

4 GB VRAM • 96 GB/s

Find the best GPU for InternLM2 5B

Build Hardware for InternLM2 5B