Meta/Dense

Llama-3.1-8B

Name: Llama-3.1-8B
Author: Meta

The Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction tuned generative models in 8B, 70B and 405B sizes (text in/text out).

chat

Parameters

Context length

Benchmarks

Quantizations

1.3M

HF downloads

Architecture

Dense

Released

2024-07-23

Layers

KV Heads

Head Dim

128

Family

llama

Quantization Options

Quant	Bits	VRAM	Quality
Q4_K_M	4.89	5.4 GB	good
Q5_K_S	5.57	6.1 GB	good
Q5_K_M	5.7	6.2 GB	good
Q6_K	6.56	7.0 GB	excellent
Q8_0	8.5	9.0 GB	lossless
FP16	16	16.5 GB	lossless

Select your GPU above to see speed estimates and compatibility for each quantization.

▸ READY TO RUN THIS?RENT BY THE HOUR

RENT A GPU AND RUN LLAMA-3.1-8B NOW

Rent on RunPod →Or Vast.ai →

Spin up an A100 / H100 / 4090 in ~60s. Pay by the second. Cancel anytime.

Community Ratings

Loading ratings...

Benchmarks (17)

Arena Elo1191

HumanEval62.8

MBPP55.6

IFEval49.2

BigCodeBench32.8

MMLU-PRO31.1

BBH29.4

GPQA Diamond27.0

MATH-50021.8

MATH15.6

SciCode9.1

GPQA8.7

MUSR8.6

LiveCodeBench8.5

AA Intelligence7.6

HLE4.3

AIME4.3

Run this model

▸Easiest way to get started·Beginners

DOCS ↗

curl -fsSL https://ollama.com/install.sh | sh

$ollama run llama3.1:8.0b-instruct-q4_K_M

Downloads and runs automatically. Add --verbose for speed stats.

▸ SETUP GUIDE

Auto-setup with fitmyllm CLI

Detects your GPU, recommends the best model, downloads it, and starts chatting — zero config. Benchmarks your speed and contributes anonymous data to improve predictions.

pip install fitmyllmthen run fitmyllmLearn more

Auto-detect GPULive tok/s in chatSpeed benchmarks9 inference engines

HuggingFace Ollama Library GGUF Downloads Build Hardware

GPUs that can run this model

At Q4_K_M quantization. Sorted by minimum VRAM.

NVIDIA RTX 3050 6GB

6 GB VRAM • 168 GB/s

NVIDIA

$169