granite/Dense

Granite 3.3 8B

Name: Granite 3.3 8B
Author: granite

Granite-3.3-8B-Instruct is a 8-billion parameter 128K context length language model fine-tuned for improved reasoning and instruction-following capabilities.

chatreasoningcodingtool_use

Parameters

125K

Context length

Benchmarks

Quantizations

Architecture

Dense

Released

2025-04-16

Layers

KV Heads

Head Dim

128

Family

granite

Quantization Options

Context length:

Quant	Bits	VRAM @ 16K	Quality
Q4_K_M	4.89	6.9 GB 5.4 + 1.5 KV	good
Q5_K_S	5.57	7.6 GB 6.1 + 1.5 KV	good
Q5_K_M	5.7	7.7 GB 6.2 + 1.5 KV	good
Q6_K	6.56	8.5 GB 7.0 + 1.5 KV	excellent
Q8_0	8.5	10.5 GB 9.0 + 1.5 KV	lossless
FP16	16	18.0 GB 16.5 + 1.5 KV	lossless

Select your GPU above to see speed estimates and compatibility for each quantization.

Deploying for a team or in production? Size GPUs, cost & scaling in Enterprise →

▸ READY TO RUN THIS?RENT BY THE HOUR

RENT A GPU AND RUN GRANITE 3.3 8B NOW

Rent on Vast.ai →Or RunPod →

Spin up an A100 / H100 / 4090 in ~60s. Pay by the second. Cancel anytime.

Community Ratings

Loading ratings...

Benchmarks (19)

HumanEval89.7

IFEval74.8

MATH-50069.0

BBH34.7

GPQA Diamond33.8

MMLU-PRO27.9

MATH23.8

IFBench22.4

MUSR16.8

LiveCodeBench12.7

τ²-Bench10.5

SciCode10.1

GPQA8.7

AA Intelligence7.0

AIME6.7

AA Math6.7

AA Long Context4.3

HLE4.2

AA Coding3.4

Run this model

▸Easiest way to get started·Beginners

DOCS ↗

curl -fsSL https://ollama.com/install.sh | sh

$ollama run granite3.3:8b

Tag may need adjustment — check ollama.com/library/granite3.3 for available tags.

▸ SETUP GUIDE

Auto-setup with fitmyllm CLI

Detects your GPU, recommends the best model, downloads it, and starts chatting — zero config. Benchmarks your speed and contributes anonymous data to improve predictions.

pip install fitmyllmthen run fitmyllmLearn more

Auto-detect GPULive tok/s in chatSpeed benchmarks9 inference engines

HuggingFace Ollama Library GGUF Downloads Build Hardware

GPUs that can run this model

At Q4_K_M quantization. Sorted by minimum VRAM.

NVIDIA RTX 3050 6GB

6 GB VRAM • 168 GB/s

NVIDIA

$169

QUANT	BPW	VRAM	QUALITY
Q4_K_M	4.89	5.4 GB	94%
Q5_K_S	5.57	6.1 GB	96%
Q5_K_M	5.7	6.2 GB	96%
Q6_K	6.56	7.0 GB	97%
Q8_0	8.5	9.0 GB	100%
FP16	16	16.5 GB	100%