FitMyLLM
Databricks/Dense

DatabricksDolly v2 12B

Largest Dolly. Decent instruction following for its era, outpaced by modern models.

chat
12B
Parameters
2K
Context length
6
Benchmarks
10
Quantizations
60K
HF downloads
Architecture
Dense
Released
2023-04-12
Layers
36
KV Heads
20
Head Dim
80
Family
dolly

Quantization Options

QuantBitsVRAM @ 2KQuality
Q3_K_M4
6.5 GB
low
Q3_K_L4.3
6.9 GB
moderate
IQ4_XS4.46
7.2 GB
moderate
Q4_K_S4.67
7.5 GB
moderate
Q4_K_M4.89
7.8 GB
good
Q5_K_S5.57
8.8 GB
good
Q5_K_M5.7
9.0 GB
good
Q6_K6.56
10.3 GB
excellent
Q8_08.5
13.2 GB
lossless
FP1616
24.5 GB
lossless

Select your GPU above to see speed estimates and compatibility for each quantization.

Deploying for a team or in production? Size GPUs, cost & scaling in Enterprise →
Ready to run this?Rent by the hour

Rent a GPU and run Dolly v2 12B now

Spin up an A100 / H100 / 4090 in ~60s. Pay by the second. Cancel anytime.

Community Ratings

Loading ratings...

Benchmarks (6)

Run this model

Easiest way to get started·Beginners
Docs ↗
curl -fsSL https://ollama.com/install.sh | sh
$ollama run hf.co/TheBloke/dolly-v2-12b-GGUF:Q4_K_M

Not in Ollama's library: Ollama pulls the GGUF from Hugging Face (TheBloke/dolly-v2-12b-GGUF). If Q4_K_M is not in that repo, pick a file listed there.

Setup guide
>_

Auto-setup with fitmyllm CLI

Detects your GPU, recommends the best model, downloads it, and starts chatting — zero config. Benchmarks your speed and contributes anonymous data to improve predictions.

pip install fitmyllmthen run fitmyllmLearn more
Auto-detect GPULive tok/s in chatSpeed benchmarks9 inference engines

GPUs that can run this model

At Q4_K_M quantization. Sorted by minimum VRAM.

Apple M1 (8GB)
8 GB VRAM • 68 GB/s
APPLE
$499
Apple M2 (8GB)
8 GB VRAM • 100 GB/s
APPLE
$599
Apple M3 (8GB)
8 GB VRAM • 100 GB/s
APPLE
$599
NVIDIA Tesla K8
8 GB VRAM • 160 GB/s
NVIDIA
NVIDIA Tesla M60
8 GB VRAM • 160 GB/s
NVIDIA

Find the best GPU for Dolly v2 12B

Build Hardware for Dolly v2 12B

Largest Dolly. Decent instruction following for its era, outpaced by modern models.

Spec sheet

Dolly v2 12B — 12B Dense.

Specifications
Parameters
12B
Architecture
Dense Transformer
Context Length
2K tokens
Capabilities
chat
Release Date
2023-04-12
Provider
Databricks
Family
dolly
VRAM requirements
QuantBPWVRAMQuality
Q3_K_M46.5 GB88%
Q3_K_L4.36.9 GB90%
IQ4_XS4.467.2 GB92%
Q4_K_S4.677.5 GB93%
Q4_K_M4.897.8 GB94%
Q5_K_S5.578.8 GB96%
Q5_K_M5.79.0 GB96%
Q6_K6.5610.3 GB97%
Q8_08.513.2 GB100%
FP161624.5 GB100%
Benchmark scores
MMLU-PRO1.4
MATH1.4
IFEval23.6
BBH6.4
GPQA0.0
MUSR5.5
Compatible GPUs
30 @ Q4_K_M