LLaVA Team/Dense

LLaVA-1.6 Yi 34B

visionchat
34B
Parameters
4K
Context length
8
Benchmarks
4
Quantizations
120K
HF downloads
Architecture
Dense
Released
2024-01-30
Layers
60
KV Heads
8
Head Dim
128
Family
other

LLaVA Model Card

Model details

Model type: LLaVA is an open-source chatbot trained by fine-tuning LLM on multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture. Base LLM: NousResearch/Nous-Hermes-2-Yi-34B

Model date: LLaVA-v1.6-34B was trained in December 2023.

Paper or resources for more information: https://llava-vl.github.io/

Quantizations & VRAM

Q4_K_M4.5 bpw
20.5 GB
VRAM required
94%
Quality
Q6_K6.5 bpw
27.3 GB
VRAM required
97%
Quality
Q8_08 bpw
35.2 GB
VRAM required
100%
Quality
FP1616 bpw
68.4 GB
VRAM required
100%
Quality

Benchmarks (8)

MMBench79.3
IFEval68.0
MMLU-PRO58.0
BBH58.0
MMMU46.7
GPQA35.0
MUSR18.0
MATH16.5

Run with Ollama

$ollama run llava:34b

GPUs that can run this model

At Q4_K_M quantization. Sorted by minimum VRAM.

Find the best GPU for LLaVA-1.6 Yi 34B

Build Hardware for LLaVA-1.6 Yi 34B