LLaVA Team/Dense
LLaVA-1.6 Mistral 7B
visionchat
7B
Parameters
4K
Context length
10
Benchmarks
4
Quantizations
245K
HF downloads
Architecture
Dense
Released
2024-01-30
Layers
32
KV Heads
8
Head Dim
128
Family
other
Model Card
View on HuggingFaceLLaVA Model Card
Model details
Model type: LLaVA is an open-source chatbot trained by fine-tuning LLM on multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture. Base LLM: mistralai/Mistral-7B-Instruct-v0.2
Model date: LLaVA-v1.6-Mistral-7B was trained in December 2023.
Paper or resources for more information: https://llava-vl.github.io/
Quantizations & VRAM
Q4_K_M4.5 bpw
5.1 GB
VRAM required
94%
Quality
Q6_K6.5 bpw
6.6 GB
VRAM required
97%
Quality
Q8_08 bpw
8.0 GB
VRAM required
100%
Quality
FP1616 bpw
15.1 GB
VRAM required
100%
Quality
Benchmarks (10)
MMBench67.4
IFEval58.0
MMLU-PRO45.0
MBPP42.1
BBH42.0
MMMU35.3
GPQA28.0
HumanEval23.8
MUSR12.0
MATH8.5
Run with Ollama
$
ollama run llava:7bGPUs that can run this model
At Q4_K_M quantization. Sorted by minimum VRAM.
NVIDIA RTX 3050 6GB
6 GB VRAM • 168 GB/s
NVIDIA
$169
Intel Arc A380
6 GB VRAM • 186 GB/s
INTEL
$129
NVIDIA RTX 2060 6GB
6 GB VRAM • 336 GB/s
NVIDIA
$150
NVIDIA GTX 1660 SUPER
6 GB VRAM • 336 GB/s
NVIDIA
$150
NVIDIA GTX 1660 Ti
6 GB VRAM • 288 GB/s
NVIDIA
$140
NVIDIA GTX 1060 6GB
6 GB VRAM • 192 GB/s
NVIDIA
$80
NVIDIA Tesla C2070
6 GB VRAM • 143 GB/s
NVIDIA
NVIDIA Tesla C2075
6 GB VRAM • 150 GB/s
NVIDIA
NVIDIA Tesla C2090
6 GB VRAM • 177 GB/s
NVIDIA
NVIDIA Tesla M2070
6 GB VRAM • 150 GB/s
NVIDIA
NVIDIA Tesla M2070-Q
6 GB VRAM • 150 GB/s
NVIDIA
NVIDIA Tesla M2075
6 GB VRAM • 150 GB/s
NVIDIA
NVIDIA Tesla M2090
6 GB VRAM • 177 GB/s
NVIDIA
NVIDIA Tesla X2070
6 GB VRAM • 177 GB/s
NVIDIA
NVIDIA Tesla X2090
6 GB VRAM • 177 GB/s
NVIDIA
NVIDIA Tesla K20X
6 GB VRAM • 250 GB/s
NVIDIA
NVIDIA Tesla K20Xm
6 GB VRAM • 250 GB/s
NVIDIA
NVIDIA GeForce GTX 1060 6 GB
6 GB VRAM • 192 GB/s
NVIDIA
NVIDIA GeForce GTX 1060 6 GB 9Gbps
6 GB VRAM • 217 GB/s
NVIDIA
NVIDIA GeForce GTX 1060 6 GB GDDR5X
6 GB VRAM • 192 GB/s
NVIDIA
NVIDIA GeForce GTX 1060 6 GB GP104
6 GB VRAM • 192 GB/s
NVIDIA
NVIDIA GeForce GTX 1060 6 GB Rev. 2
6 GB VRAM • 192 GB/s
NVIDIA
NVIDIA GeForce GTX 1660
6 GB VRAM • 192 GB/s
NVIDIA
NVIDIA GeForce GTX 1660 SUPER
6 GB VRAM • 336 GB/s
NVIDIA
NVIDIA GeForce GTX 1660 Ti
6 GB VRAM • 288 GB/s
NVIDIA
NVIDIA GeForce RTX 2060
6 GB VRAM • 336 GB/s
NVIDIA
$140
NVIDIA GeForce RTX 2060 TU104
6 GB VRAM • 336 GB/s
NVIDIA
$140
AMD Radeon RX 5600 OEM
6 GB VRAM • 288 GB/s
AMD
AMD Radeon RX 5600 XT
6 GB VRAM • 288 GB/s
AMD
$90
AMD Radeon RX 5600M
6 GB VRAM • 288 GB/s
AMD
Find the best GPU for LLaVA-1.6 Mistral 7B
Build Hardware for LLaVA-1.6 Mistral 7B