DeepSeek-Coder-V2-Lite 15.7B — 15.7B Parameter Mixture of Experts LLM
Model Specifications
- Parameters
- 15.7B (2.4B active)
- Architecture
- Mixture of Experts
- Context Length
- 160K tokens
- Capabilities
- chat, coding
- Release Date
- 2024-12-13
- Provider
- DeepSeek
- Family
- deepseek
VRAM Requirements
| Quantization | BPW | VRAM | Quality |
|---|---|---|---|
| Q3_K_M | 4 | 8.3 GB | 88% |
| Q3_K_L | 4.3 | 8.9 GB | 90% |
| IQ4_XS | 4.46 | 9.2 GB | 92% |
| Q4_K_S | 4.67 | 9.7 GB | 93% |
| Q4_K_M | 4.89 | 10.1 GB | 94% |
| Q5_K_S | 5.57 | 11.4 GB | 96% |
| Q5_K_M | 5.7 | 11.7 GB | 96% |
| Q6_K | 6.56 | 13.4 GB | 97% |
| Q8_0 | 8.5 | 17.2 GB | 100% |
| FP16 | 16 | 31.9 GB | 100% |
Benchmark Scores
MMLU-PRO40.7
MATH57.0
IFEval43.8
BBH40.7
MMMU49.0
GPQA18.3
MUSR28.7
BigCodeBench36.8
MMBench73.0
How to Run DeepSeek-Coder-V2-Lite 15.7B
Run DeepSeek-Coder-V2-Lite 15.7B locally with Ollama (needs 10.1 GB VRAM at Q4_K_M):
ollama run deepseek:15bCompatible GPUs (30)
GPUs that can run DeepSeek-Coder-V2-Lite 15.7B at Q4_K_M quantization:
NVIDIA RTX 2080 Ti(11GB, 616 GB/s)NVIDIA GTX 1080 Ti(11GB, 484 GB/s)NVIDIA GeForce GTX 1080 Ti(11GB, 484 GB/s)NVIDIA GeForce RTX 2080 Ti(11GB, 616 GB/s)NVIDIA RTX 5070(12GB, 672 GB/s)NVIDIA RTX 4070 Ti(12GB, 504 GB/s)NVIDIA RTX 4070 SUPER(12GB, 504 GB/s)NVIDIA RTX 4070(12GB, 504 GB/s)NVIDIA RTX 3080 Ti(12GB, 912 GB/s)NVIDIA RTX 3080 12GB(12GB, 912 GB/s)NVIDIA RTX 3060 12GB(12GB, 360 GB/s)AMD RX 7700 XT(12GB, 432 GB/s)AMD RX 6700 XT(12GB, 384 GB/s)AMD RX 6750 XT(12GB, 432 GB/s)Intel Arc B580(12GB, 456 GB/s)