Ling-lite 16.8B — 16.8B Parameter Mixture of Experts LLM
Model Specifications
- Parameters
- 16.8B (2.4B active)
- Architecture
- Mixture of Experts
- Context Length
- 32K tokens
- Capabilities
- chat
- Release Date
- 2025-02-28
- Provider
- Ant Group
- Family
- other
VRAM Requirements
| Quantization | BPW | VRAM | Quality |
|---|---|---|---|
| Q3_K_M | 4 | 8.9 GB | 88% |
| Q3_K_L | 4.3 | 9.5 GB | 90% |
| IQ4_XS | 4.46 | 9.9 GB | 92% |
| Q4_K_S | 4.67 | 10.3 GB | 93% |
| Q4_K_M | 4.89 | 10.8 GB | 94% |
| Q5_K_S | 5.57 | 12.2 GB | 96% |
| Q5_K_M | 5.7 | 12.5 GB | 96% |
| Q6_K | 6.56 | 14.3 GB | 97% |
| Q8_0 | 8.5 | 18.3 GB | 100% |
| FP16 | 16 | 34.1 GB | 100% |
How to Run Ling-lite 16.8B
Run Ling-lite 16.8B locally with Ollama (needs 10.8 GB VRAM at Q4_K_M):
ollama run other:16bCompatible GPUs (30)
GPUs that can run Ling-lite 16.8B at Q4_K_M quantization:
NVIDIA RTX 2080 Ti(11GB, 616 GB/s)NVIDIA GTX 1080 Ti(11GB, 484 GB/s)NVIDIA GeForce GTX 1080 Ti(11GB, 484 GB/s)NVIDIA GeForce RTX 2080 Ti(11GB, 616 GB/s)NVIDIA RTX 5070(12GB, 672 GB/s)NVIDIA RTX 4070 Ti(12GB, 504 GB/s)NVIDIA RTX 4070 SUPER(12GB, 504 GB/s)NVIDIA RTX 4070(12GB, 504 GB/s)NVIDIA RTX 3080 Ti(12GB, 912 GB/s)NVIDIA RTX 3080 12GB(12GB, 912 GB/s)NVIDIA RTX 3060 12GB(12GB, 360 GB/s)AMD RX 7700 XT(12GB, 432 GB/s)AMD RX 6700 XT(12GB, 384 GB/s)AMD RX 6750 XT(12GB, 432 GB/s)Intel Arc B580(12GB, 456 GB/s)