c4ai-command-r-v01 35B β 35B Dense.
Run c4ai-command-r-v01 35B locally with Ollama β needs 21.9 GB VRAM at Q4_K_M:
ollama run command-r:35b| Quant | Bits | VRAM @ 16K | Quality |
|---|---|---|---|
| IQ3_XXS | 3.25 | 16.6 GB 14.7 + 1.9 KV | low |
| IQ3_XS | 3.5 | 17.7 GB 15.8 + 1.9 KV | low |
| Q3_K_S | 3.64 | 18.3 GB 16.4 + 1.9 KV | low |
| IQ3_M | 3.76 | 18.8 GB 16.9 + 1.9 KV | low |
| Q3_K_M | 4 | 19.9 GB 18.0 + 1.9 KV | low |
| Q3_K_L | 4.3 | 21.2 GB 19.3 + 1.9 KV | moderate |
| IQ4_XS | 4.46 | 21.9 GB 20.0 + 1.9 KV | moderate |
| Q4_K_S | 4.67 | 22.8 GB 20.9 + 1.9 KV | moderate |
| Q4_K_M | 4.89 | 23.8 GB 21.9 + 1.9 KV | good |
| Q5_K_S | 5.57 | 26.7 GB 24.9 + 1.9 KV | good |
| Q5_K_M | 5.7 | 27.3 GB 25.4 + 1.9 KV | good |
| Q6_K | 6.56 | 31.1 GB 29.2 + 1.9 KV | excellent |
| Q8_0 | 8.5 | 39.6 GB 37.7 + 1.9 KV | lossless |
| FP16 | 16 | 72.4 GB 70.5 + 1.9 KV | lossless |
Select your GPU above to see speed estimates and compatibility for each quantization.
Rent a GPU and run c4ai-command-r-v01 35B now
Spin up an A100 / H100 / 4090 in ~60s. Pay by the second. Cancel anytime.
Detects your GPU, recommends the best model, downloads it, and starts chatting β zero config. Benchmarks your speed and contributes anonymous data to improve predictions.
At Q4_K_M quantization. Sorted by minimum VRAM.
Find the best GPU for c4ai-command-r-v01 35B
Build Hardware for c4ai-command-r-v01 35Bπ¨ This model is non-quantized version of Cohere Labs Command-R. You can find the quantized version of Cohere Labs Command-R using bitsandbytes here.
Run c4ai-command-r-v01 35B locally with Ollama β needs 21.9 GB VRAM at Q4_K_M:
ollama run command-r:35b