Mistral-Large-Instruct-2407 is an advanced dense Large Language Model (LLM) of 123B parameters with state-of-the-art reasoning, knowledge and coding capabilities.
chattool_useThinkingTool Use
123B
Parameters
128K
Context length
11
Benchmarks
17
Quantizations
0
Architecture
Dense
Released
2024-07-24
Layers
88
KV Heads
8
Head Dim
128
Family
mistral
Quantization Options
Context length:
Quant
Bits
VRAM @ 16K
Quality
IQ2_XXS
2.38
41.2 GB
37.1 + 4.1 KV
low
IQ2_M
2.93
49.7 GB
45.5 + 4.1 KV
low
Q2_K
3.16
53.2 GB
49.1 + 4.1 KV
low
IQ3_XXS
3.25
54.6 GB
50.5 + 4.1 KV
low
IQ3_XS
3.5
58.4 GB
54.3 + 4.1 KV
low
Q3_K_S
3.64
60.6 GB
56.5 + 4.1 KV
low
IQ3_M
3.76
62.4 GB
58.3 + 4.1 KV
low
Q3_K_M
4
66.1 GB
62.0 + 4.1 KV
low
Q3_K_L
4.3
70.7 GB
66.6 + 4.1 KV
moderate
IQ4_XS
4.46
73.2 GB
69.1 + 4.1 KV
moderate
Q4_K_S
4.67
76.4 GB
72.3 + 4.1 KV
moderate
Q4_K_M
4.89
79.8 GB
75.7 + 4.1 KV
good
Q5_K_S
5.57
90.3 GB
86.1 + 4.1 KV
good
Q5_K_M
5.7
92.3 GB
88.1 + 4.1 KV
good
Q6_K
6.56
105.5 GB
101.3 + 4.1 KV
excellent
Q8_0
8.5
135.3 GB
131.2 + 4.1 KV
lossless
FP16
16
250.6 GB
246.5 + 4.1 KV
lossless
Select your GPU above to see speed estimates and compatibility for each quantization.
Detects your GPU, recommends the best model, downloads it, and starts chatting — zero config. Benchmarks your speed and contributes anonymous data to improve predictions.
Mistral-Large-Instruct-2407 is an advanced dense Large Language Model (LLM) of 123B parameters with state-of-the-art reasoning, knowledge and coding capabilities.
▸ SPEC SHEET
Mistral-Large 123B — 123B Dense.
▸ SPECIFICATIONS
PARAMETERS
123B
ARCHITECTURE
Dense Transformer
CONTEXT LENGTH
128K tokens
CAPABILITIES
chat, tool_use
RELEASE DATE
2024-07-24
PROVIDER
Mistral AI
FAMILY
mistral
▸ VRAM REQUIREMENTS
QUANT
BPW
VRAM
QUALITY
IQ2_XXS
2.38
37.1 GB
65%
IQ2_M
2.93
45.5 GB
75%
Q2_K
3.16
49.1 GB
78%
IQ3_XXS
3.25
50.5 GB
82%
IQ3_XS
3.5
54.3 GB
84%
Q3_K_S
3.64
56.5 GB
85%
IQ3_M
3.76
58.3 GB
86%
Q3_K_M
4
62.0 GB
88%
Q3_K_L
4.3
66.6 GB
90%
IQ4_XS
4.46
69.1 GB
92%
Q4_K_S
4.67
72.3 GB
93%
Q4_K_M
4.89
75.7 GB
94%
Q5_K_S
5.57
86.1 GB
96%
Q5_K_M
5.7
88.1 GB
96%
Q6_K
6.56
101.3 GB
97%
Q8_0
8.5
131.2 GB
100%
FP16
16
246.5 GB
100%
§ 01BENCHMARK SCORES
MMLU-PRO50.7
MATH49.5
IFEval84.0
BBH52.7
GPQA24.9
MUSR17.2
Arena Elo1267.0
LiveCodeBench17.8
AIME0.0
GPQA Diamond35.1
HLE3.4
§ 02RUN COMMAND
Run Mistral-Large 123B locally with Ollama — needs 75.7 GB VRAM at Q4_K_M:
