NousResearch/Mixture of Experts

Nous-Hermes-2-Mixtral-8x7B-DPO

Name: Nous-Hermes-2-Mixtral-8x7B-DPO
Author: NousResearch

Nous Hermes 2 Mixtral — community fine-tune of Mixtral with improved helpfulness.

chat

46.7B

Parameters (13B active)

32K

Context length

Benchmarks

Quantizations

HF downloads

Architecture

MoE

Released

2024-01-11

Layers

KV Heads

Head Dim

128

Family

mistral

Quantization Options

Context length:

Quant	Bits	VRAM @ 16K	Quality
IQ3_XXS	3.25	20.0 GB 19.5 + 0.5 KV	low
IQ3_XS	3.5	21.4 GB 20.9 + 0.5 KV	low
Q3_K_S	3.64	22.2 GB 21.7 + 0.5 KV	low
IQ3_M	3.76	22.9 GB 22.4 + 0.5 KV	low
Q3_K_M	4	24.3 GB 23.8 + 0.5 KV	low
Q3_K_L	4.3	26.1 GB 25.6 + 0.5 KV	moderate
IQ4_XS	4.46	27.0 GB 26.5 + 0.5 KV	moderate
Q4_K_S	4.67	28.2 GB 27.7 + 0.5 KV	moderate
Q4_K_M	4.89	29.5 GB 29.0 + 0.5 KV	good
Q5_K_S	5.57	33.5 GB 33.0 + 0.5 KV	good
Q5_K_M	5.7	34.3 GB 33.8 + 0.5 KV	good
Q6_K	6.56	39.3 GB 38.8 + 0.5 KV	excellent
Q8_0	8.5	50.6 GB 50.1 + 0.5 KV	lossless
FP16	16	94.4 GB 93.9 + 0.5 KV	lossless

Select your GPU above to see speed estimates and compatibility for each quantization.

Deploying for a team or in production? Size GPUs, cost & scaling in Enterprise →

▸ READY TO RUN THIS?RENT BY THE HOUR

RENT A GPU AND RUN NOUS-HERMES-2-MIXTRAL-8X7B-DPO NOW

Rent on Vast.ai →Or RunPod →

Spin up an A100 / H100 / 4090 in ~60s. Pay by the second. Cancel anytime.

Community Ratings

Loading ratings...

Benchmarks (7)

Arena Elo1099

IFEval59.0

BBH37.1

MMLU-PRO29.6

MUSR16.7

MATH12.2

GPQA9.5

Run this model

▸Easiest way to get started·Beginners

DOCS ↗

curl -fsSL https://ollama.com/install.sh | sh

$ollama run nous-hermes2-mixtral:8x7b-dpo-q4_K_M

Downloads and runs automatically. Add --verbose for speed stats.

▸ SETUP GUIDE

Auto-setup with fitmyllm CLI

Detects your GPU, recommends the best model, downloads it, and starts chatting — zero config. Benchmarks your speed and contributes anonymous data to improve predictions.

pip install fitmyllmthen run fitmyllmLearn more

Auto-detect GPULive tok/s in chatSpeed benchmarks9 inference engines

HuggingFace Ollama Library GGUF Downloads Build Hardware

GPUs that can run this model

At Q4_K_M quantization. Sorted by minimum VRAM.

NVIDIA RTX 5090

32 GB VRAM • 1792 GB/s

NVIDIA

$1999

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

Apple M1 Max (32GB)

32 GB VRAM • 400 GB/s

APPLE

$1499

Apple M2 Max (32GB)

32 GB VRAM • 400 GB/s

APPLE

$1799

NVIDIA V100 SXM2 32GB

32 GB VRAM • 900 GB/s

NVIDIA

$3500

Apple M2 Pro (32GB)

32 GB VRAM • 200 GB/s

APPLE

$1499

Apple M4 (32GB)

32 GB VRAM • 120 GB/s

APPLE

$1199

NVIDIA Tesla V100 DGXS 32 GB

32 GB VRAM • 897 GB/s

NVIDIA

NVIDIA Tesla V100 PCIe 32 GB

32 GB VRAM • 897 GB/s

NVIDIA

NVIDIA Tesla V100 SXM2 32 GB

32 GB VRAM • 898 GB/s

NVIDIA

NVIDIA Tesla V100 SXM3 32 GB

32 GB VRAM • 981 GB/s

NVIDIA

AMD Radeon Instinct MI60

32 GB VRAM • 1020 GB/s

AMD

NVIDIA Tesla V100S PCIe 32 GB

32 GB VRAM • 1130 GB/s

NVIDIA

AMD Radeon Instinct MI100

32 GB VRAM • 1230 GB/s

AMD

$5000

NVIDIA RTX 5000 Ada Generation

32 GB VRAM • 576 GB/s

NVIDIA

$4000

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA GeForce RTX 5090

32 GB VRAM • 1790 GB/s

NVIDIA

$1999

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA GeForce RTX 5090 D

32 GB VRAM • 1790 GB/s

NVIDIA

$1999

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA Jetson AGX Xavier 32 GB

32 GB VRAM • 136 GB/s

NVIDIA

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA Quadro GV100

32 GB VRAM • 868 GB/s

NVIDIA

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA TITAN V CEO Edition

32 GB VRAM • 868 GB/s

NVIDIA

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA Tesla PG500-216

32 GB VRAM • 1130 GB/s

NVIDIA

NVIDIA Tesla PG503-216

32 GB VRAM • 1130 GB/s

NVIDIA

AMD Radeon Pro Vega II

32 GB VRAM • 825 GB/s

AMD

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

AMD Radeon Pro Vega II Duo

32 GB VRAM • 1020 GB/s

AMD

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

AMD Radeon PRO V620

32 GB VRAM • 512 GB/s

AMD

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

AMD Radeon PRO W6800

32 GB VRAM • 512 GB/s

AMD

$2249

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

AMD Radeon Pro W6800X

32 GB VRAM • 512 GB/s

AMD

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

AMD Radeon Pro W6800X Duo

32 GB VRAM • 512 GB/s

AMD

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

AMD Radeon Pro W6900X

32 GB VRAM • 512 GB/s

AMD

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA Jetson AGX Orin 32 GB

32 GB VRAM • 205 GB/s

NVIDIA

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

AMD Radeon PRO W7800

32 GB VRAM • 576 GB/s

AMD

$2499

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

Find the best GPU for Nous-Hermes-2-Mixtral-8x7B-DPO

Build Hardware for Nous-Hermes-2-Mixtral-8x7B-DPO

Model Card

View on HuggingFace

Nous Hermes 2 Mixtral — community fine-tune of Mixtral with improved helpfulness.

QUANT	BPW	VRAM	QUALITY
IQ3_XXS	3.25	19.5 GB	82%
IQ3_XS	3.5	20.9 GB	84%
Q3_K_S	3.64	21.7 GB	85%
IQ3_M	3.76	22.4 GB	86%
Q3_K_M	4	23.8 GB	88%
Q3_K_L	4.3	25.6 GB	90%
IQ4_XS	4.46	26.5 GB	92%
Q4_K_S	4.67	27.7 GB	93%
Q4_K_M	4.89	29.0 GB	94%
Q5_K_S	5.57	33.0 GB	96%
Q5_K_M	5.7	33.8 GB	96%
Q6_K	6.56	38.8 GB	97%
Q8_0	8.5	50.1 GB	100%
FP16	16	93.9 GB	100%

Nous-Hermes-2-Mixtral-8x7B-DPO

Quantization Options

Community Ratings

Benchmarks (7)

Run this model

Auto-setup with fitmyllm CLI

GPUs that can run this model

Model Card

Nous-Hermes-2-Mixtral-8x7B-DPO — 46.7B MoE.