FITMYLLM·AUGUST 17, 2026·OPEN METHODOLOGY · COMMUNITY BENCHMARKS

LIVE·UPDATED DAILY

Independent benchmarks for self-hosted AI infrastructure.

▸Privacy Shield

TII/Dense

Falcon Mamba 7B

- Developed by: https://www.tii.ae - Model type: Causal decoder-only - Language(s) (NLP): Mainly English - License: TII Falcon-Mamba License 2.0

chat

7.27B

Parameters

8K

Context length

6

Benchmarks

6

Quantizations

0

Architecture

Dense

Released

2024-08-12

Layers

64

KV Heads

1

Head Dim

64

Family

falcon

Quantization Options

Context length:

Quant	Bits	VRAM @ 8K	Quality
Q4_K_M	4.89	5.0 GB 4.9 + 0.1 KV	good
Q5_K_S	5.57	5.6 GB 5.5 + 0.1 KV	good
Q5_K_M	5.7	5.7 GB 5.7 + 0.1 KV	good
Q6_K	6.56	6.5 GB 6.4 + 0.1 KV	excellent
Q8_0	8.5	8.3 GB 8.2 + 0.1 KV	lossless
FP16	16	15.1 GB 15.0 + 0.1 KV	lossless

Select your GPU above to see speed estimates and compatibility for each quantization.

Deploying for a team or in production? Size GPUs, cost & scaling in Enterprise →

▸ READY TO RUN THIS?RENT BY THE HOUR

RENT A GPU AND RUN FALCON MAMBA 7B NOW

Rent on Vast.ai →Or RunPod →

Spin up an A100 / H100 / 4090 in ~60s. Pay by the second. Cancel anytime.

Community Ratings

Loading ratings...

Benchmarks (6)

IFEval76.1

MATH40.9

BBH37.9

MMLU-PRO34.3

MUSR21.2

GPQA8.1

Run this model

▸Easiest way to get started·Beginners

curl -fsSL https://ollama.com/install.sh | sh

$ollama run falcon:7b

Tag may need adjustment — check ollama.com/library/falcon for available tags.

▸ SETUP GUIDE

>_

Auto-setup with fitmyllm CLI

Detects your GPU, recommends the best model, downloads it, and starts chatting — zero config. Benchmarks your speed and contributes anonymous data to improve predictions.

pip install fitmyllmthen run fitmyllmLearn more

Auto-detect GPULive tok/s in chatSpeed benchmarks9 inference engines

HuggingFace Ollama Library GGUF Downloads Build Hardware

GPUs that can run this model

At Q4_K_M quantization. Sorted by minimum VRAM.

NVIDIA Tesla K20c

5 GB VRAM • 208 GB/s

NVIDIA

NVIDIA Tesla K20m

5 GB VRAM • 208 GB/s

NVIDIA

NVIDIA Tesla K20s

5 GB VRAM • 208 GB/s

NVIDIA

NVIDIA GeForce GTX 1060 5 GB

5 GB VRAM • 160 GB/s

NVIDIA

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA P102-100

5 GB VRAM • 440 GB/s

NVIDIA

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA Quadro P2000

5 GB VRAM • 140.2 GB/s

NVIDIA

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA Quadro P2200

5 GB VRAM • 200.2 GB/s

NVIDIA

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA RTX 3050 6GB

6 GB VRAM • 168 GB/s

NVIDIA

$169

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

6 GB VRAM • 186 GB/s

INTEL

$129

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA RTX 2060 6GB

6 GB VRAM • 336 GB/s

NVIDIA

$150

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA GTX 1660 SUPER

6 GB VRAM • 336 GB/s

NVIDIA

$150

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA GTX 1660 Ti

6 GB VRAM • 288 GB/s

NVIDIA

$140

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA GTX 1060 6GB

6 GB VRAM • 192 GB/s

NVIDIA

$80

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA Tesla C2070

6 GB VRAM • 143 GB/s

NVIDIA

NVIDIA Tesla C2075

6 GB VRAM • 150 GB/s

NVIDIA

NVIDIA Tesla C2090

6 GB VRAM • 177 GB/s

NVIDIA

NVIDIA Tesla M2070

6 GB VRAM • 150 GB/s

NVIDIA

NVIDIA Tesla M2070-Q

6 GB VRAM • 150 GB/s

NVIDIA

NVIDIA Tesla M2075

6 GB VRAM • 150 GB/s

NVIDIA

NVIDIA Tesla M2090

6 GB VRAM • 177 GB/s

NVIDIA

NVIDIA Tesla X2070

6 GB VRAM • 177 GB/s

NVIDIA

NVIDIA Tesla X2090

6 GB VRAM • 177 GB/s

NVIDIA

NVIDIA Tesla K20X

6 GB VRAM • 250 GB/s

NVIDIA

NVIDIA Tesla K20Xm

6 GB VRAM • 250 GB/s

NVIDIA

NVIDIA GeForce GTX 1060 6 GB

6 GB VRAM • 192 GB/s

NVIDIA

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA GeForce GTX 1060 6 GB 9Gbps

6 GB VRAM • 217 GB/s

NVIDIA

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA GeForce GTX 1060 6 GB GDDR5X

6 GB VRAM • 192 GB/s

NVIDIA

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA GeForce GTX 1060 6 GB GP104

6 GB VRAM • 192 GB/s

NVIDIA

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA GeForce GTX 1060 6 GB Rev. 2

6 GB VRAM • 192 GB/s

NVIDIA

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

NVIDIA GeForce GTX 1660

6 GB VRAM • 192 GB/s

NVIDIA

Amazon Newegg eBay PCPartPicker Rent on RunPod Rent on Vast.ai

Find the best GPU for Falcon Mamba 7B

Build Hardware for Falcon Mamba 7B

▸ SPEC SHEET

Falcon Mamba 7B — 7.27B Dense.

▸ SPECIFICATIONS

PARAMETERS: 7.27B
ARCHITECTURE: Dense Transformer
CONTEXT LENGTH: 8K tokens
CAPABILITIES: chat
RELEASE DATE: 2024-08-12
PROVIDER: TII
FAMILY: falcon

▸ VRAM REQUIREMENTS

QUANT	BPW	VRAM	QUALITY
Q4_K_M	4.89	4.9 GB	94%
Q5_K_S	5.57	5.5 GB	96%
Q5_K_M	5.7	5.7 GB	96%
Q6_K	6.56	6.4 GB	97%
Q8_0	8.5	8.2 GB	100%
FP16	16	15.0 GB	100%

§ 01BENCHMARK SCORES

MMLU-PRO34.3

MATH40.9

IFEval76.1

BBH37.9

GPQA8.1

MUSR21.2

§ 02RUN COMMAND

Run Falcon Mamba 7B locally with Ollama — needs 4.9 GB VRAM at Q4_K_M:

$ollama run falcon:7b

§ 03COMPATIBLE GPUs

30 @ Q4_K_M

NVIDIA Tesla K20c

5 GB · 208 GB/s

NVIDIA Tesla K20m

5 GB · 208 GB/s

NVIDIA Tesla K20s

5 GB · 208 GB/s

NVIDIA GeForce GTX 1060 5 GB

5 GB · 160 GB/s

NVIDIA P102-100

5 GB · 440 GB/s

NVIDIA Quadro P2000

5 GB · 140.2 GB/s

NVIDIA Quadro P2200

5 GB · 200.2 GB/s

NVIDIA RTX 3050 6GB

6 GB · 168 GB/s

6 GB · 186 GB/s

NVIDIA RTX 2060 6GB

6 GB · 336 GB/s

NVIDIA GTX 1660 SUPER

6 GB · 336 GB/s

NVIDIA GTX 1660 Ti

6 GB · 288 GB/s

NVIDIA GTX 1060 6GB

6 GB · 192 GB/s

NVIDIA Tesla C2070

6 GB · 143 GB/s

NVIDIA Tesla C2075

6 GB · 150 GB/s