How the numbers work.

What we measured ourselves

There are two kinds of number on this site and they are never mixed. Most are estimates: physics, fitted to measurements, honestly labelled. A growing minority are measurements: a machine was rented, the model was downloaded onto it, loaded and timed. Everything on this page describes how the first kind is built, and the second kind is what corrects it.

Every VRAM calculator on the internet runs the same roofline we do and then tells you to go validate it yourself. We went and validated it, and published what came back — including the parts that disagreed with us:

• • The context curve, the offload curve and the KV-cache formula on this site were all rewritten from those files after the machine contradicted them.
• • The speed model went from 36% to 5.5% mean error because of them.
• • Two silent defects in our own harness — an unset thread count, and VRAM peaks read device-wide without waiting for the previous model to be freed — were found by auditing a published report and are documented rather than quietly patched. Runs made before the fix are not shown at all.

Each report carries the llama.cpp build, the driver, the kernel, NUMA topology, BIOS and board, the PCIe link each card actually negotiated, power limits and clocks — because a card sitting at gen 3 ×8 changes the offload curve and nothing else in a report would show it. Headline speed is the median of independent runs with every raw value published and the peak-to-peak spread beside it.

Read the reports → Each one ships the raw JSON behind it at /api/rig/<slug>, free and without a key, so anything here can be checked against the source rather than taken on trust. That is the whole point of publishing them.

Model Size Estimation

The memory footprint of a model depends on its parameter count and quantization level:

Quantization levels:

Token Generation Speed

Decode is memory-bandwidth bound, and for years every calculator on the internet — this one included — wrote that as a single division: bandwidth over model size, times a fudge factor per GPU tier. Measuring it ourselves showed that form is wrong in a way no amount of tuning fixes. Time per token is affine in bytes, not proportional to them:

The fixed term is everything that costs the same whatever the model weighs: kernel launches, sampling, the Python and HTTP layers around the engine. It does not scale with size, so a one-term model has to absorb it into the efficiency factor — which is why that factor kept needing a different value for every GPU tier. It was not measuring the GPU. It was measuring how much of each card’s time was fixed cost.

Against the same measurements, the old one-term form came in at 36% mean error. This one is at 5.5%, and by context +1.3% / −0.0% / −7.4% at 4K / 16K / 32K. Putting the cache bytes inside the same division reproduces the context curve on its own, so the separate context coefficient the page used to document — a log-linear factor calibrated from one third-party 4090 run — is gone. It was fitting a curve that falls out of the arithmetic.

A sceptical detail, because it is the one that convinced us: the fit was redone from scratch on corrected-only data after three cards were re-run, and landed at 0.882 / 1.32 ms against the shipped 0.868 / 1.27. Nothing needed re-shipping. A fit that survives its own data being replaced is a fit, not a curve drawn through noise.

Mixture of Experts

A MoE does not read only its routed experts. Measured across six models, it reads the active parameters plus about 6.5% of everything it did not route to — prefetch, shared layers, and experts that happen to be resident:

Mean error 8.2%, worst 25.8%. Qwen3 30B-A3B, our densest sample at seven points across three cards, sits at −1.4%.

Where this model does not apply

The measured path is deliberately narrow. It runs for NVIDIA consumer cards up to 2,000 GB/s, dense and non-latent MoE, when the caller supplies the cache size. Everything else falls back to the older bandwidth model and is labelled as an estimate like any other:

• • AMD, Apple, Intel, datacenter — not measured by us. Apple Silicon is roughly 18% of the hardware our visitors browse from and is our largest coverage gap; we say so rather than extrapolating into it.
• • Latent attention (MLA) — its cost at depth scales with KV head count, not with bytes. DeepSeek Coder V2 Lite, 16 KV heads, falls to 32% of its 4K speed by 16K; GLM 4.7 Flash, same family and one KV head, only to 86%. A global fit including MLA gives 58.9% error, so MLA is excluded until there are measurements across varied head counts.
• • Partial offload to system RAM — three sweeps disagree by 5× at 75% residency, and we have not been able to separate the CPU of one rented pod from the law. Offload numbers are the weakest on the site and are marked as such.
• • Above 9.2× expert ratio — the MoE fit tops out there. Beyond it we are extrapolating.

Where the remaining error lives

The residual is not noise; it is structure we can name and have not yet modelled. Publishing it is cheaper than being caught by it:

• • Q3_K_M is slower than Q4_K_M — but only on Ampere. 3090: 119.1 against 138.0 tok/s. A6000: 168.7 against 182.6. On the Ada 4090 the whole ladder is monotonic (Q3 186.5 → Q8 103.7). So it is a kernel property of one architecture, not of the quantisation, and a global quant term would have smeared a real ~40% effect into a wrong ~15% one everywhere. Two cards of one architecture is too thin to ship a branch on, so it is documented and not applied.
• • Q8_0 runs faster than its byte count implies (~0.90 implied efficiency against ~0.84 on Q4), because Q8 dequantisation is nearly a scalar multiply. Part of what we call “efficiency” is really a Q4-kernel property.
• • Gemma 3 is consistently slower than its bytes predict — 1.10–1.23× across every quant level and both cards. Flat along the ladder, which is what says it is an architecture term rather than a quantisation one. Unmodelled.
• • Worst single point today: Qwen3.5 9B at 32K on a 4090, −34%. Its K and V are 256 wide on 4 KV heads, so its cache grows unusually fast. A test asserts no point exceeds 35%, mean error stays under 10% and bias under 5%, against every depth measurement on disk — so this gets worse only loudly.

Prefill Speed (Prompt Processing)

Processing the input prompt is compute-bound:

Real-world utilization is much lower than theoretical due to attention O(n²) complexity, memory bandwidth contention, and kernel overhead:

• • With tensor cores (consumer GPU): ~4%
• • Without tensor cores: ~1.5%
• • Datacenter (>500 TFLOPS): ~6%

Capped at realistic maximums: 500 (consumer), 1000 (>100 TFLOPS), 3000 (datacenter). This determines Time to First Token (TTFT) — how long before generation starts.

KV Cache

The cache holds attention state for every token seen so far, and on long contexts it is routinely larger than the weights. The formula in most calculators — including the one this page used to print — is a curve fitted to parameter count. It is wrong on four of the architectures we ship, so the cache is now computed from each model’s own geometry, read from the GGUF header the runtime actually loads:

• • Grouped-query (most models). The formula above. K and V are sized independently in the file, and are only equal by coincidence on the models everyone measures.
• • Sliding-window and hybrid attention. Gemma 3 holds a bounded window on five layers in six. Qwen 3.5 publishes its KV head count as a per-layer array with a full-attention interval of 4 — three layers in four keep no growing cache at all. Treating that as uniform overstates the cache about three times: 128 KiB/token predicted against 40 KiB/token measured on an RTX 4060.
• • Latent attention (MLA). DeepSeek, GLM, Kimi, Ling, Mistral Small 4 — 27 models in the catalogue — do not cache K and V per head at all. They cache one compressed latent per token per layer, shared across every head. The head-count formula does not mis-size that; it computes a different quantity. DeepSeek Coder V2 236B reads as 128 heads × 128 dims and comes out 57 times its real cache. Every one of those models was being told it needed tens of gigabytes it does not.
• • Chunked attention. A layer restricted to its own attention chunk holds no more than one with a window that size.

Quantising the cache scales it directly: --cache-type-k q8_0 halves K. Halving V needs flash attention enabled as well, which is why people set the flag and conclude it did nothing.

The MLA widths are read from each model’s published config, not measured on our own rig. We say so on the page rather than in a footnote: an unconfirmed 1× is still better than a confirmed 25×, but it is not a measurement and is not presented as one.

Runtime Overhead

Weights plus KV cache was never the whole bill. The runtime reserves compute buffers, CUDA context and workspace before a single token is generated, and leaving it out is what turned a model at 72% residency into a green FITS badge on this site.

The overhead is scaled against what Ollama actually reserved across eight measured models, and multimodal models carry their own term — a vision tower’s activation buffers ran to 908 MiB on a 4B and 1.9 GiB on a 9B, close to a third of their weights. On top of that, the fit budget leaves 10% of the card free, because on the machine these were measured on the desktop alone held ~700 MiB before the engine started.

Multi-GPU

This section used to say that a second consumer card adds about 30% more bandwidth. It does not, and the error was visible on our own page: a model small enough to fit inside one RTX 3060 got 21% faster when a second RTX 3060 was added to the form. Nothing about the first card’s memory had changed.

llama.cpp and Ollama split a model by layer by default. Each card holds a contiguous block, so generating one token walks them in order and exactly one GPU is reading weights at a time while the others wait on a hidden state a few KB wide. Per-token time is the sum of the per-card read times, which makes the effective bandwidth their share-weighted harmonic mean:

For identical cards that is one card’s bandwidth. Two RTX 3060s decode at the speed of one RTX 3060 — and hold a model that neither could hold alone. Capacity is what scales; the win is a model that stops spilling into system RAM, which is worth far more than the few percent we used to invent. A mixed pair lands between its two cards, pulled toward the slower one, because the larger card holds more layers and the slower one is never off the critical path.

Bandwidth only aggregates under a tensor-parallel split, where every card holds a slice of every tensor and they read at once — vLLM’s -tp, exllamav2, llama.cpp’s -sm row. It is paid for with an all-reduce at every layer, which is what a fast interconnect is for. So NVLink changes nothing for a single user on the default path, and everything for a batched vLLM deployment. The site asks which one you are running rather than which cable you own.

Interconnect never changes capacity. Only the number of cards does.

When a Measurement Overrides the Estimate

Visitors submit runs from their own hardware, and every result on the site carries a badge saying which kind of number it is: measured, community or estimated, with the sample count.

Submissions do not simply replace the estimate, because a number anyone can post is a number anyone can move. Theory stays as the anchor and the measured median may only refine it inside a fence:

• • Below 3 distinct contributors, a measurement is displayed but does not move the number.
• • From 3 to 10, its weight grows with the count.
• • At 10 or more it drives the figure — but never further than ±25% from what the physics says. A run outside that band moves the estimate to the edge of it and no further.

Runs are also separated by where they happened. A model fully resident on the GPU, one partially offloaded to system RAM and one on CPU are three different measurements of three different things, and the published median only ever uses full-GPU runs. Mixing them is how an average becomes a number that describes nobody’s machine.

The gates are deliberately conservative and will relax as volume grows. The honest state today: measurement coverage is thin against a 449-model catalogue, so most cells you see are still physics, and they say so.

Scoring System

We use a quality-first multiplicative model. Unlike additive systems (quality + speed + quant), multiplication ensures quality always dominates rankings — speed only penalizes truly slow configurations:

Why multiplicative? With additive scoring, a tiny fast model can outscore a much smarter but slower model. Multiplicative scoring means a fast 2B model can never beat a 9B just because it generates tokens faster — speed matters only when it becomes a usability problem.

1. Quality Score

Benchmarks are normalized to random baseline (following HuggingFace Open LLM Leaderboard v2). This prevents benchmarks with different chance levels from being compared directly — GPQA 37% barely beats 25% random, but IFEval 37% is meaningful:

Marked FROZEN: supplied by the Open LLM Leaderboard, which has been offline since March 2025. Those columns cannot be refreshed for a model released since, so their weight is being retired in favour of benchmarks that are still being run. A model scored partly on a frozen column is scored on evidence that stopped arriving, and pretending otherwise is how a ranking quietly ages.

2. Arena Elo Blend

For models with LMSYS Chatbot Arena ratings (6M+ human preference votes), we blend 70% benchmarks + 30% Arena Elo. When benchmark coverage is poor, Arena weight increases up to 50% to rescue models with strong real-world quality but incomplete benchmark data.

3. Speed Viability

Speed is a non-linear multiplier (0 → 1.0), not a competing score. Once speed is comfortable, more speed barely matters:

Interactive = chat, creative, roleplay. Batch = coding, reasoning, agentic, embedding.

4. Quantization & Coverage

Quantization is applied as a natural quality multiplier (Q4 = 0.94×, Q8 = 0.995×, FP16 = 1.0×), not a separate competing component.

Coverage penalty: Models with fewer benchmarks get penalized (2/4 = 57.5% without Arena, 70% with Arena). Arena Elo reduces the penalty because it independently validates quality.

Enterprise TCO Calculation

The Enterprise deployment planner compares three options: pay-per-use APIs, cloud GPU rental, and on-premise hardware.

On-Premise Running Costs

Break-Even Calculation

The interactive break-even chart lets you drag a time slider to visualize when self-hosting becomes cheaper than API or cloud GPU rental.

Data Sources

• •Our own measurements: Rented GPUs, llama.cpp and vLLM, published in full with the raw JSON — the only source here we control end to end
• •Model architecture: Read from each model’s GGUF header over an HTTP range request: layer count, KV heads, key and value lengths, latent ranks. Not a model card, not a config.json — the file the runtime loads
• •Model benchmarks: Artificial Analysis, EvalPlus, BigCodeBench, and published model cards. The Open LLM Leaderboard, which supplied IFEval / BBH / MATH / MuSR, has been offline since March 2025 — those columns are frozen and are being retired rather than refreshed
• •GPU and CPU specifications: Vendor spec sheets — 1,720 GPUs and the CPU table behind offload estimates
• •GPU prices: MSRP for current-generation cards. Discontinued cards show no price at all, because a stale MSRP is worse than a blank
• •Cloud GPU and API pricing: RunPod, Vast.ai, Lambda, AWS, GCP, Azure; OpenAI, Anthropic, Google. Refreshed on a schedule and stamped with the date it was read
• •Community measurements: Submitted runs, gated as described above and never averaged across placement
• •KV cache and parallelism literature: vLLM / PagedAttention (Kwon et al., 2023) and Megatron-LM scaling (Shoeybi et al., 2020) — used for the batched serving model, not for single-user decode

Data Validation

All automated data updates pass through validation guardrails:

• • GPU prices must be within $80–$100K range
• • Price swings >50% from previous value are rejected
• • Model benchmark scores must be 0–100
• • Duplicate model IDs are detected and deduplicated
• • Model count guardrail prevents accidental data wipe
• • All validation failures block the update — no bad data is committed

Limitations

The short version: single-user decode on NVIDIA consumer hardware is the case we have measured and the one we are confident about. Confidence drops as you move away from it, and here is the order it drops in.

If you find a number here that your machine contradicts, that is the most useful thing you can send us — and it is how most of the corrections above were found.