Rig reports. Nothing estimated.
Every other number on this site is a prediction from memory bandwidth, honestly labelled as one. These are not. Each report below is a physical machine we rented, on which each model was actually loaded and timed — decode and prefill, TTFT at the 95th percentile, the VRAM it really took, how it degrades under concurrent users, watts at the wall, and what a million tokens costs. The llama.cpp build and the driver version are recorded so the numbers can be reproduced, or argued with.
Tokens per second generating with the model fully resident on the GPU. Median of several independent runs, each from a cold load.
Tokens per second processing the prompt — the half of latency that a decode figure does not describe.
Time to first token at the median and the 95th percentile. p95 is what an interactive client actually experiences.
The maximum the driver reported during the run, not the weight size. The gap between them is the runtime’s own allocation, and it is why a model fits on paper and not on the card.
The same weights at several rungs on the same card: what each extra bit per weight costs in tokens per second and in VRAM.
Speed with the KV cache already filled to 4K, 8K, 16K, 32K and beyond — not the empty-cache figure, and with the depth at which the model stops loading.
Tokens per second at each fraction of the model kept resident, for the case where it does not fit and layers cross PCIe every token.
Aggregate throughput, per-stream throughput and TTFT p95 at 1, 2, 4, 8 and 16 streams, measured over the seconds the server was actually saturated.
Deterministic probes at temperature 0 with a programmatic check, plus a separate looping test — because a damaged quantisation posts a higher token rate, not a lower one.
Watts sampled at the card during generation, and cost per million tokens at the rate the machine was billed, single stream and batched.
On and off, at depth. Both LM Studio and Ollama enable it by default; whether it helps depends on the model and the cache size.
The card’s decode constants and implied bandwidth, the fixed VRAM overhead before any weights, and how TTFT scales with concurrency — each with its n and its correlation.
Every speed is the median of several independent processes, each of which reloaded the model from cold; the individual runs and their spread are printed, so a figure quoted to three digits can be checked against the precision it actually has. Models that failed to load are listed with the stage they failed at. The llama.cpp build, the driver version and the machine are recorded on every report, and each one carries the raw JSON it was rendered from.
NVIDIA GeForce RTX 4090
FREEFastest measured: 288.8 tok/s on gpt-oss 20B · largest that fits: Qwen3 30B A3B Instruct 2507
READ THE FULL REPORT →NVIDIA GeForce RTX 3090
FREEFastest measured: 227.4 tok/s on gpt-oss 20B · largest that fits: Qwen3 30B A3B Instruct 2507
READ THE FULL REPORT →Renting the machine is the cheap part. What takes the time is knowing which axes are worth sweeping — context depth, the quantisation ladder, the offload curve, concurrency — and building a harness that does not quietly measure itself instead of the hardware. Each report opens on the fastest five models measured, so you can check them against a card you already own before deciding.
NVIDIA L4
€242 quant ladders · offload curve ×8 · concurrency ×11 · accuracy ×11 · watts + €/Mtok ×11
PREVIEW AND BUY →NVIDIA RTX 4000 Ada Generation
€242 quant ladders · offload curve ×11 · concurrency ×17 · accuracy ×17 · watts + €/Mtok ×17
PREVIEW AND BUY →NVIDIA A40
€242 quant ladders · offload curve ×11 · concurrency ×17 · accuracy ×17 · watts + €/Mtok ×17
PREVIEW AND BUY →2× NVIDIA A40
€242 quant ladders · offload curve ×11 · concurrency ×17 · accuracy ×17 · watts + €/Mtok ×17
PREVIEW AND BUY →NVIDIA L40S
€242 quant ladders · offload curve ×11 · concurrency ×17 · accuracy ×17 · watts + €/Mtok ×17
PREVIEW AND BUY →NVIDIA GeForce RTX 5090
€242 quant ladders · context curve ×11 · offload curve ×11 · concurrency ×17 · accuracy ×17 · watts + €/Mtok ×17
PREVIEW AND BUY →NVIDIA RTX A6000
€241 quant ladder · context curve ×10 · concurrency ×12 · accuracy ×12 · watts + €/Mtok ×12
PREVIEW AND BUY →One payment for the report you want — no subscription, no bundle to work out, and no expiry. Everything else on this site, the market report below included, is free.
These machines were rented and measured because we wanted the numbers. If you need the same for hardware we have not run — your models, your context lengths, your concurrency ceiling, your quantisations — that is work we take on commission.
You would get exactly what is on this page: every measurement, the raw JSON, the host it ran on, the build it was measured against — and the results that do not flatter anyone, because the alternative is a report nobody can cite. Tell us the machine and the workload and we will say whether it is worth measuring at all.
ASK ABOUT A REPORT →August 2026 — what rent did.
Computed, not written. Every figure comes out of the rates we record from RunPod, Vast.ai and DataCrunch each morning — which means it can be regenerated for any past month and come out the same, and that it cannot exist for a month we did not watch. Free, like the rest of this page except the measurements above.
August 2026 has 3 days recorded, across 91 cards. A report needs 14.
Below that, one provider outage or a single odd weekend moves every figure in the document and nothing on the page would show it. The record grows by a day each morning, and this becomes a report on its own.
Nothing earlier yet — the record starts the day we began keeping it, and this is the first month of it. Each month from here joins the archive.
Every VRAM calculator on the internet runs the same roofline we do and tells you to go validate it yourself. We went and validated it. The measurements here are also what corrects the estimator: the context curve, the offload curve and the KV-cache formula on this site were all rewritten from these files after they disagreed with the machine. How the estimates work → What a commissioned report contains →