Rig reports. Nothing estimated.

Tokens per second generating with the model fully resident on the GPU. Median of several independent runs, each from a cold load.

Tokens per second processing the prompt — the half of latency that a decode figure does not describe.

Time to first token at the median and the 95th percentile. p95 is what an interactive client actually experiences.

The maximum the driver reported during the run, not the weight size. The gap between them is the runtime’s own allocation, and it is why a model fits on paper and not on the card.

The same weights at several rungs on the same card: what each extra bit per weight costs in tokens per second and in VRAM.

Speed with the KV cache already filled to 4K, 8K, 16K, 32K and beyond — not the empty-cache figure, and with the depth at which the model stops loading.

Tokens per second at each fraction of the model kept resident, for the case where it does not fit and layers cross PCIe every token.

Aggregate throughput, per-stream throughput and TTFT p95 at 1, 2, 4, 8 and 16 streams, measured over the seconds the server was actually saturated.

Deterministic probes at temperature 0 with a programmatic check, plus a separate looping test — because a damaged quantisation posts a higher token rate, not a lower one.

Watts sampled at the card during generation, and cost per million tokens at the rate the machine was billed, single stream and batched.

On and off, at depth. Both LM Studio and Ollama enable it by default; whether it helps depends on the model and the cache size.

The card’s decode constants and implied bandwidth, the fixed VRAM overhead before any weights, and how TTFT scales with concurrency — each with its n and its correlation.