Quant benchmarks

Where the community benchmarks report what people say, these reports measure what the numbers are: for each model, the full-precision (BF16/FP8) baseline scores sit next to the scores of its published quantized checkpoints, and the gap between the two is the quality delta — what you give up in accuracy when you shrink the model to fit your hardware.

A delta row lists one benchmark, the baseline score, the quantized score and the difference; a null baseline means the source card published the quantized number without a reference to diff against, so the row shows the absolute score only. Every number is sourced from official vendor model cards, quantization publisher cards and serving recipe pages — no community opinions — and gaps are called out explicitly where nothing is published.

How to read them: deltas under one point are effectively lossless for most use cases, a few points of drift is the usual cost of 4-bit formats, and anything larger is a real trade-off to weigh against the memory and serving-speed gains the smaller checkpoint brings.