Quantization — Quantization is shrinking a model by storing its numbers with less precision, such as 8 bits or 4 bits instead of 16. It makes models smaller, faster, and cheaper to run, sometimes on laptops or phones. Aggressive quantization can reduce quality. It is common for running open-weight models locally.

Added October 21, 2026