Distillation — Distillation is training a smaller model to imitate a larger, more capable one. The smaller model learns from the larger model’s outputs instead of only from raw data. The result runs faster and cheaper while keeping much of the larger model’s skill. Many small, efficient models are made this way.

Added October 21, 2026