Mixture of experts (MoE) — A mixture of experts is a model design that contains many specialized sub-networks, called experts, and activates only a few of them for each token. This gives a model large total capacity while keeping each answer relatively cheap to compute. Several leading models use this design. It is one way to scale models efficiently.

Added October 21, 2026