Transformer — The transformer is the neural network design behind nearly all modern language models. Introduced by Google researchers in 2017, it uses a mechanism called attention to weigh how words relate to each other. Transformers process text efficiently in parallel, which made very large models practical. The T in GPT stands for transformer.

Added October 21, 2026