Attention mechanism — An attention mechanism lets a model focus on the most relevant parts of its input when producing each piece of output. When reading a sentence, it helps the model connect a pronoun to the noun it refers to. It is the core idea of the transformer. Attention is why language models can track meaning across long passages.

Added October 21, 2026