Next-token prediction — Next-token prediction is the basic task language models are trained on: guessing the next token in a passage of text. Repeated billions of times over huge datasets, this simple task teaches grammar, facts, and reasoning patterns. When a model writes, it predicts one token at a time. Much of what modern AI can do grows from this one skill.

Added October 21, 2026