Googlebot — Googlebot is the crawler Google uses to index pages for Google Search. Blocking it removes a website from Google Search results, so most businesses always allow it. It is separate from Google-Extended, which controls AI training use. A careful robots.txt allows Googlebot while making a separate choice about training.
Added October 21, 2026

