Retrieval agent — A retrieval agent is an automated program that fetches web content at the moment a user asks an AI/SI system a question. It lets the system cite current information and link back to the source. Allowing retrieval agents keeps a business visible in AI/SI answers. Most businesses want retrieval agents allowed even if they block training crawlers.

Added October 21, 2026