In one sentence: OAI-SearchBot is the crawler OpenAI uses for search, fetching and indexing public pages so they can be returned as results and cited as sources when someone asks a question inside ChatGPT, which is a separate job from gathering training text.
OpenAI does not use one crawler for everything, and that split is the whole reason this term is worth learning. One name gathers text that may be used for training. OAI-SearchBot fetches and indexes public pages so they can be shown and cited when a question gets asked. A third name covers one off fetches made because a person pasted or referenced a specific link.
Only the middle one decides whether your pages are eligible to appear as a source. It behaves like a search crawler because that is what it is: it requests pages, reads the text, and keeps an index so an answer can point at something real instead of reciting from memory.
Picture a homeowner asking who reglazes bathtubs in their town. The answer comes back written out, with a short list of sources under it. Getting into that list is a search job, decided by what the search crawler could reach. It has nothing to do with whether a model read your words two years ago.
Indexing is the part worth pausing on. The crawler does not glance at a page and move along. It keeps what it read, so that later, when a question comes in, there is something on hand to search through and quote. A page it never stored cannot be quoted, however well it would have answered.
This is where local businesses lose without deciding anything. Blanket rules get written into robots.txt during a migration and never revisited. Some hosting firewalls and security plugins filter unfamiliar user agents by default. A staging password left on part of the site keeps everything behind it invisible. None of those started as a position on AI search, and every one of them can quietly keep you off the source list.
The distinction worth holding on to is this. Declining the training crawler is a preference about your writing. Blocking the search crawler is a decision to be left out of answers people are reading right now. Owners who genuinely want the second thing are rare. Owners who did it by accident are not, and the accident is invisible from the front end of the site. Nothing errors, no report turns red, and the pages look fine to every human who visits them.
Blanket advice does most of the damage. Somebody reads a publisher forum, decides to block every AI user agent in one pass, and treats four separate decisions as a single switch. Publishers who sell subscriptions have their reasons for that. A service business that wants the phone to ring is buying a different outcome, and usually not the one it thought it was buying.
Access only makes you eligible. Which sources get chosen is not something anyone controls, and it moves from question to question. If you want to see how assistants describe your business today, we publish more than 50 free tools at kellywm.com/tools with no email wall, including an AI visibility checker.
The general category is the AI crawler. The training half of OpenAI's split is GPTBot. The links that appear under a written answer are AI citations, and the back and forth way of asking that produces them is conversational search.
Usually because something else made the call for you: an old blanket rule left in robots.txt, a security layer that filters unknown user agents, or a section still sitting behind a staging password.
Not necessarily. Being crawlable makes a page eligible to be chosen. Whether it gets chosen depends on the question asked and what else is available, and it varies from one answer to the next.
AI visibility checker · AI crawler · GPTBot · AI citations · Conversational search · All glossary terms · Plain-English answers · AI search optimization services
Free consultation, plain-English advice. If you don't need us, we'll say so.
Book a free consultation → Or call/text directly: (407) 694-2055Tell us a little about the business and we will come back with an honest read: what we would fix first, what it costs, and whether you need us at all. Prefer to see work before you talk numbers? Get a free homepage mockup, built for your business, yours to keep either way.
Brandon reads every one of these himself. You will hear back shortly with an honest read on what we would do first, what it costs, and whether it is worth it for you.