Your guide to vendor risk assessments ranks on page one of Google. A prospect asks ChatGPT the same question, and the answer cites a competitor ranking well below you. Your CMO wants to know why.
Here's why: how AI search chooses sources isn't the same as how Google ranks pages. Rankings help a page get found; the answer is built from the passages each platform judges most useful.
|
The short answer: how do AI search engines choose sources? AI search engines that use live retrieval interpret the question, often split it into related searches, pull candidate pages from a search index, and select the passages that answer each part most directly. Pages must be crawlable and indexable, and credible, clearly structured answers help. Ranking makes a page discoverable but doesn't guarantee a citation. |
|
Key Takeaways
|
Most AI search products use retrieval-augmented generation (RAG): the system retrieves relevant documents, then writes an answer grounded in them (Lewis et al., 2020). The stages below are typical.
A model learns from training data collected up to a cutoff date. That knowledge stays fixed until the next model version and usually carries no citation.
Live retrieval is different: when ChatGPT search, Perplexity, or Google's AI features search the web, they read current pages and can link to them.
Not every AI answer uses live search. An answer from training data alone may name brands without citing anything.
Buyer prompts are often layered: "What should a mid-market manufacturer look for in a demand generation partner, and how do pricing models compare?" The system may split that into narrower searches.
Query fan-out means turning one question into several related searches. Google says AI Mode "uses our query fan-out technique, breaking down your question into subtopics" (Google, May 2025), and its AI features guidance says AI Overviews and AI Mode may use it. OpenAI says ChatGPT search may rewrite a prompt into targeted queries for its search providers (OpenAI Help Center).
Each query returns candidate pages from a search index. Pages that aren't indexed or can't be fetched never become candidates. Google requires pages to be indexed and snippet-eligible to appear as supporting links (Google Search Central).
Re-ranking means scoring retrieved candidates a second time, often passage by passage, to find the text that best answers each sub-question.
Hypothetically, a buyer asks how long a SOC 2 Type II audit takes. Vendor A's compliance page ranks second but buries the timeline in a features paragraph. Vendor B's guide ranks ninth, yet its heading asks the exact question and a two-sentence answer follows. Vendor B's passage is the cleaner match and the stronger citation candidate.
The model writes an answer from the selected material and links sources supporting specific claims. How each platform weighs sources is proprietary and changes over time, so treat claims of a precise "AI ranking algorithm" with caution.
Figure 1. A simplified five-stage model of how AI search engines select and cite sources.
Reading the diagram: a prompt is interpreted and may fan out, pages are retrieved, the best passages are re-ranked, and the model writes a cited answer. Beneath each stage is what B2B brands can influence. Platforms differ.
When a prompt needs current information, ChatGPT searches the web, partly through third-party providers, and shows its sources. OpenAI separates crawlers by purpose (OpenAI crawler documentation):
OpenAI says each setting is independent, so you can block GPTBot for training and still allow OAI-SearchBot.
Perplexity is search-first; answers include "numbered citations linking to the original sources" (Perplexity Help Center). Its crawler documentation says PerplexityBot is "designed to surface and link websites in search results" and isn't used for foundation-model training. Perplexity-User, which fetches pages on request, "generally ignores robots.txt rules."
These are separate products: AI Overviews summarize topics within results, while AI Mode is a conversational experience for "complex comparisons." Both draw on Google's index, and Google says they "are rooted in our core Search ranking and quality systems" (Google Search Central). Both may use query fan-out.
Google lists no requirements beyond indexing and snippet eligibility. Google-Extended, the token for Gemini training and grounding, "does not impact a site's inclusion in Google Search" (Google crawler documentation). SEO makes pages eligible, not certain to be cited.
Platform comparison at a glance
|
Platform |
Source discovery |
Crawlers |
Citation display |
Practical B2B optimization priority |
|
ChatGPT Search |
Live web search via third-party providers; may rewrite prompts |
OAI-SearchBot (search); GPTBot (training); ChatGPT-User (user-initiated) |
Linked sources when search is used |
Allow OAI-SearchBot; decide on GPTBot separately |
|
Perplexity |
Real-time web search; Research mode runs dozens of searches |
PerplexityBot (search, not training); Perplexity-User (user-initiated) |
Numbered citations |
Allow PerplexityBot in robots.txt and firewalls |
|
Google AI Overviews / AI Mode |
Google Search index and core ranking systems; may use fan-out |
Googlebot (index); Google-Extended (Gemini training and grounding, not Search) |
Supporting links to indexed, snippet-eligible pages |
Technical SEO and coverage of related subtopics |
Source: official platform documentation, verified November 12, 2026.
No platform publishes a scoring formula, so each signal below is labeled by evidence type.
Evidence type: Documented requirement
This is the hard gate. Check robots.txt for Googlebot, OAI-SearchBot, and PerplexityBot, and remember CDN bot protection and firewalls can block crawlers robots.txt allows. Keep key content in rendered HTML, not behind forms or client-side scripts alone.
Evidence type: Reasoned recommendation
Selection happens at the passage level, so a section that answers one question directly is easier to match. Use headings phrased as buyers ask, each followed by a short answer. Google says there's no need to chop content into tiny pieces; aim for clarity.
Evidence type: Practitioner observation
Practitioners commonly report that brands described consistently across credible publications, review sites, and communities appear more often in AI answers, though no platform documents corroboration as a scoring factor. Google cautions that "seeking inauthentic 'mentions' across the web isn't as helpful as it might seem," and its spam policies apply to generative AI responses.
Evidence type: Reasoned recommendation
Google says retrieval helps "improve the quality, accuracy, and freshness of AI responses." Review key pages on a schedule, replace outdated statistics, and show visible publication and revision dates. Change the date only when the content changes.
Evidence type: Reasoned recommendation
One-sentence definitions, numbered steps, tables, and short paragraphs are easy to quote accurately. Use real HTML, not text inside images.
Evidence type: Reasoned recommendation
If your website and profiles describe your company differently, systems have to guess. Keep one consistent description, a detailed About page, unambiguous service names, and Organization schema that matches visible content. Structured data aids machine understanding but guarantees no citation.
A traditional ranking reflects one query. An AI answer may draw on several fanned-out sub-queries with their own results, so a page ranking ninth for your head term might rank first for a sub-question the model needed.
Passage relevance adds a second filter: the clearest passage for a claim wins, not the strongest page. Platforms also use different indexes, and answers weighing several viewpoints may cite a spread of sources.
SEO makes your pages discoverable; AEO makes what's inside them useful enough to select. See PMG360's guide to B2B content that earns pipeline and AI citations.
These are common barriers, not proven reasons any particular system excludes a page:
To prioritize, use this Citation Readiness Check: a practical evaluation framework, not a reverse-engineered ranking algorithm. Rate each dimension red, amber, or green for your top ten buyer questions.
|
Dimension |
Question to ask |
Quick test |
|
Source accessibility |
Can crawlers fetch and render the page? |
Review robots.txt, CDN/firewall logs, rendered HTML |
|
Answer relevance |
Does one section answer the question directly? |
Read only the heading and first two sentences |
|
Information credibility |
Are claims sourced and corroborated? |
Check statistic sources and third-party descriptions |
|
Brand entity clarity |
Would a system describe you accurately? |
Compare About page, schema, and profiles |
|
Citation monitoring |
Do you know where you're cited today? |
Run the same buyer prompts monthly; log sources |
Related: preparing for fewer search clicks.
PMG360 is a B2B demand generation partner, and its AEO / AI Visibility Optimization service puts technical health and rankings first, then rewrites key pages so each answers one buyer question in plain language. PMG360 won't trade your Google rankings for AI mentions or promise a guaranteed spot in ChatGPT, because nobody can honestly promise that.
PMG360 tests real buyer prompts in ChatGPT, Perplexity, Gemini, and Google AI Overviews to show where AI mentions you, where it cites competitors instead, and what keeps your content out. Reporting covers citations, AI referral traffic, and leads, tied to pipeline.
AI search engines that use live retrieval interpret the question, search an index (often with several related queries), select the passages that answer it most directly, and link supporting sources. Exact selection logic isn't public.
No. A page must be indexed and snippet-eligible to appear as a supporting link, but AI features may use query fan-out and select passages from other pages. A top ranking improves discoverability, not certainty.
No. According to OpenAI, GPTBot governs whether content may be used for model training, while OAI-SearchBot determines whether a site can appear in ChatGPT search answers. The settings are independent.
They use different crawlers, indexes, search providers, and models, and may rewrite the same prompt into different searches. Results can also change over time.
There's no published timeline. A page must first be crawled and indexed by each platform; OpenAI says robots.txt changes take about 24 hours to register for OAI-SearchBot. Track results over weeks and months.
Yes, it's possible. AI answers select passages for specific questions, so a smaller company with a clearer, better-supported answer to a niche question can be cited alongside or ahead of larger brands. Nothing is guaranteed.
PMG360's free AI visibility audit shows which buyer prompts surface your brand, which competing sources appear in AI answers instead, and which technical and content improvements to prioritize first.
Request your free AI visibility audit →
1. OpenAI — Overview of OpenAI Crawlers (OAI-SearchBot, GPTBot, ChatGPT-User). https://developers.openai.com/api/docs/bots
2. OpenAI Help Center — ChatGPT search. https://help.openai.com/en/articles/9237897-chatgpt-search
3. Perplexity — Perplexity Crawlers (PerplexityBot, Perplexity-User). https://docs.perplexity.ai/docs/resources/perplexity-crawlers
4. Perplexity Help Center — How does Perplexity work?. https://www.perplexity.ai/help-center/en/articles/10352895-how-does-perplexity-work
5. Google Search Central — AI features and your website. https://developers.google.com/search/docs/appearance/ai-features
6. Google Search Central — Optimizing your website for generative AI features on Google Search. https://developers.google.com/search/docs/fundamentals/ai-optimization-guide
7. Google Search Central — Google's common crawlers (Googlebot, Google-Extended). https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers
8. Google — AI in Search: Going beyond information to intelligence (May 20, 2025). https://blog.google/products/search/google-search-ai-mode-update/
9. Google Search Central — Latest documentation updates (FAQ rich result removal; spam policies and generative AI). https://developers.google.com/search/updates
10. Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020). https://arxiv.org/abs/2005.11401