When marketing leaders ask whether their technical SEO covers them for AI search, Michael's answer is mostly yes. Google's AI Overviews and AI Mode run on Google's regular index, and most of a standard technical audit carries over to the other platforms. This week he sorts technical work into what to keep, what to start, and the short list of things to skip, including a training-bot decision that removes your own description of your company from what the next model learns.

In this episode

* Why indexability now means two things: a search index and a model's training data

* How OpenAI and Anthropic split their crawlers by job, and why blocking the training bot still costs you in answers that don't involve a search

* Which index each platform searches: Google's for AI Overviews and AI Mode, a mix led by OpenAI's own index for ChatGPT, Brave for Claude, and Bing for Copilot

* Two Google details teams miss: Google-Extended doesn't affect AI Overviews, and old nosnippet tags can keep pages out of them

* The six areas of a technical audit that carry over, and why schema facts also belong in visible text

* Why every important page should make sense as the only page an assistant reads

* How to block training narrowly by folder, and why robots.txt is a request your server may not enforce

* Why Michael recommends skipping LLMs.txt for most sites

* What could change: less live search, paid web search, agents that read pages visually, and agentic checkout

* The order to work through it with your team

Chapters

[00:00:00] Intro

[00:02:26] This week in search: Google's September 2026 spam update, Cloudflare's AI crawler defaults, Google's Web Search Service API docs disappear

[00:05:33] Does your technical SEO cover AI search?

[00:06:04] Keep, start, stop

[00:07:33] Crawlability and the two kinds of indexability

[00:09:31] Separate bots for training and search

[00:10:32] AI Overviews and AI Mode: Google-Extended and snippet controls

[00:12:11] ChatGPT's own search index

[00:14:13] Claude, Brave, and Copilot

[00:15:04] What to do for each platform

[00:17:00] Keep: the six areas of a technical audit

[00:17:54] Site architecture and internal linking

[00:20:16] What your server logs can tell you

[00:20:42] Rendering and page speed

[00:22:00] Structured data

[00:23:47] Start: make every page make sense on its own

[00:25:42] Start: make crawler decisions on purpose

[00:28:40] Blocking training narrowly

[00:29:12] Robots.txt is a request

[00:30:58] Stop: LLMs.txt

[00:33:46] Every black hat tactic was once white hat

[00:34:38] What could change

[00:39:13] Where to start with your team

Find all the resources from this episode on our website.

Sources

Search Engine Journal, "Google Rolls Out September 2026 Spam Update Globally," Sept. 2026, searchenginejournal.com/google-september-2026-spam-update/590828/

The Next Web, "Cloudflare gives AI crawlers a September deadline: pay publishers or get blocked," July 2, 2026, thenextweb.com/news/cloudflare-block-ai-crawlers-pay-publishers

Google Search Central, "AI features and your website," developers.google.com/search/docs/appearance/ai-features

OpenAI, "Overview of OpenAI Crawlers," platform.openai.com/docs/bots

Peec AI, "ChatGPT built its own search index," Sept. 2026, peec.ai/blog/chatgpt-built-its-own-search-index

HasData, "AI Crawler Block Index," hasdata.com/blog/ai-crawler-block-index

Forrester, "What It Means That The Leader In 'Agentic Commerce' Just Pulled Back," March 2026, forrester.com/blogs/what-it-means-that-the-leader-in-agentic-commerce-just-pulled-back