Is Cloudflare’s AI Bot Defense Secretly Sabotaging Your Site’s Google Indexing?

Ever find yourself caught in one of those “wait, what just happened?” moments with your website? Well, that’s exactly the pickle site owners are dialing up after a Redditor flagged that blocking AI training bots through Cloudflare might be accidentally throwing Googlebot and Bingbot under the bus too — yep, those very same bots you depend on to get your site indexed and ranking. Now, Cloudflare’s set to officially roll out these new blocking rules come September 15, but it looks like some are feeling the pinch a bit early. So, what’s a savvy site owner to do when your AI training bot defense starts turning away the good guys? It’s like setting a trap and catching your own courier! Strap in, because this tale of mixed-usage bots and sneaky 403 errors might just change how you configure your defenses — or at least make you double-check those Bot Fight Mode settings. Ready to dive deeper? LEARN MORE

There’s a report that blocking AI training bots with Cloudflare is also blocking legitimate Googlebot and Bingbot from indexing a website. Cloudflare recently announced that this would happen beginning in September. It’s unclear whether this is a fluke or user error but it does serve as a reminder to be aware of the pending change and to adjust settings accordingly.

Redditor Claims Cloudflare Blocked Googlebot

A Redditor in the r/SEO community asked if anyone else has experienced Cloudflare blocking Googlebot as an AI trainer.

They asked:

“Has anyone else run into Cloudflare AI Crawlers blocking Googlebot/Bingbot?
I’m testing Cloudflare’s new AI Crawlers & Scrapers feature and noticed something unexpected.

When I set AI Training = Block, both Googlebot and Bingbot start receiving HTTP 403 responses when trying to fetch my sitemap. As soon as I disable the AI Training block, the sitemap is accessible again.

From what I understand, Cloudflare now classifies Googlebot and Bingbot as “Search + Training” bots, so enabling the training block appears to affect them as well.

This leaves me with a dilemma:

• Allow AI Training so search engines can access my sitemap.
• Block AI Training and risk Google/Bing getting 403s.

Has anyone found a clean workaround?”

Google’s John Mueller responded to ask for a DM to take a closer look. Another Redditor asked if maybe Cloudflare was blocking rogue bots that were spoofing Googlebot’s user agent.

The original poster replied that wasn’t the case and offered more information:

“I understand that Cloudflare blocks fake Googlebots, but in my case I’m referring to what Cloudflare itself shows in the dashboard.

When I enable Bot Fight Mode, uploading my sitemap returns a 403 error. As soon as I disable it, the 403 disappears.

Also, in the AI Crawlers section, Cloudflare shows Googlebot and Bingbot as blocked automatically. That’s why I’m wondering if Bot Fight Mode is interfering with legitimate crawlers or if the dashboard is reporting something else.”

Cloudflare’s Official Documentation

In it’s official announcement states that beginning September 15 Cloudflare will begin blocking mixed-usage bots, which are bots like Googlebot that simultaneously index websites for search and collect data AI training. The effect is that Cloudflare will block Googlebot.

Cloudflare’s announcement explains:

“On September 15, 2026, Cloudflare will set updated defaults for new domains: bots classified as Training or as Agent will be blocked on pages that display ads, and Search will remain allowed.

Mixed-purpose crawlers that combine Search and Training will also be blocked by all configurations to block AI training, including the legacy “Block AI bots” option. Before September 15, all customers can opt out of these new defaults.”

That kind of blocking is supposed to go into effect in September so it’s curious that someone is reporting that this is happening right now.

Featured Image by Shutterstock/Erlin Diah