Troubleshooting

Two different problems produce the same symptom - an empty Bot Traffic dashboard - and they have nothing to do with each other.

Symptom	Cause	Where to look
ChatGPT or Claude says it got a 403	something refuses AI crawlers before your site sees them	the reachability panel, below
Counts look far lower than your server log	a page cache answers without running your site’s code	page caching

AI assistants are blocked

The panel at the top of Bot Traffic fetches your site from SnowSEO with every published crawler user agent - ChatGPT-User, OAI-SearchBot, GPTBot, Claude-User, Claude-SearchBot, ClaudeBot, Perplexity-User, PerplexityBot - and compares each response with what a browser gets from the same place at the same moment. A difference is the finding, and the response headers usually say which layer produced it. This runs from outside because it has to. A block at Cloudflare or at your host happens above your site entirely; from the inside there is nothing to see.

Cloudflare

The most common cause, and the one with the most moving parts.

Security → Bots carries Search, Agent and Training categories. From 15 September 2026 Cloudflare blocks Training and Agent by default on ad-bearing pages for new zones and existing free zones, unless you opt out.
Bot Fight Mode (free plan) issues CPU challenges that no crawler can solve. It runs outside the Ruleset Engine, so a WAF Skip rule cannot bypass it - the only fix is turning it off, or moving to Super Bot Fight Mode on Pro.
AI Crawl Control / pay-per-crawl answers 402 at the edge. The request never reaches your origin.
A WAF custom rule matching http.user_agent will do it too. On Pro and above, add a Skip rule.

Multi-purpose crawlers such as Googlebot and Applebot are classified as both Search and Training, so a restrictive Training policy can take out search crawling with it.

Managed hosting

Host	Behaviour	Fix
WP Engine	Rate-limits ClaudeBot, GPTBot, Amazonbot and Bytespider at the platform level. Returns `429`.	Support ticket. This cannot be disabled per-bot from the customer Web Rules Engine.
SiteGround	Blocks training crawlers server-side by default; allows user-triggered ones.	Support.
Kinsta, Pantheon, Pressable, Flywheel	No platform-level block.	Nothing to do.

robots.txt

Each token is independent. Blocking ClaudeBot does not block Claude-User, and blocking GPTBot does not block ChatGPT-User. That asymmetry is why a site can believe it is open to assistants while blocking the exact agent a person’s question would use, or the reverse. The reachability panel evaluates every token against your live robots.txt and names the one that matched.

Security plugins and server WAFs

Imunify360 (common on cPanel shared hosting) ships ModSecurity rules blocking 16 AI bots - rule IDs 88345386-88345402. Ask your host to disable them.
Wordfence has no deliberate AI blocking, but its rate limiting and bad-bot heuristics throttle crawlers. There is no “allow well-known AI crawlers” toggle.
Blackhole for Bad Bots bans anything that follows its honeypot link, and AI crawlers are not on its default whitelist. Add them and clear the existing ban.
OWASP CRS blocks AI training bots by default while allowing Google and Bing.

Anything blocked at the edge leaves no trace in a plugin’s own log, because the plugin never ran.

Checking by hand

# Baseline
curl -sI -A "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36" https://example.com/

# Live assistant fetch - the one that matters most
curl -sI -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot" https://example.com/
curl -sI -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +Claude-User@anthropic.com)" https://example.com/

Anything other than a matching 200 is a finding. Watch for 429 (host rate limit) and 307 (redirect to a challenge) as well as 403. Run this from outside your own network. A request from an address your firewall trusts proves nothing about one from Anthropic’s or OpenAI’s.

What is genuinely unmeasurable

Not every gap is a misconfiguration.

Google AI Overviews and AI Mode do not fetch your page. They answer from the Search index that Googlebot already built, so there is no request to count and no user agent to see.
Answers from training data involve no request at all.
Cached assistant fetches. Anthropic’s web_fetch caches by default, so a repeated fetch of the same URL may never reach you.

These are limits of the medium, not of the integration.

Overview

Install

Self-hosting

Reference