> ## Documentation Index
> Fetch the complete documentation index at: https://snowseo.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# AI Crawler Tracking
> See which AI crawlers and assistants read your pages, how often, and whether they are who they claim to be.
The **Bot Traffic** dashboard shows every request an AI crawler or assistant made to your site: which company, which page, whether it was a bulk training sweep or an assistant answering a real person right now, and whether the request came from an IP that provider actually owns.
Getting data into it takes one install on your site. Pick your platform in [Setup](#setup) below.
***
## The one thing to understand first
**AI crawlers do not run JavaScript.** GPTBot, ClaudeBot, PerplexityBot and the coding agents that fetch your pages issue a single HTTP request and read the bytes. They never build a page, never run a script, never load an image.
That has one consequence, and it decides everything else: a tracking snippet pasted into your page records **zero** crawler visits. Not fewer, not less accurate - none at all. This is not a limitation we are working around; it is what a crawler is.
So tracking has to happen on the server that answers the request. That is why setup means a WordPress plugin, an npm package or a one-line PHP config rather than a tag in your ``.
***
## What gets recorded
For a request that looks like it came from an AI client, your site sends:
| | |
| ------------------------------ | ---------------------------------------------------------- |
| The path requested | `/pricing`, `/docs/getting-started` |
| The `User-Agent` header | `GPTBot/1.2`, `axios/1.8.4` |
| The IP address | Used to check the crawler is who it says, then hashed |
| The `Referer` header | Where the request said it came from |
| The response status | So a crawler hitting a 404 is visible |
| Whether Markdown was requested | The strongest signal of an AI client |
| Any signature headers | Cryptographic proof of identity, when the crawler sends it |
**Requests from ordinary browsers are never sent.** The check that decides runs before anything else, so a normal page view costs your site nothing measurable.
IP addresses are hashed on arrival and never stored in raw form. They exist only long enough to answer "does this address belong to OpenAI?"
***
## How a crawler gets verified
Anyone can put `GPTBot` in a User-Agent header. SnowSEO checks three independent things, in order of strength:
Some crawlers now sign their requests with a private key published under a domain they control. This is proof, not a claim, and it survives a crawler egressing through a proxy pool.
The address is checked against the CIDR ranges each provider publishes, refreshed daily. For providers that publish no ranges, a reverse-DNS lookup is confirmed in both directions.
Matched against documented crawler tokens only, never a vendor name appearing anywhere in the string. On its own this is the weakest signal, which is why an unverifiable request is labelled **Unverified** rather than trusted.
A request claiming a provider from an address that provider does not own is marked **Spoof suspected**.
***
## Setup
Our plugin. No code, and the key is fetched for you.
Middleware, a few lines.
One middleware, plus a couple of lines per framework.
Laravel, Drupal, Craft or custom. One config line.
Not listed? See [what can and cannot be measured](/docs/beacon/platform-support).
***
## Getting the site key
Every install needs a site key, which tells SnowSEO which brand the traffic belongs to.
Open **Settings → Integrations → Beacon** and generate one. It is shown once, so copy it then.
On WordPress you can skip this. If the site is already connected to SnowSEO for publishing, the plugin fetches its own key - open **SnowSEO → AI Crawlers** in wp-admin and click **Get site key from SnowSEO**.
One key per brand. Rotating it invalidates the old one immediately, so anything still using it stops reporting until you update it.
***
## You do not have to send this to us
Both halves of beacon - the client that installs on your site and the server that receives from it - are [MIT-licensed and open source](https://github.com/Snow-SEO/beacon).
Point the endpoint at a collector you run and the hits go there instead, classified and verified by the same code this dashboard uses. You give up the dashboard, the maintained crawler dataset and attribution; you keep the data.
Run your own collector, or write one against the frozen wire protocol.
***
## Why your numbers may look low
Two honest caveats, both worth knowing before you compare against a server log.
**Page caches hide traffic.** If a cache answers a request from a stored copy without running your site's code, nothing can observe that request. On WordPress the plugin ships a coverage check that measures exactly how much this affects you. A CDN cache in front of your origin has the same effect on every platform.
**Some AI usage produces no request at all.** Google's AI Overviews and AI Mode answer from the Search index, so your server is never contacted. Neither is it when a model repeats something it learned during training, possibly years ago. Beacon measures crawls, and crawls are not the whole of AI visibility - which is what the [AI Visibility](/docs/user-guide/analysis/ai-visibility) dashboard is for.