Skip to main content
Use this when your site is behind Cloudflare and traffic is cached at the edge. Workers run ahead of the cache, so this sees every request - including the ones your origin never hears about, which no server-side integration can observe.
Cloudflare APO is the worst case for measurement and the best case for this Worker: APO bypasses its cache for requests carrying WordPress cookies, and crawlers carry none, so cookieless AI crawlers are the most reliably cached segment on the whole site.
This is additive. Keep your plugin or SDK installed - the Worker reports the same hits, and SnowSEO deduplicates on ingest.

​
Setup

1

Get a site key

In SnowSEO, open Settings → Integrations → Beacon and generate a site key. It looks like sb_live_....
2

Create the Worker

In the Cloudflare dashboard, go to Workers & Pages → Create → Worker, then Deploy and Edit code. Paste the snippet from the Bot Traffic dashboard - it comes with your key already filled in.
3

Route it at your zone

Under the Worker’s Settings → Domains & Routes, add a route for example.com/*.
The Worker never changes a response. It passes the request through untouched and reports who asked, using ctx.waitUntil so reporting happens after the visitor already has their page.

​
What it reports

The same fields every other client sends: path, user agent, status, whether Markdown was requested, source IP, and any Web Bot Auth signature headers, which SnowSEO verifies against the operator’s published key directory. It filters before sending. Ordinary browser traffic never leaves the edge - only requests that asked for Markdown, arrived without a browser engine, or name themselves as automation. Classification happens on SnowSEO’s side, so the Worker never needs updating when a new crawler appears.

​
Limits

  • The free Workers tier allows 100,000 requests a day. The Worker runs on every request to your zone, not just crawler ones.
  • One report per qualifying hit. Very high crawler volume can reach the ingest limit of 600 requests a minute per site key.
  • A Worker is per zone. A site on several Cloudflare zones needs a route on each.