Skip to content

Latest news

Eleutheros Ltd
Aerial view of a data center roof lined with cooling units at dusk

Artificial intelligence

Google asks sites to allow Google-Agent through CDN and WAF

Google published a guide on 17 November 2026 explaining how to let its AI agents through CDNs, web application firewalls and bot-detection services. Requests are identified by a user-agent and by addresses the report does not detail.

Abondance reported on 19 November 2026 that Google published, two days earlier, a guide titled « Allow Google-Agent traffic » describing how sites should let its AI agents through content delivery networks, web application firewalls (WAF) and bot-detection services. According to the report, Google-Agent requests can be recognised by their user-agent and by addresses it does not enumerate.

That is the whole of what the announcement makes public so far. The report does not specify which products the agents power, what volume of traffic is involved, whether any crawling or retrieval behaviour is attached, or what Google recommends when a site chooses to refuse. Those gaps matter before any configuration change is made.

Why bot defences are the new gate

For a decade, the practical question around crawlers was robots.txt and server logs. Bot management has since moved up the stack: most serious sites now sit behind a CDN and a WAF that score every request, and automated traffic from cloud ranges is routinely challenged or blocked. Legitimate infrastructure and hostile scraping look alike at the network layer.

AI agents make that ambiguity sharper. When an automated system fetches a page to answer a user question, the publisher gets a request but not necessarily a visit: no session, no ad impression, no newsletter sign-up. Allowing agent traffic and monetising it are two different problems, and the Google guide addresses only the first.

What to check this week

Three verifications are worth running before touching anything, because they produce facts rather than opinions.

  • Pull WAF and CDN logs for the last 30 days and count requests matching the user-agent pattern described in the guide, along with the share currently challenged or blocked.
  • Check whether those requests hit pages that carry your revenue: product pages, paywalled articles, listing pages. An agent reading a help page costs nothing; an agent reading your entire catalogue is a different conversation.
  • Confirm who owns bot-management rules. A silent allow-list added by an infrastructure team can quietly widen access for every automated client that reuses the same signature.

None of this requires opening the firewall. It requires knowing what is already arriving.

The strategic question behind the request

Publishers and e-commerce teams should read the guide as a positioning move as much as a technical note. A large platform asking the open web to admit its agents is asking for a distribution channel, and the terms of that channel are being set now, before most sites have measured the traffic.

The lever available to a site is not only block or allow. Rate limits, path-level rules, separate handling for content that carries commercial value, and logging that distinguishes agent requests from human sessions are all available today. The report does not say whether Google offers or endorses any of these, and it does not describe verification mechanisms beyond the identifiers mentioned.

A reasonable posture for this quarter: measure first, document the decision, and revisit when Google publishes more detail. Refusing outright and accepting everything are both defensible; doing either without logs is not.

Eleutheros

Also worth reading

All news

Aerial night view of Alexander Nevsky Cathedral in central Sofia

Contact

Tell us where the project stands.

A few lines are enough: what you are building, where it is today, and what is blocking it. We answer every message personally.

Contact us contact@eleutheros.eu