Cloudflare will block AI crawlers on pages with ads

Cloudflare has announced that, starting September 2026, it will block artificial intelligence crawlers by default on websites that use ads, unless the operators of those bots clearly separate their use for indexed search from data collection for training models. This measure responds to the growing concern of publishers over the uncompensated exploitation of their content. In an ecosystem where non-human traffic is already the majority, tools such as those offered by AI for businesses must be managed transparently. Cloudflare's decision forces giants like Google, Apple, and Microsoft to rethink their mixed crawlers, although they offer exclusion options such as Google-Extended or Applebot-Extended. For organizations developing custom applications or implementing AWS and Azure cloud services, this regulatory change reinforces the need for robust cybersecurity and granular control over access to their digital assets. Additionally, Cloudflare introduces a business intelligence dashboard that allows monitoring bot usage, as well as pay-per-use models that reward publishers when their content generates real value. In this context, solutions like Power BI and process automation become essential for interpreting this data and optimizing the digital strategy. Cloudflare's initiative marks a milestone toward a more sustainable internet, where AI agents and business intelligence services must operate under clear rules that balance innovation with the rights of content creators.