Cloudflare Tightens Rules for AI Crawlers, Strengthening Publishers

I’d like to share an update from the world of AI and digital media: Cloudflare is changing its access policy for AI crawlers.

Starting September 15, 2026, mixed-use crawlers will be blocked by default on ad-supported sites unless the owner changes the settings.

Cloudflare is now placing greater emphasis on separating search, agentic, and training use cases.

It is also expanding its Pay Per Use model, enabling publishers to be compensated not only for crawling, but also for the value their content delivers to AI services.

Why this matters: the market is gradually moving toward a more transparent and fairer content model.

Do you think this could become a new industry standard?
#AI #Cloudflare #publishers #HackerNews