Today's Edition
MAITHEM AL ANBARI (ABU DHABI)
Internet company Cloudflare has launched a new “pay-per-crawl” system, which allows website owners to charge AI crawlers whenever they seek access to their content.
Pay-per-crawl will give domain owners complete control over their monetisation strategies, Cloudflare said. They can establish a consistent fee per request for their entire site.
By default, this new system will prevent artificial intelligence crawlers from accessing content without the consent or financial remuneration of website owners, the company explained.
Cloudflare mentioned that the request headers will indicate whether access is granted or if a payment is required, along with the associated pricing. The company will act as the Merchant of Record for the pay-per-crawl system and will also provide the necessary technical infrastructure.
On average, Cloudflare handles 78 million HTTP requests per second and provides data services for 330 cities across more than 125 countries.
Approximately 16% of worldwide internet traffic is processed directly by Cloudflare’s content delivery network, according to the company's estimates.
