Cloudflare introduces new tool to safeguard websites from AI-Powered bots

Cloudflare is responding to customer dissatisfaction with AI bots by launching a new tool, available free of charge to all its clients, including those on free plans. ByteDance and OpenAI are among the top four identified AI bot crawlers. Cloudflare has introduced a feature to prevent AI bots from scraping content for model training without permission. The company emphasizes ongoing updates to the tool, which uses digital fingerprints to detect and distinguish bots from legitimate users. In an official statement, Cloudflare acknowledges customer concerns and announces a new one-click feature to block all AI bots from accessing websites, particularly those engaging in unauthorized scraping.

Cloudflare identifies leading AI bot crawlers through new blocking tool launch

Cloudflare has released data alongside the introduction of their new tool designed to block all AI bots. According to internal statistics, 85.2% of customers actively choose to block scrapers, even those identified as legitimate. ByteDance's ByteSpider bot, affiliated with TikTok, was highlighted for attempting to access 40% of websites monitored by Cloudflare, while OpenAI's ChatGPT was active on 35%. Amazonbot and Claudebot also rank among the top four AI bot crawlers based on requests across Cloudflare's network.

Addressing the ongoing challenge posed by AI advancements and their voracious content consumption, Cloudflare emphasizes the necessity for continual evolution in combating these issues. The new tool is universally available to users and can be accessed via the Security > Bots menu, enabling the blocking of "AI Scrapers and Crawlers."