Helping you block AI bots in Premium • Yoast

We are pleased to announce Yoast SEO 21.5, packed with valuable features to elevate your optimization efforts. This release introduces a highly requested option allowing you to block AI web crawlers. Now, you can effortlessly control access to your content and ensure it remains where you want it to be.

Blocking bots used to train AI systems (Premium)

Having the option to block an AI from accessing your content with a web crawler is essential. It helps prevent unauthorized scraping, protect your intellectual property, maintain data privacy, and control your valuable content.

Yoast SEO Premium now introduces a user-friendly feature that simplifies the process in response to the increasing demand for controlling AI robots. With a simple toggle switch, you can effortlessly block AI robots like GPTBot, CCBot, and Google-Extended.

This automated functionality seamlessly integrates with the robots.txt file by automatically adding the necessary lines to disallow access for these crawlers. This solution helps you efficiently protect your content from AI bots without the hassle of manual configuration or intricate technical adjustments.

Think before you do

It is essential to note that blocking or allowing AI bots remains a matter of free choice for each site owner. Every website is unique, with its specific considerations, objectives, and content requirements. Consequently, the decision to block or allow AI bots is one that you must carefully make for your websites.

By...