Why Google Indexes Blocked Web Pages – Search Engine Journal
Google’s John Mueller explains why disallowed pages are sometimes indexed and that related Search Console reports can be dismissed
Google’s John Mueller answered a question about why Google indexes pages that are disallowed from crawling by robots.txt and why the it’s safe to ignore the related Search Console reports about those crawls.
Bot Traffic To Query Parameter URLs
The person asking the question documented that bots were creating links to non-existent query parameter URLs (?q=xyz) to pages with noindex meta tags that are also blocked in robots.txt. What prompted the question is that Google is crawling the links to those pages, getting blocked by robots.txt (without seeing a noindex robots meta tag) then getting reported in Google Search Console as “Indexed, though blocked by robots.txt.”
The person asked the following question:
“But here’s the big question: why would Google index pages when they can’t even see the content? What’s the…
The post Why Google Indexes Blocked Web Pages – Search Engine Journal first appeared on One SEO Company News.
source: https://news.oneseocompany.com/2024/09/06/why-google-indexes-blocked-web-pages-search-engine-journal_2024090658262.html
Your content is great. However, if any of the content contained herein violates any rights of yours, including those of copyright, please contact us immediately by e-mail at media[@]kissrpr.com.