Google is Dropping Support nofollow, and noindex in robots.txt
Google posted this morning they’re likely to stop unofficially behind the noindex, nofollow and crawl-delay directives inside robots.txt files. Google was saying to not do so for many years really and theorized that this was forthcoming very shortly and it is here.
Google composed”While open-sourcing that our parser library, we examined the use of robots.txt rules. Specifically, we concentrated on principles unsupported from the online draft, for example as crawl-delay, follow, and noindex. Since these principles were not recorded by Google, obviously, their use in regard to Googlebot is quite low. Digging further, we noticed that their use had been contradicted by other principles whatsoever but 0.001percent of most robots.txt files online. These errors hurt sites’ existence in Google’s search results in ways we do not believe webmasters “
They now do honor a few of those implementations, despite the fact that they are “jobless and unpublished rules” but may quit doing this on September 1, 2019.
Google may send notifications out through Google Search Console if you’re utilizing these unsupported commands on your robots.txt files.