Sponsor of the Day:
Jerkmate
https://www.screamingfrog.co.uk/seo-spider/issues/directives/
Issues - Directives | Screaming Frog
Robots directives using either the robots meta tag or X-Robots-Tag HTTP Header give search engine crawlers instructions about how to crawl and index web pages....
issues directivesscreaming frog
https://www.screamingfrog.co.uk/seo-spider/issues/directives/noydir/
Issues - Directives : NoYDIR | Screaming Frog
Feb 23, 2024 - URLs containing a ‘noydir’ directive in either a robots meta tag or X-Robots-Tag in the HTTP header. This is an old meta tag that used to instruct Google not...
issues directivesscreaming frog
https://www.screamingfrog.co.uk/seo-spider/issues/directives/nosnippet/
Issues - Directives : NoSnippet | Screaming Frog
Feb 23, 2024 - URLs containing a ‘nosnippet’ directive in either a robots meta tag or X-Robots-Tag in the HTTP header. This instructs Google not to show a text snippet or...
issues directivesscreaming frog
https://www.screamingfrog.co.uk/seo-spider/issues/directives/noimageindex/
Issues - Directives : NoImageIndex | Screaming Frog
Feb 23, 2024 - URLs containing a ‘noimageindex’ directive in either a robots meta tag or X-Robots-Tag in the HTTP header. This instructs Google not to show the page as the...
issues directivesscreaming frog
https://www.screamingfrog.co.uk/seo-spider/issues/directives/none/
Issues - Directives : None | Screaming Frog
Feb 23, 2024 - URLs containing a ‘none’ directive in either a robots meta tag or X-Robots-Tag in the HTTP header. The ‘none’ directive is the equivalent to ‘noindex,...
issues directivesscreaming frognone
https://www.screamingfrog.co.uk/seo-spider/issues/directives/notranslate/
Issues - Directives : NoTranslate | Screaming Frog
Feb 23, 2024 - URLs containing a ‘notranslate’ directive in either a robots meta tag or X-Robots-Tag in the HTTP header. This instructs Google to not offer a translation of...
issues directivesscreaming frog
https://www.screamingfrog.co.uk/seo-spider/issues/directives/nofollow/
Issues - Directives : Nofollow | Screaming Frog
Feb 23, 2024 - URLs containing a ‘nofollow’ directive in either a robots meta tag or X-Robots-Tag in the HTTP header. This is a ‘hint’ which tells the search engines not to...
issues directivesscreaming frognofollow