(Entry was posted by Neeraj Pandey on this post in the Dumb SEO Questions community on Facebook, 01/20/2017).
Blocked by robots notificatio
Hi,
I saw today some not expected behavior- I blocked all the bots with robots.txt file and selected html meta, pasted that in head section and clicked for verification. Google verified so does it mean google will crawl the page despite blocking by robots.txt? I was expecting the message like notification that my website is blocked by robots and that it should be allowed
View original question
in the Dumb SEO Questions
community on Facebook, 01/20/2017).