(Entry was posted by Kath Dawson Courtney on this post in the Dumb SEO Questions community on Facebook, 07/06/2017).
SEMrush won`t crawl a site
SEMrush won`t crawl a site (for a site audit on a project) - why would this happen? I can`t see anything in the robots file that would block it.
I`ve seen Disallow: /
User-agent: SemrushBot on another site that stopped the PPC info being pulled into SEMrush but there is nothing on this other site.
SEMrush says you can add a line in robots to allow the crawl but could there be another issue causing the problem?
View original question
in the Dumb SEO Questions
community on Facebook, 07/06/2017).
