Dumb SEO Questions

(Entry was posted by Jobin John on this post in the Dumb SEO Questions community on Facebook, Thursday, February 15, 2018).

I want to hide my backlinks from competitors

I want to hide my backlinks from competitors so I added the following to Robots.txt, will this get the job done or do you think something better can be done? My target is to block ahrefs & semrush and other tools and then allow Google, Yahoo & Bing bots hope this does the trick.  

User-agent: MJ12bot
Disallow: /
User-agent: SemrushBot
Disallow: /
User-agent: SemrushBot-SA
Disallow: /
User-agent: rogerbot
Disallow:/
User-agent: dotbot
Disallow:/
User-agent: AhrefsBot
Disallow: /
User-agent: Alexibot
Disallow: /
User-agent: SurveyBot
Disallow: /
User-agent: Xenu’s
Disallow: /
User-agent: Xenu’s Link Sleuth 1.1c
Disallow: /

This question begins at 00:26:16 into the clip. Did this video clip play correctly? Watch this question on YouTube commencing at 00:26:16
Video would not load
I see YouTube error message
I see static
Video clip did not start at this question

YOUR ANSWERS

Selected answers from the Dumb SEO Questions G+ community.

  • Neil Cheesman: This is probably like trying to stop the tide from coming in...
  • Michael Martinez: Your backlinks are on other sites, not on the site they are pointing to. Unless you control the other sites you cannot hide the links from competitors. Blocking link crawlers from YOUR site is a good idea, though, because they take your information and sell it to other people without compensating you. Also, their crawlers use your server resources without compensating you. You owe them nothing. Unfortunately, most link crawlers ignore robots meta directives. There are only a few in your list I would trust to honor the block. And any crawler that allows people to change the user-agent (like Xenu) will only honor the robots directives for the currently active user-agent. A lot of people change the user-agent to "googlebot". If you really want to block rogue crawlers you have to dig into your server log files and build a detailed firewall list. But you risk blocking legitimate users and email if you are too aggressive with your firewall.
  • Dan Thies: No, this won’t help.
  • Tyron Love: The only reason you’d want to do this is if you were trying to hide a PBN.
  • Jenny Halasz: My question is why? This is a lot of work for no benefit that I can see.
  • Rob Woods: Why? Do things that attract links they can’t replicate. Then you don’t have to worry about them seeing your links. Think you’ve come up with some “tactic” they don’t know about? You haven’t.
  • Rishi Lakhani: That list is from Craigs blog - its correct and a way to block all link crawlers to answer your initial question.

View original question in the Dumb SEO Questions community on G+, Thursday, February 15, 2018).

Reference Links