(Entry was posted by Ankit Patel on this post in the Dumb SEO Questions community on Facebook, 11/15/2016).

Prevent Google from Indexing Multisite Network Website

How to Prevent Google from Indexing Multisite Network Website? For example

Xyz.com/location1. (Prevent index)
Xyz.com/location1 (Needs to index)

In short , How I can apply robots.txt to Multisite network website?

This question begins at 00:12:22 into the clip. Did this video clip play correctly? Watch this question on YouTube commencing at 00:12:22

Share on Twitter

Share on Facebook

YOUR ANSWERS

Selected answers from the Dumb SEO Questions Facebook & G+ community.

Ankit Patel: How to Prevent Google from Indexing Multisite Network Website? For example

Xyz.com/location1. (Prevent index)
Xyz.com/location1 (Needs to index)

In short , How I can apply robots.txt to Multisite network website?
Jim Munro: Robots.txt can only prevent crawling, it has no bearing on indexing.

I think the most efficient way to achieve this is to a pply X-Robots-Tag: noindex as a header response to prevent indexing and also cause indexed pages to be dropped. It's more efficient because you can apply it to the whole directory and avoid having to apply the noindex tag to individual pages.

X-Robots-Tag: noindex

https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag