Dumb SEO Questions

(Entry was posted by Athanasios Giannias on this post in the Dumb SEO Questions community on Facebook, 07/22/2015).

Blocked by robots.txt

Hi everyone.

I have a beta site which is mainly used to experiment. It is blocked by robots.txt. Within GWT, shouldnt the the index status bar be zero?

Thanks everyone!
This question begins at 01:04:52 into the clip. Did this video clip play correctly? Watch this question on YouTube commencing at 01:04:52
Video would not load
I see YouTube error message
I see static
Video clip did not start at this question

YOUR ANSWERS

Selected answers from the Dumb SEO Questions Facebook & G+ community.

  • Athanasios Giannias: Hi everyone.

    I have a beta site which is mainly used to experiment. It is blocked by robots.txt. Within GWT, shouldnt the the index status bar be zero?

    Thanks everyone!
  • Federico Sasso: a site: search would find anything?
    is the site reachable through another host name (e.g. non-www) non-blocked?
  • Athanasios Giannias: hi +Federico Sasso ;sorry for the delay. Yes my site is reachable with the site:search.  ;But when using the fetch as google within GWT,  ;" Blocked
    robots.txt Tester " appears. ;
    My beta site is:  ;  ;http://beta(dot)softmakers(dot)gr/

    Thank you ;
  • Federico Sasso: +Athanasios Giannias ;not all the pages are indexed, they mostly look deep link. My guess is somehow for a while the beta site was not robots.txt blocked and got fully indexed or so, not that it's robots.txt blocked Google is slowly de-indexing its pages.
  • Athanasios Giannias: +Federico Sasso ; so i should wait until all the pages are de-indexed? ;
    I guess that when this happens the index status bar within GWT be zero?
  • Federico Sasso: Yes, that's what I think. You could of course use the URL removal tool, but it only works one URL at the time.
  • Athanasios Giannias: I will wait for a while - probably will take weeks - till its totally dropped from search. ;
    Thank you +Federico Sasso ;
  • Daniel Mullins: Hi +Athanasios Giannias​ I would suggest adding the meta=noindex tag to all the pages in ur beta site. 

View original question in the Dumb SEO Questions community on Facebook, 07/22/2015).