(Entry was posted by Dixon Jones on this post in the Dumb SEO Questions community on Facebook, 08/01/2019).

Disallowing a directory via robots.txt

Hmm... Does this work? Disallowing a directory and then allowing a subdirectory? and anyway, should I want Googlebot looking at that ajax thing:

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

This question begins at 00:26:01 into the clip. Did this video clip play correctly? Watch this question on YouTube commencing at 00:26:01

Share on Twitter

Share on Facebook

YOUR ANSWERS

Selected answers from the Dumb SEO Questions Facebook & G+ community.

Ryan Jones: I donâ€™t think google needs to see that file. But yes it will work
Gautam: I think yes and yes.
Michael Martinez: If you`re running on Apache then use "options -indexes" to prevent browsing or crawling of folders lacking an "index.???" file. Anything that is linked to within the folders can still be fetched.