Robots.txt query.

Selected answers from the Dumb SEO Questions Facebook & G+ community.

• 
  The Law Office of Kenneth M Mollins P.C.: Hello all. Advice please!
  
  When I do a fetch and render query in Google webmaster tools it shows that the Google spiders are not properly reading my website. I put a screen shot below. I know that I need to change my robot.txt settings but am unsure specifically what to do. My robot.txt settings are currently;
  
  User-agent: *
  Disallow: /wp-admin/
  Disallow: /wp-includes/
  Disallow: */xmlrpc.php
  Disallow: /wp-.php
  Disallow: */trackback/
  Disallow: *?wptheme=
  Disallow: *?comments=
  Disallow: *?replytocom
  Disallow: */comment-page-
  Disallow: *?s=
  Disallow: */wp-content/
  Allow: */wp-content/uploads/
  
  Thoughts?
• 
  Pau Lica: There are a lot of things that I would change to your robots.txt, so I'll just give you an example:
  
  Sitemap: http://example.com/sitemap.xml
  
  User-agent: *
  Disallow: /cgi-bin/
  Disallow: /wp-admin/
  Disallow: /wp-includes/
  Disallow: /wp-content/plugins/
  Disallow: /wp-content/cache/
  Disallow: *?replytocom
  Disallow: /trackback/
  Disallow: /feed/
  Disallow: /comments/
  Disallow: */trackback/
  Disallow: */feed/
  Disallow: */comments/
  Disallow: /*?
  Allow: /wp-content/uploads/
• 
  John Britsios: First to your rendering question. Try removing these lines in your robots.txt and test if it renders with the GWT rendering test feature:
  
  Disallow: *?wptheme=
  Disallow: */wp-content/
  
  Second, using robots.txt should be your last chance if you are trying to block site sections and/or pages.
  
  For example with your robots.txt you expose information about your site to hackers which should be avoided.
  
  There are also PageRank related issues attached to that, but this requires an own discussion.
  
  Therefore, the best way to prevent indexing of individual pages is implementing the X-Robots or Meta Robots directives, with "noindex,noarchive,nosnippet,follow".
  
  I like very much the plugin UltimateSEO which is a great aid for this task, but if I recall it does not give the option to add 4 directives, so I would at least use "noindex,follow".
  
  Hope all above help.
• 
  Edwin Jonk: From the expert panel in this weeks SEO Questions hangout on air on 00:41:21 into the YouTube video: https://dumbseoquestions.com/q/robotstxt_query +The Law Office of Kenneth M Mollins P.C.
  
  The cuetimes listed on this page are functional while the live broadcast continues and also once Youtube has finished processing the public broadcast. Processing time varies between 6-24 hours. During this period the clip is viewable in full but the cuetimes cannot be relied on.
  
  If our assistance with this issue was useful to you, please consider sharing your success story so that others might benefit.