Googlebot encountered 18 errors while attempting to access your robots.txt

Selected answers from the Dumb SEO Questions Facebook & G+ community.

• 
  Rand Wilson: he didn't even create the site with a robots.txt file
  Maybe that's the issue.. Maybe the poor little GBot is trying to find it, and cant?
• 
  Rotimi Orimoloye: So I guess I will have to go under the hood myself and create one then? What to do?
  ...I wouldn't wanna spoil anything! ;
• 
  Rand Wilson: I think all you need to do is create a new, blank TXT file, save it as ROBOTS.TXT, and upload it into the root of your website.
• 
  Kent Lodberg: Hi, you need to create it yourself. Most developers doesn't give a shit about seo and site optimization ( in my experience). Good thing for you is that it's easy to write, create and upload the robot.txt file :)
• 
  Ian Dixon: googlebot will seek out robots.txt but doesnt need it to be there so that is no the problem.
  Is it a wordpress site because the way I read things then that might put up a page that looks like robots.txt?
  Then crawl around the site to find if there are any files named robots.txt in any directory
• 
  Tony McCreath: The file does not have to exist, but when the bot tries to access it either has to exist or say it is missing (404 code) and not return an error code like 500. I suggest you use the GWT fetch as googlebot on it to see what gets returned.
• 
  Jim Munro: Would it be OK to share a (disabled) link, Rotimi so it can be tested for you?
  
  If not, have you tried "fetching as Googlebot" and what was the response?
• 
  Rotimi Orimoloye: ok will do the fetch.
  
  And btw +Jim Munro ; I don't know how to share a disabled link... yet. :)
• 
  Rand Wilson: this is a disabled link:  ; google. com (space after the . )
  (i think.. or did you mean something else +Jim Munro ;?)
• 
  Rotimi Orimoloye: I think he means the kind of link that hides my site's url from you guys. :)
• 
  Mark Steenbakkers: How to create a robots.txt
  https://support.google.com/webmasters/answer/156449?hl=en
  
  Did your web developer created a sitemap? This will help Google to better crawl your website. ;
  
  https://support.google.com/webmasters/answer/156184?hl=en ;
• 
  Rotimi Orimoloye: No! the developer didn't create a sitemap.
  
  But I did. About 2 months ago.
  And I tested it to see it was working.
  
  In fact, after I had created the site map and fixed some of the developer's links (where he had links pointing to .htmls intead of .phps, I went to rest thinking I had resolved the sites crawling problems.
  
  Until about 2 weeks later, it came up with this robots.txt issue for the first time. So I started to read up info online. There I read that it wasn't compulsory to have a robots txt file unless the site is getting larger and I need to exclude some pages like duplicate content and stuff like that. 
• 
  Mark Steenbakkers: I would create the robots.txt. It is easy enough and it will make makes the search engine robots life easier. It is not mandatory, but I think SEO is about making search engines understand your website better. To quote +Rand Wilson ; ; “Maybe that's the issue.. Maybe the poor little GBot is trying to find it, and cant?” So help the poor robots as much as you can.
  
  You can also add pages (pictures, PDFs, etc )  ;in the robots.txt that you don’t want to show up in the search results. For example if you have a “thank you” page, by excluding it in the robots.txt it will not show up in the search results. This can be handy on smaller websites ; because you have a change that the “thank you” page will appear in the search result as a site link. The links shown below your companies main URL in Google's search results. ;