From the article:
Crawl Optimization Checklist
Here’s a quick crawl optimization checklist to get you started.
Track and Monitor Googlebot
I don’t care how you do it but you need this type of visibility to make any inroads into crawl optimization. Information is power. Learn to grep, perfect your RegEx. Be a collaborative partner with your technical team to turn this into an automated daily process.
Manage URL Parameters
Yes, it’s confusing. You will probably make some mistakes. But that shouldn’t stop you from using this feature and changing Googlebot’s diet.
Use Robots.txt Wisely
Stop feeding Googlebot empty calories. Use robots.txt to keep Googlebot focused and remember to make use of pattern matching.
Don’t Forget HTML Sitemap(s)
Seriously. I know human users might not be using these, but Googlebot is a different type of user with slightly different needs.
Optimize Your Internal Link Structure
Whether you try depagination to flatten your architecture, re-evaluate navigation menus, or play around with crosslink modules, find ways to optimize your internal link structure to get those low PageRank pages crawled more frequently.
via +Max Minzer
