Reported 6 May 2015
Apple releases Applebot, it`s new webcrawler for siri and spotlight.
"Underpinning those results is a new in-house Web crawler called AppleBot. AppleBot allows Apple to index the web and deliver search results without depending on third parties, which could spell trouble for mainstream search engines like Google."
Additional information can also be found on https://support.apple.com/en-us/HT204683