Apple confirms existence of Applebot web crawler

Siri and Spotlight do not rely on Google or Bing, it seems.


Applebot has been revealed as Apple's web crawler, the first port of call for services on the firm's mobile devices.

In an updated support document first noted by Apple Insider, the iPad and iPhone maker says Applebot is the company's web crawler, and is used by products including the voice-activated assistant Siri and Spotlight Suggestions.

The web crawler, which scours website pages in order to create entries for a search engine, respects customary robots.txt rules and robots meta tags, according to Apple. In addition, the crawler originates from the net block.

The document says:

"User-agent strings will contain "Applebot" together with additional agent information. For example:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1)"

If robot instructions do not mention Applebot but instead refer to Googlebot -- Google's well-known web crawler -- the Apple robot will follow Googlebot instructions instead.

Developers are asked to contact the tech giant with questions or queries concerning the robot.

In related news, the Cupertino, Calif.-based firm recently launched a set of guidelines for would-be Watch band developers. The Made for Apple Watch program lays out mandatory guidelines, including the stipulation that devices be environmentally friendly.

Read on: Apple