Translate

Thursday, January 24, 2013

What is a Google Spider?


The Google Spider is the bot (robot) that crawls the web officially referred to as Googlebot. Crawling is the action of scanning the world wide web for updated pages to add to the index. The Spider uses an algorithm to determine what sites and how many pages to crawl, and how often. The process starts with a list of urls from previous crawls, and compounded with the a html and xml sitemap data provided by a webmaster if you would like to learn how to do this feel free to contact a local SEO expert like ROI Vault your local Digital Marketing Agency