Building BobSmithPhotography.net has stopped expanding because I've been overwhelmed with spam on the comment forums. So, if you're here and have questions about web design come and visit Forums, Blogs, Wikis dot com. It has articles that I've written, some of which are also here, and a web design forum as well so you can ask any questions you may have. There is still a lot of good stuff here though, so poke around the links and take a look.

Home » Glossary » DeepBot & FreshBot

DeepBot & FreshBot

Google runs several indexing robots that search the web and find sites.   The main robot, GoogleBot, is split into two different variants - DeepBot and FreshBot.

The DeepBot is the one that will crawl many pages on your site.  The idea is that DeepBot will go through all your pages adding news ones that it finds.

FreshBot is intended more to find pages that have changed. 

How to tell apart
Both DeepBot and FreshBot come with the user-agent 'Googlebot/2.1 (+http://www.googlebot.com/bot.html)' so you can't tell simply by the user-agent.  But, they do come from different IP blocks.  DeepBot comes from IP addresses starting with 216.* and FreshBot comes from IP addresses starting with 64.*. 

Content managed by the Etomite Content Management System.