Google Sitemaps
One new free service for webmasters that Google created this year involved using XML-based sitemaps to keep the world’s biggest search engine informed as to when new pages exist on your site, as well as when previous pages have been updated. Basically, Google uses this sitemap feature to cut down on Googlebot’s time spent crawling pages that haven’t changed in weeks or months while at the same time getting to brand new and updated pages in a much more timely fashion.
Use of Google’s Sitemap service should become a standard item on any webmaster’s checklist. It’s easy to use, and there are plenty of free sitemap generators out there if you don’t want to set it up on your own hosting server (requires Python). The XML code is simple enough to just write out and update by hand, which is what I choose to do, and Google’s service will inform you of any errors or broken links in your code. Yahoo also has a sitemap service, but it is rather simple and featureless. You basically can submit a TXT file listing all the pages on your site. I use both, but as far as I’m concerned, Google Sitemaps is a must while Yahoo’s service is pretty much optional.