A sitemap is an often overlooked file that helps modern search engines index and understand websites better. Google was the first to introduce sitemaps with the Google XML sitemap format in 2005. A little more than a year later, Google abandoned his proprietary sitemap format and joined other search companies to create an XML sitemap standard. This new standard has replaced the previous Google XML standard and is used now by Google, Bing, Yahoo, and other search companies. As the web evolves, the standard evolves with it and the search engines look toward the standard for guidance on how best to complete their website indexing and website crawling.

Essentially an XML sitemap is just an XML file placed in a directory of a website that contains URLs and some information about those URLs. A website can have multiple sitemaps placed in multiple directories. To help search engines discover the various sitemaps an organization might have, the locations of the XML file are listed at the bottom of a website’s robots.txt file.

Read the rest of this entry »