XML Sitemaps: The Complete Guide
An XML sitemap tells search engines exactly which pages exist on your site. Here is how to build one that works.
What is an XML sitemap?
An XML sitemap is a file that lists every URL on your site that you want search engines to know about. It is not a ranking factor — having one does not make you rank higher. But it is a discovery mechanism, and discovery is the first step to everything else.
When you need one
Every site benefits from a sitemap, but some need one more than others:
- Large sites (10,000+ pages) — crawlers have a crawl budget, and a sitemap helps them prioritize.
- New sites — no external links means no discovery path. The sitemap is often the only way Google finds your pages.
- Sites with orphan pages — pages not linked from anywhere else on the site.
- JavaScript-heavy sites — where rendered content may not be visible to a simple crawl.
What to include
Include every canonical, indexable URL. Exclude:
- Pages with noindex tags
- Non-canonical URLs (let the canonical version represent them)
- Redirect sources (they are not real pages)
- Paginated URLs beyond page 1 (use rel=next/prev instead)
- Admin, login, or internal-only pages
The lastmod field
Only set lastmod when the content actually changed. Many CMS platforms update lastmod on every deploy or every save, even when the content is identical. Google has said they ignore lastmod when it is unreliable. Make it accurate or leave it out.
Validation
Your sitemap should:
- Return a 200 with content-type application/xml
- Contain only canonical URLs that return 200
- Match what robots.txt declares
- Be reachable at /sitemap.xml or the location robots.txt points to
ColorGears validates all of this in one pass.
Related: How to Read a robots.txt File explains the other file crawlers check before they start.