XML sitemap
Also called sitemap.xml, sitemap index
A machine-readable list of the URLs a site wants crawled, with optional last-modified dates. It is a hint about discovery and freshness, not an instruction to index, and a URL's presence in one guarantees nothing.
A sitemap answers one question for a crawler: which URLs does this site
consider worth knowing about, and when did each last change. It does not
request indexation, it does not override a noindex, and it does not outrank a
canonical that points elsewhere.
lastmod is the field worth getting right, and the one most often wrong. A
sitemap that stamps every URL with the build timestamp is telling a crawler
that the entire site changed on every deploy. Crawlers learn to disregard a
lastmod that behaves that way, and once disregarded it is not selectively
re-trusted for the pages where it was honest. A date should be absent rather
than invented — this site’s own sitemap gives a lastmod only to the pages
that carry a real one in their front matter.
Listing everything is the wrong instinct. A sitemap is a statement about
what matters. Machine surfaces, duplicate representations of a page, tag
archives that hold one item each, and anything carrying noindex do not belong
in it — putting two URLs for one document in front of a crawler that is trying
to pick one is not helpful to either of you.
The site audit reads the sitemap as part of a crawl and reports the URLs in it that a crawler cannot reach.
Last reviewed . Also served as markdown at /glossary/xml-sitemap.md.