Ds Moz Sitemap

How it works

A sitemap is an XML file that lists URLs a site owner wants search engines to discover. It provides crawl hints such as the last modified date (lastmod), change frequency, and alternate language versions (hreflang). Google reads sitemaps to crawl sites more efficiently, especially for large, new, or frequently updated sites. Sitemaps can also include video, image, and news URLs.

Search engines process sitemaps to prioritise crawl activity. The file signals which URLs the site owner considers important, but it does not guarantee indexing. Metadata like lastmod helps Google decide when to recrawl. Sitemaps must use absolute, canonical URLs and respect size limits: 50MB uncompressed or 50,000 URLs per file. If a site exceeds these limits, the sitemap must be split into multiple files with a sitemap index.

Submission is straightforward. Most web hosting platforms and CMSs automatically generate a sitemap. Google recommends submitting the sitemap URL in Search Console’s Sitemaps report or referencing it in robots.txt. This ensures search engines can find the file. Regularly check Search Console for processing errors to confirm the sitemap is being read correctly.

Common mistakes

• Including non-canonical, redirected, or parameter-heavy URLs instead of the primary version. Consequence: wasted crawl budget and confusion for search engines, which may index the wrong URL.
• Treating a sitemap as a ranking tool rather than a crawl/discovery aid. Consequence: unrealistic expectations; sitemaps do not directly improve rankings.
• Submitting a sitemap but not checking Search Console for processing errors or crawl status. Consequence: undetected issues like broken URLs or format errors can prevent pages from being indexed.
• Using a sitemap to list pages you do not want crawled or indexed, such as admin or password-protected URLs. Consequence: security risk and wasted crawl budget; use robots.txt or noindex tags instead.
• Not updating the sitemap when content changes. Consequence: stale signals reduce crawl efficiency.

Using a tool like moz can help identify non-canonical URLs in your sitemap.