XML Sitemap Optimization Guide with Auditite

Overview

XML sitemaps tell search engines which pages exist on your site and how important they are. A well-maintained sitemap accelerates indexing for new content and helps search engines prioritize crawling. This guide covers setup, optimization, and ongoing maintenance.

Step 1: Sitemap Structure

Single vs. Multiple Sitemaps

• Sites with under 50,000 URLs can use a single sitemap file.
• Larger sites should use a sitemap index file that references multiple sitemaps.
• Segment sitemaps by content type: products, categories, blog posts, static pages.
• Each individual sitemap file must contain no more than 50,000 URLs and be no larger than 50MB uncompressed.

Sitemap Index Example

Your sitemap index at /sitemap.xml should reference individual sitemaps:

• /sitemap-pages.xml — Static pages and landing pages
• /sitemap-blog.xml — Blog posts and articles
• /sitemap-products.xml — Product pages
• /sitemap-categories.xml — Category pages

Step 2: What to Include

Include

• All pages that return a 200 status code and are set to index
• Pages with canonical tags pointing to themselves
• Pages that you want search engines to discover and index

Exclude

• Pages with noindex meta tags
• Redirect URLs (301, 302)
• Pages blocked by robots.txt
• Duplicate pages (canonicalized to another URL)
• Paginated pages (unless they contain unique, valuable content)
• URL parameter variations
• Internal search result pages

Step 3: Optimize Sitemap Elements

lastmod

• Only use lastmod when the page content has actually changed.
• Use ISO 8601 format: 2026-03-15T10:30:00+00:00 or 2026-03-15.
• Do not update lastmod dates without content changes — Google will learn to ignore your lastmod values.

priority and changefreq

Google has confirmed it ignores both priority and changefreq. You can omit these elements to keep your sitemaps smaller and simpler.

Step 4: Technical Requirements

1. Serve sitemaps in UTF-8 encoding.
2. Use absolute URLs (including protocol and domain).
3. URL-encode special characters in URLs.
4. Reference your sitemap in robots.txt: Sitemap: https://yourdomain.com/sitemap.xml.
5. Submit your sitemap in Google Search Console and Bing Webmaster Tools.
6. Serve sitemaps with gzip compression to reduce transfer size.

Step 5: Dynamic Sitemap Generation

1. Generate sitemaps automatically from your CMS or database rather than maintaining them manually.
2. Regenerate sitemaps when content is published, updated, or deleted.
3. For large sites, implement incremental sitemap generation that only updates the affected sitemap file.
4. Include only canonical URLs — the sitemap generation logic should query for indexable pages only.

Step 6: Audit and Maintenance

1. Run a monthly sitemap audit with Auditite to identify discrepancies between your sitemap and your actual site.
2. Check for common issues:
   • URLs in the sitemap that return 404 or redirect
   • URLs in the sitemap that are noindexed
   • Important pages missing from the sitemap
   • Sitemap URLs that differ from canonical URLs
3. Compare the number of URLs in your sitemap against the number of indexed pages in Search Console.
4. If indexed pages are significantly fewer than sitemap URLs, investigate why pages are not being indexed.

Step 7: Specialized Sitemaps

Image Sitemaps

Include image sitemap tags for important images that you want indexed in Google Images. This is especially valuable for e-commerce product images.

Video Sitemaps

If you host video content, use video sitemap tags with thumbnail URL, title, description, and duration.

News Sitemaps

For news publishers, create a separate news sitemap containing articles published in the last 48 hours.

Monitoring

1. Check Google Search Console’s Sitemaps report weekly for errors.
2. Track the “Discovered - currently not indexed” count — a growing number may indicate quality or crawl budget issues.
3. Monitor the time between publishing new content and it appearing in Google’s index.