Products SEO & GEO Agent Paid Ads Agent Website Agent Conversion Agent
How it works Pricing FAQ Blog Book a call

Sitemap best practices today: lastmod, IndexNow and AI bots

Updated · 7 min read

Short answer List only the pages you want in search, with full addresses and an honest lastmod date. Point to the file from robots.txt and submit it in Google Search Console and Bing Webmaster Tools. Skip priority and changefreq: Google and Bing ignore them. Add IndexNow so Bing and Copilot hear about changes right away. Google doesn't use IndexNow.

Want help applying this to your business? Talk it through on a free 30-minute call.

Book a call

The sitemap checklist

A sitemap is a file that lists your site's pages so search engines can find them. Most website builders make one for you. Google's sitemap guide says WordPress, Wix and Blogger usually publish one already. Your job is to check it, not build it.

CheckWhyWho says so
Only pages you want in searchThe file tells engines which addresses you prefer to showGoogle
One address per page (the canonical, or main, version)Google shows the main version, and your sitemap helps it pickGoogle
Full addresses, like `https://example.com/roof-repair/`Google asks for complete addresses, not `/roof-repair/`Google
An honest `lastmod` date on each pageThe only tag both engines useGoogle, Bing
No `priority` or `changefreq` neededBoth engines ignore themGoogle, Bing
Under 50,000 addresses and 50MB per fileSplit larger lists into several filesGoogle; Bing for the 50,000
A `Sitemap:` line in robots.txtBoth engines find the file there on their ownGoogle, Bing
Submitted in Search Console and Bing Webmaster ToolsShows read errors and the last read dateGoogle, Bing
IndexNow turned on, if your platform offers itBing hears about changes right awayBing, IndexNow

Robots.txt is a small text file at your site's root that tells search robots what they may crawl. One line there is enough:

`Sitemap: https://example.com/sitemap.xml`

What changed since 2024?

Older advice about sitemaps is out of date in four ways.

ThenNowSince
"Ping" Google after each updateGoogle turned the ping address off; it returns an errorAnnounced June 2023, off six months later
Set priority to rank key pagesGoogle and Bing both say they ignore itGoogle since at least 2023; Bing said so July 2025
lastmod is optional decorationBing calls it a key signal for recrawling, including for AI answersJuly 2025
Wait for engines to re-read the sitemapIndexNow tells Bing and others the moment a page changesLive now; Google not part of it

Google's June 2023 post explained the ping change: anonymous pings mostly brought spam. Old plugins that still ping do no harm, it said, but do nothing useful either.

lastmod: the one tag that still counts

`lastmod` is the date a page last really changed. Update it when the main text, the structured data or the links change. Don't update it for a footer or sidebar tweak. Google uses the date only if it's consistently accurate, and its June 2023 post says what happens when it isn't:

"if your page changed 7 years ago, but you're telling us in the lastmod element that it changed yesterday, eventually we're not going to believe you anymore when it comes to the last modified date of your pages." (Google Search Central Blog, June 2023)

Bing warns about the most common lie: a builder that stamps every page with the moment the sitemap was generated. Bing also asks for the full date and time format, like `2026-11-16T09:30:00+00:00`, not just the day.

How to check yours: open `yoursite.com/sitemap.xml` in a browser. If every page shows today's date, your platform is resetting it. Ask your web host or check the SEO plugin's settings.

IndexNow: tell Bing the moment a page changes

IndexNow is a free way to tell search engines that a page was added, changed or deleted. You send the address; they come to crawl it. Its home page lists support from Microsoft Bing, Naver, Seznam.cz, Yandex and Yep. Its engine list also names the Internet Archive and Amazonbot. Google is not on either list, so it changes nothing on Google.

Why it matters for a small business: Bing's July 2025 post is about keeping pages fresh in both Bing and Copilot, and it recommends a full sitemap plus IndexNow together:

"Use IndexNow for real-time URL submission, instantly notifying Bing and participating search engines when individual URLs are added, updated, or removed." (Bing Webmaster Blog, July 2025)

How to turn it on:

  1. Check your platform first. The IndexNow FAQ lists WordPress, Shopify, Wix, GoDaddy, Duda, Joomla! and Drupal among those with built-in support or a plugin, and Cloudflare among networks.
  2. If yours supports it, switch it on. No key file needed; it submits on its own.
  3. If not, a developer creates a key (8 to 128 letters, numbers or dashes), saves it as a text file at your site's root, and sends changed addresses to the IndexNow address. One message can carry up to 10,000 addresses.

Keep the sitemap anyway. The IndexNow FAQ says IndexNow is for fresh changes, and the sitemap is the complete list, including pages that rarely change. A success reply only means the engine got your address, not that it will index the page.

Do AI bots read your sitemap?

Partly, and it depends on the bot.

  • Copilot: Bing's July 2025 post says sitemaps support its crawling for both normal search and Copilot, and that lastmod affects how fast updates reach AI answers.
  • ChatGPT search: OpenAI's crawler page says its search bot is OAI-SearchBot, and that sites blocking it won't be shown in ChatGPT search answers. OpenAI doesn't say whether that bot reads sitemaps. So the sitemap step that matters is simpler: make sure robots.txt doesn't block it.
"Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links." (OpenAI: Overview of OpenAI Crawlers)
  • Google AI Overviews and AI Mode: Google's AI features page says a page only needs to be indexed and eligible to show in normal search with a snippet. There are no extra technical steps. Your Search Console sitemap already covers that.

Blocking training bots like GPTBot while allowing search bots is a separate choice. See should you block AI crawlers. How ChatGPT picks local businesses is in where ChatGPT finds local businesses.

Do you even need a sitemap?

A small, well-linked site can do without one. Google defines small as about 500 pages or fewer that you want in search, where every important page can be reached by links from the home page. Most service businesses fit that. A sitemap still helps, costs nothing, and gives Search Console a list to report against. It never guarantees indexing:

"A sitemap helps search engines discover URLs on your site, but it doesn't guarantee that all the items in your sitemap will be crawled and indexed." (Google Search Central: Learn about sitemaps)

An HTML sitemap is a normal page listing your pages for people. It can help visitors on a big site, but the XML file is what search engines read. If pages listed in your sitemap still aren't in Google, see crawled or discovered, currently not indexed.

Example

A plumber on Wix with 25 pages

Wix makes the sitemap automatically. The owner adds the site to Google Search Console and Bing Webmaster Tools, submits `sitemap.xml` in each, and confirms Bing shows a recent last read date. Total time: about 30 minutes. No IndexNow setup needed if Wix handles it.

A law firm on WordPress that redesigned its site

The new site changed 60 page addresses. The firm checks that the sitemap lists only the new addresses, not old ones that now redirect. Its SEO plugin's IndexNow switch sends all 60 to Bing in one go. The IndexNow FAQ says submitting a whole site after a redesign is acceptable.

A clinic whose sitemap shows today's date on every page

All 40 pages say they changed today, every day. The web developer fixes the setting so each page shows its real last edit date. Now when the clinic updates its prices page, that date is the one that stands out.

FAQ

Does Google use IndexNow?

No. Google isn't an IndexNow engine. Keep your sitemap in Search Console; request recrawls in URL Inspection. Google says crawling takes days to weeks.

Should I delete priority and changefreq from my sitemap?

You don't have to. Google and Bing ignore them, so leaving them in does no harm and removing them gains nothing.

How often do Google and Bing read my sitemap?

Bing reads it right after you submit it, then usually daily. Google publishes no schedule; Search Console's Sitemaps report shows its last read.

Can I just ping my sitemap after an update?

Not Google: that address stopped working in late 2023. Use IndexNow for Bing, and the Sitemaps report or URL Inspection in Search Console for Google.

Do I need separate sitemaps for images or video?

Usually not. Google says sitemaps help most when a site has lots of video, images or news pages. Few media files means little gain.

Different for you?

Keep learning

NextSitemap errors: fix common XML sitemap problemsWhat each Search Console sitemap status and error means, from Couldn't fetch to URL not allowed and blocked by robots.txt, and how to fix each one.

Do it yourself

Crawlability checklist: make every page reachable by GoogleA crawlability checklist for a small business site: find orphan pages, robots.txt blocks, broken links, redirect chains and server errors, with free tools.

Ask us one question

One line is enough. We reply by email.

Sources

  1. Google Search Central: Build and submit a sitemap
  2. Google Search Central: Learn about sitemaps
  3. Google Search Central Blog: Sitemaps ping endpoint is going away (June 2023)
  4. Bing Webmaster Blog: Keeping Content Discoverable with Sitemaps in AI Powered Search (July 2025)
  5. IndexNow: Home
  6. IndexNow: Documentation
  7. IndexNow: FAQ
  8. IndexNow: Participating search engines (searchengines.json)
  9. OpenAI: Overview of OpenAI Crawlers
  10. Google Search Central: AI features and your website
  11. Google Search Central: Ask Google to recrawl your URLs