Products SEO & GEO Agent Paid Ads Agent Website Agent Conversion Agent
How it works Pricing FAQ Blog Book a call

Website indexing tools: check and monitor index status

Updated · 6 min read

Short answer Only the search engine knows what it has indexed, so its own free tools are the ones to trust: Google Search Console and Bing Webmaster Tools. A crawler such as Screaming Frog shows which pages could be indexed and can pull Google's status for up to 2,000 URLs a day. IndexNow notifies engines; it checks nothing.

Want help applying this to your business? Talk it through on a free 30-minute call.

Book a call

Which tool tells you if a page is indexed?

The search engine's own tool. Indexed means the engine has stored the page and may show it in results. Only Google knows what is in Google's index, and only Bing knows Bing's. Every other tool either asks them or guesses.

ToolQuestion it answersCostLimit
Google Search Console: URL InspectionIs this one page in Google, and if not, why?FreeOne page at a time
Google Search Console: Page indexing reportHow many pages are indexed, and why are the rest not?FreeExample lists stop at 1,000 URLs
Bing Webmaster Tools: URL InspectionIs this page in Bing, and can Bingbot crawl it?FreeOne page at a time
Site crawler (Screaming Frog SEO Spider)Which pages could be indexed, and which are blocked?Free up to 500 URLs; £199 a year per licenceCan't see the index unless linked to Search Console
URL Inspection API (through a crawler or script)Google's status for many pages at onceFree2,000 checks a day per site
IndexNowNothing: it tells engines a page changedFreeNot used by Google
A `site:yourdomain.com` searchA rough sample of indexed pagesFreeNot a full or reliable count

Sources: Search Console Help (Page indexing report; URL Inspection tool), Bing Webmaster Blog, Screaming Frog pricing page, Google's API usage limits, IndexNow FAQ.

Google Search Console: the one tool every site needs

Search Console gives you two views of the same data.

URL Inspection checks one address. Google describes it plainly:

"The URL Inspection tool provides information about Google's indexed version of a specific page, and also allows you to test whether a URL might be indexable." (Search Console Help: URL Inspection tool)

"URL is on Google" means the page can appear in results. It does not mean it will rank. Google says the status "doesn't guarantee that your page is appearing in Search results." "URL is not on Google" means it can't appear, and the tool usually gives the reason. For a new or changed page, click Request indexing to ask Google to crawl it; Google says this doesn't guarantee the page will appear, and that indexing can take up to a week or two.

The Page indexing report shows the whole site: a count of indexed pages over time and a list of reasons the others were left out. Two of Google's own notes keep owners from panicking:

"If your site has fewer than 500 pages, you probably don't need to use this report." (Search Console Help: Page indexing report)
"Your goal is to get the canonical version of every important page indexed." (Search Console Help: Page indexing report)

The canonical version is the one copy Google picks when several addresses show the same page. Don't aim for 100%: pages you blocked on purpose, duplicates and deleted pages are meant to stay out.

A useful filter: set the report to All submitted pages to see only URLs in your sitemap (the file listing the pages you want found). If those are indexed, you're fine. How to read each status: Search Console Page indexing and URL Inspection.

Why a site: search isn't an index checker

Searching `site:yourdomain.com` on Google is fast, but Google warns it was "designed primarily for search users" and "doesn't necessarily return all the URLs that are indexed" (Google Search Central: site: search operator). A missing page may still be indexed, and the count changes from search to search. Use it as a quick look, then confirm in URL Inspection. The two-minute version of this check is in indexing in SEO.

Bing Webmaster Tools: the second half of the picture

Google isn't the only engine your customers use. Bing Webmaster Tools is free, and its URL Inspection tool lets you "Analyze indexing status, crawlability, HTTP response codes, and blocked resources for specific URLs" (Bing Webmaster Blog, June 2025). If your site is already verified in Search Console, Bing lets you import it, so setup takes minutes.

Crawlers: "indexable" is not "indexed"

A crawler such as Screaming Frog SEO Spider visits your pages like a search bot. It finds what would keep a page out of the index: a noindex tag, a robots.txt block, a redirect, a broken link, a page nothing links to. That's the cause side. It can't see Google's decision on its own.

Connect it to Search Console and it can. Screaming Frog says the link pulls "Google index status data for up to 2,000 URLs per property a day" through Google's URL Inspection API. Google set that quota in January 2022: 2,000 checks a day and 600 a minute per site (Google Search Central Blog). The free version crawls 500 URLs and includes the Search Console link (Screaming Frog pricing page), which covers most small business sites.

What fixes the causes it finds: indexing blockers.

IndexNow notifies; it doesn't monitor

IndexNow is a free signal your site sends when a page is added, changed or deleted. It does not report what got indexed:

"Submitting a URL does not guarantee immediate indexing." (IndexNow: FAQ)

The FAQ sends you back to "each engine's site inspection or reporting tools" to check status. Google isn't among the participating engines. Setup with sitemaps: submit a sitemap to Google and Bing.

How to monitor index status, by site size

SiteRoutine
Under 50 pagesInspect each new or changed page once in Search Console. Glance at the indexed count monthly.
50 to 500 pagesMonthly: Page indexing report filtered to submitted pages, plus a free crawl to catch new blocks.
Over 500 pagesMonthly crawl linked to the URL Inspection API for your key pages; watch the indexed count for sudden drops.

A sudden drop with no new errors usually means pages were blocked. Google's Page indexing help says to check for robots.txt rules, noindex tags or a new login requirement. After a fix, press Validate fix: Google emails you when it confirms the fix or finds the problem again.

Example

Illustrative cases with round numbers.

An electrician with a 20-page site

He adds Search Console and Bing Webmaster Tools (imported from Search Console). Each time he publishes a page, he inspects it once in each. Cost: $0, about 10 minutes a month.

A dental group with 300 pages

Their indexed count in the Page indexing report fell from 280 to 150 in a week, with no new errors. A free crawl shows a theme update added noindex to every service page. They remove it, validate the fix and wait for Google's email.

A retailer with 4,000 product pages

He buys a Screaming Frog licence and links Search Console. Each month the crawler pulls Google's status for 2,000 key product and category URLs, and he fixes the ones marked not indexed that should be.

FAQ

What's the best free website indexing tool?

Google Search Console, because it reports Google's own index. Add Bing Webmaster Tools for Bing.

Can I check the index status of many URLs at once?

Yes, through the URL Inspection API: 2,000 checks a day per site. Crawlers like Screaming Frog connect to it without code.

Why does my indexed count differ between tools?

Google says report totals won't match your own page count, and site: searches aren't a full count. Trust URL Inspection for single pages.

Do third-party SEO suites show what Google indexed?

Their site audits show what blocks indexing. Only Google's tools, or software calling its API, show Google's actual status.

Different for you?

Keep learning

NextSEO copywriting tools: checkers that work while you writeWhich SEO copywriting tools check grammar, readability and keywords as you type, where each one runs, what is free, and what AI writers add in November 2026.

Ask us one question

One line is enough. We reply by email.

Sources

  1. Search Console Help: Page indexing report
  2. Search Console Help: URL Inspection tool
  3. Google Search Central: site: search operator
  4. Google Search Central Blog: URL Inspection API (January 2022)
  5. Google for Developers: Search Console API usage limits
  6. Bing Webmaster Blog: Start using Bing Webmaster Tools (June 2025)
  7. IndexNow: FAQ
  8. Screaming Frog: SEO Spider pricing
  9. Screaming Frog: SEO Spider configuration guide (Search Console integration)