Bulk URL inspection: check indexing for many pages at once
Short answer Start free: in Search Console's Page indexing report, filter to your sitemap and read which pages are left out and why. For page-by-page detail, the URL Inspection API checks up to 2,000 URLs per property a day. A script, Screaming Frog (£199 a year) or a crawler suite can call it. No tool gets past that daily cap.
Want help applying this to your business? Talk it through on a free 30-minute call.
Book a callHow do I check indexing for many URLs at once?
Use the Page indexing report first, then the URL Inspection API for the pages it doesn't settle. Search Console's inspection box takes one address per search, but Google offers two ways around that.
| Method | What you get | Limit | Cost |
|---|---|---|---|
| Page indexing report, filtered to a sitemap | Indexed vs not indexed, with a reason, for the pages you submitted | Example lists stop at 1,000 URLs per reason | Free |
| URL Inspection API, run by your own script | Full inspection data for each URL | 2,000 URLs per property a day, 600 a minute | Free |
| Screaming Frog SEO Spider with the API switched on | Index status added to each crawled page | Same 2,000 a day | £199 a year (the free version has no Search Console link) |
| Crawler suites such as JetOctopus | Inspection plus crawling, logs and alerts on one dashboard | Same 2,000 a day | From $349 a month, billed yearly |
Sources: Page indexing report, Search Console API: Usage limits, Screaming Frog pricing, JetOctopus pricing, all checked November 16, 2026.
The daily cap is Google's, not the tool's. Every paid tool asks the same API, so paying more buys a nicer dashboard, not more inspections.
Do I even need bulk inspection?
Probably not if your site has under 500 pages. Google says so about its own report:
"If your site has fewer than 500 pages, you probably don't need to use this report." (Search Console Help: Page indexing report)
A plumber with 30 pages can inspect the home page and each service page by hand in ten minutes. Bulk checks pay off on sites with hundreds of product, location or blog pages, or after a redesign that changed many addresses.
The free way: filter Page indexing to your sitemap
A sitemap is the file listing the pages you want Google to find. Filtering to it answers "which of my pages are missing?" without inspecting each one.
- In Search Console, open Indexing > Pages.
- In the filter above the chart, pick All submitted pages, or one sitemap. Google: this shows "only URLs listed in a specific sitemap or sitemap index" (Search Console Help).
- Under "Why pages aren't indexed", click each reason to see its example pages.
- Export the table to a spreadsheet, or click the inspect icon on any row to open URL Inspection for that page.
The catch: the example list "is limited to 1,000 rows" and "does not necessarily show all URLs with that issue" (same page). For a full page-by-page answer, you need the API. What each reason means is in crawled or discovered, currently not indexed. No sitemap yet? See submit a sitemap to Google.
How the URL Inspection API works
The API returns, for any page, the same data the inspection box shows. Google launched it in January 2022 with 2,000 queries a day and 600 a minute, and:
"the quota is enforced per Search Console website property" (Google Search Central Blog)
The current limits page lists the same per-site figures, "2000 QPD" and "600 QPM" (Search Console API: Usage limits). A property is a site you have verified in Search Console. You can only inspect pages on properties you manage.
Three things to know before you run it:
- It shows Google's stored copy, not your live page. Google: "Presently only the status of the version in the Google index is available; you cannot test the indexability of a live URL" (API reference). A fix you made yesterday won't show until Google crawls again.
- More properties, more quota. The cap counts per property, so verifying folders such as `example.com/blog/` as their own properties gives each one its own 2,000. Screaming Frog uses this: with several properties set up it "will automatically detect all relevant properties" and can go past 2,000 in one crawl (Screaming Frog).
- Spend the quota on pages that matter. Screaming Frog queries the first 2,000 HTML pages it finds, and can skip pages already marked non-indexable (same page). Inspect service, product and location pages; skip tag archives and filters.
With a script, you need a Google Cloud project with the Search Console API turned on and access to the property. If that sentence lost you, the sitemap filter or Screaming Frog is the better route.
What the results tell you
Each inspected URL returns these fields. Read them as a checklist, top to bottom.
| Field | Google's description | What to look for |
|---|---|---|
| verdict | "whether the URL is indexed" | FAIL or NEUTRAL (excluded) on a page you want found |
| coverageState | "Could Google find and index the page" | The reason in words, as in the Page indexing report |
| robotsTxtState | "blocked to Google by a robots.txt rule" | BLOCKED on a real page means a robots.txt mistake |
| indexingState | "blocks indexing through a noindex rule" | A noindex you didn't mean to set |
| pageFetchState | "Google could retrieve the page from your server" | Server errors, 404s, redirects |
| googleCanonical vs userCanonical | The main version Google chose vs the one you declared | A mismatch means Google indexes another address instead |
| lastCrawlTime | Last crawl by Google's main crawler | Months old on an important page |
| richResultsResult | "Result of the Rich Results analysis" | Errors in the markup behind review stars or prices in results |
All quotes from Search Console API: UrlInspectionResult. The mobile usability part of the result is marked deprecated there, so don't build a check on it. The full list of causes behind a "not indexed" result is in indexing blockers.
One more limit: "URL is on Google" means eligible, not guaranteed. Google says the result "doesn't actually guarantee that your page will appear in Search results" (Search Console Help: URL Inspection tool).
Can I request indexing in bulk?
No. Request indexing works one URL at a time in Search Console, with a cap:
"There is a daily limit to how many index requests you can submit. If you want many pages indexed, try submitting a sitemap to Google." (Search Console Help: URL Inspection tool)
Tools that promise bulk indexing often use Google's Indexing API. Google limits it to "pages with either JobPosting or BroadcastEvent embedded in a VideoObject" and warns that abuse, "including the use of multiple accounts", can get access revoked (Indexing API quickstart). For a service page or a blog post, it is the wrong tool.
Example
A 40-page roofing site
The owner inspects the home page and eight service pages by hand: all "URL is on Google". No bulk tool needed. Time: 15 minutes, cost $0.
A 1,500-product online store
The sitemap filter shows 400 pages "Crawled - currently not indexed", but not which pattern they share. A developer runs the API on the 1,500 product URLs in one day, under the 2,000 cap. Most share one Google-chosen canonical: a color-variant problem, fixed in the template. Cost: a few hours of the developer's time.
A multi-location clinic with 6,000 pages
They verify `/locations/`, `/services/` and `/blog/` as separate properties, each with its own 2,000 daily inspections, and run Screaming Frog weekly. Cost: £199 a year for the licence.
FAQ
Is there a free bulk URL inspection tool?
Google's Page indexing report is free and covers every submitted page in summary. The URL Inspection API is also free; you pay only if you buy a tool that runs it for you.
Why do tools all say 2,000 URLs a day?
That is Google's API limit per property, not a tool feature. Any tool claiming more is splitting work across several properties or isn't using the API.
Does bulk inspection show my live pages?
No. It reports the version Google last indexed. To test today's version, use Test live URL in Search Console, one page at a time; that test has its own daily limit per property.
How often should I run a bulk check?
After launches, redesigns and URL changes, and whenever the Page indexing chart drops. New pages take time to show up; see how long indexing takes before you worry.
Different for you?
- Your site has under 500 pages: read the reports that matter in Search Console Page indexing and URL Inspection.
- Pages show "discovered" or "crawled, not indexed": see crawled or discovered, currently not indexed.
- You suspect a sitewide block: check indexing blockers.
- You want someone to watch indexing for you: see SEO.