Check for noindex tags across your whole site
Short answer Open Search Console's Page indexing report and look for the reason "Excluded by 'noindex' tag". If pages you want found are listed, something is telling Google to skip them. Then crawl the site with a free crawler to catch every page, and check one page's HTTP header, where a noindex can hide outside the page code.
Want help applying this to your business? Talk it through on a free 30-minute call.
Book a callHow do I check my whole site for noindex?
A noindex is a line in a page's code, or in the server's reply, that tells Google to leave the page out of search. Google describes what happens next:
"When Googlebot crawls that page and extracts the tag or header, Google will drop that page entirely from Google Search results, regardless of whether other sites link to it." (Google Search Central: Block Search indexing with noindex)
One page by mistake is a small problem. A setting that puts it on every page can empty your search traffic. Run three checks, in this order:
| Check | What it catches | Cost |
|---|---|---|
| 1. Search Console, Indexing > Pages | Every page where Google found a noindex | Free |
| 2. A site crawler | Every page that has one now, including pages Google hasn't revisited | Free up to 500 URLs |
| 3. Your home page's code and HTTP header | A noindex set by the server, invisible in the page editor | Free |
A `site:yourdomain.com` search on Google is a quick first look, but Google says its results are "not always exhaustive". It shows what is in, not why something is missing.
Step 1: read the noindex list in Search Console
In Search Console, open Indexing > Pages. Under "Why pages aren't indexed", find "Excluded by 'noindex' tag" and click it to see example pages. The list shows up to 1,000 examples (Search Console Help: Page indexing report).
Read it as two piles:
- Fine to keep: thank-you pages, login, cart and checkout, internal search results, tag archives.
- A problem: your home page, service pages, location pages, blog posts.
If the problem pile is long, look for a pattern. All blog posts means a content-type setting. Every page means a sitewide switch. Search Console's help also flags the warning sign: a drop in indexed pages without new errors can mean you are blocking pages "via robots.txt, 'noindex' or a required login".
This list only shows what Google saw on its last visit. Google says that can take a while:
"Depending on the importance of the page on the internet, it may take months for Googlebot to revisit a page." (Google Search Central: Block Search indexing with noindex)
So a noindex added last week may not appear yet. That is why step 2 matters.
Step 2: crawl the site to see every page today
A crawler is a program that visits every page on your site the way Google does and lists what it finds. Screaming Frog SEO Spider crawls up to 500 URLs free, enough for most service business sites; a paid yearly licence removes the limit (Screaming Frog). It reports noindex found in either the page code or the HTTP header.
- Download it, paste your home page address, and start the crawl.
- Open the Directives tab and filter to noindex.
- Export the list and compare it with the "fine to keep" pile above.
Run it again after every launch, theme change or SEO plugin update. It takes minutes on a small site.
Step 3: check the HTTP header, not just the page
Google accepts noindex in two places: a robots meta tag in the page code (it contains `name="robots" content="noindex"`) or an HTTP header the server sends with the page (`X-Robots-Tag: noindex`). The header never shows in your page editor or "View page source".
To check it, open your home page in Chrome, right-click, choose Inspect, open the Network tab and reload. Click the first row (your page) and read the response headers. Search Console's help says the same: search "the page source or response headers for the word 'noindex'". A header noindex usually comes from hosting, a security plugin or a CDN rule, not your website builder.
For one page, Search Console's URL Inspection tool settles it. Run the live test and read "Indexing allowed?". "No" means a noindex is still there.
Where does a sitewide noindex come from?
Most come from a switch someone flipped on purpose, then forgot.
| Source | What happens | Where to look |
|---|---|---|
| WordPress "Discourage search engines" | Adds a noindex tag to every page. Often left on from building the site, or copied over from a staging (test) site | Settings > Reading (WordPress Documentation) |
| SEO plugin content types | Yoast's toggle per type (Posts, Pages, Products) noindexes the whole type when off | Yoast SEO > Settings > Content types (Yoast) |
| Plugin change | A new SEO plugin brings its own defaults, or two plugins disagree | Plugins list; keep one SEO plugin |
| Wix | The "Let search engines index your site" toggle is off | SEO Dashboard > Go to SEO Settings (Wix Help Center) |
| Shopify | Private mode hides "all of your online store's pages" from visitors and search engines | Online Store > Preferences > Private mode (Shopify Help Center) |
| Server or CDN | An `X-Robots-Tag` header added for a test site and pushed to live | Ask your host or developer |
| Theme or template | One template carries a noindex, so every page built on it does | The template's SEO settings |
Google's own launch checklist names the classic case:
"Don't forget to remove any noindex or robots.txt blocks that were only needed for the migration." (Google Search Central: Site moves with URL changes)
For the full post-launch traffic check, see why traffic dropped after a redesign.
Don't fix it with robots.txt
robots.txt is a file that tells crawlers which pages not to visit. It is not a noindex, and it can hide one. Google warns that if a page is blocked by robots.txt, "the crawler will never see the noindex rule, and the page can still appear in search results". So:
- To keep a page out of Google: use noindex and let Google crawl it.
- To bring a page back: remove the noindex. Don't add a robots.txt block "to be safe".
The same applies to crawl budget, the number of pages Google crawls on your site. Google tells large sites not to use noindex to save it, since Google "will still request, but then drop the page". A small business site doesn't need to think about crawl budget.
I removed it. How fast does Google come back?
Pages return as Google recrawls them, important pages first.
- Confirm the noindex is gone: rerun the crawler and the live URL test.
- In URL Inspection, click Request indexing for your home page and top service pages.
- Resubmit your sitemap in Search Console so Google sees every page you want found.
Rankings come back as pages are re-indexed. Nothing guarantees a date, and less-visited pages can take longer.
Example
Illustrative cases with invented numbers.
A plumber after a redesign (US dollars)
The designer built the new site on a test address with WordPress's "Discourage search engines" box checked, then moved it live. Calls from search fell from about 40 a month to 10 over six weeks. The crawler showed noindex on all 60 pages. Unchecking one box and requesting indexing fixed the cause.
A Toronto accountant who switched SEO plugins
Search Console listed all 25 blog posts under "Excluded by 'noindex' tag", but the service pages were fine. The new plugin had the Posts type set to hidden. One toggle fixed it.
A clinic with a clean page source
The page code had no noindex, but the live URL test said "Indexing allowed? No". The response headers showed `X-Robots-Tag: noindex`, added by the host for a test copy. The host removed the rule.
FAQ
Is it bad to have some noindex pages?
No. Google lists a noindex tag among the right reasons for a page not to be indexed. Thank-you, login and checkout pages are normal.
Does a noindex checker website check my whole site?
Most free checkers test one address at a time. For every page, use Search Console's Page indexing report and a crawler.
Can a noindex page still show in Google?
Yes, until Google recrawls it, or if robots.txt blocks Google from seeing the tag.
Different for you?
- Your site doesn't show on Google at all: website not showing on Google.
- Search Console says "Crawled, currently not indexed" instead: page not indexed in Search Console.
- You're new to Search Console: Google Search Console for a small business.
- You're choosing a website platform: Wix, Squarespace or WordPress for SEO.
- You'd rather have someone watch indexing for you: SEO & GEO Agent.