Excluded by ‘Noindex’ Tag: How to Find and Fix It

What does “excluded by noindex tag” mean?

“Excluded by noindex tag” means a search engine found a noindex instruction when it crawled a page. The instruction asks the search engine not to include that page in its searchable index.

The rule normally appears in one of two places:

<meta name="robots" content="noindex">

This meta tag sits in the <head> of an HTML page. A server can also send the same instruction in an HTTP response header:

X-Robots-Tag: noindex

Google Search Console may describe the issue as Excluded by ‘noindex’ tag or URL marked ‘noindex’, depending on the report you are viewing. Both descriptions point to the same basic situation: Google detected a noindex rule and did not index the page.

When is a noindex tag intentional?

A deliberate noindex tag can keep low-value or private-to-the-workflow pages out of search without removing them from your website. Common examples include:

• Account, basket and checkout pages
• Internal search-result pages
• Campaign landing pages that should only be reached through a direct link
• Thank-you and form-confirmation pages
• Thin filter or sort variations that create near-duplicate URLs
• Staging pages that are protected from public access as well as search

The decision depends on the purpose of the page. A service page, useful guide, product category or location page will usually need to be indexable if it is intended to attract search visitors. A checkout confirmation page usually will not.

Noindex is also different from privacy. Anyone with the URL may still be able to open the page. Protect genuinely private material with authentication rather than relying on a search directive.

Why an unintended noindex tag matters

An important page cannot earn normal search visibility while a search engine honours its noindex rule. The page may still work perfectly for visitors who follow a direct link, so the problem can remain unnoticed.

• Visibility: the page cannot appear in ordinary search results.
• Discovery: other useful pages may lose a search entry point if the excluded page acts as a hub.
• Internal linking: links may continue to point towards a page that cannot attract search traffic itself.
• Reporting: a sudden group of excluded pages can reveal a template, plugin or deployment change affecting a whole section.

A single intentional exclusion needs no repair. A shared noindex instruction across an important template can remove hundreds of pages from search at once, so look for patterns before working URL by URL.

How to find pages excluded by noindex

Start with a full crawl of the public site. This shows which accessible pages currently return a noindex instruction and helps you review them as a group. Then use Google Search Console to confirm how Google sees any priority URL.

Clearcrawl reports what the live page or response tells crawlers during the check. Search Console reports Google’s own last processed state. Use both when a page has recently changed: the live crawl helps you check the fix now, while Search Console may take time to update after Google recrawls the URL.

Understanding where the directive comes from

A noindex instruction is often controlled by a content-management setting rather than written directly into the page. Check the rendered HTML and the HTTP response header, then trace the rule back to its source.

You may also see values such as noindex, follow or noindex, nofollow. The word noindex is the part that prevents indexing. Removing a different value while leaving noindex in place will not solve the exclusion. To distinguish an unwanted exclusion from other indexing problems, compare the indexability checks with the reasons a crawled page may stay unindexed.

How to fix an unwanted noindex tag

Only remove the directive when the page should be eligible for search. Make the change at the source so it does not return during the next publish or template update.

1. 01
   Confirm that the page should be indexed.

   Check whether the page offers distinct, useful content and belongs in the search journey. If it is a duplicate, utility page or private step, the noindex rule may be correct.

2. 02
   Find every version of the instruction.

   Inspect the page’s robots meta tags and its HTTP response headers. Check whether the rule affects one URL, a template, or an entire site section. A second noindex instruction can keep the page excluded after you remove the first.

3. 03
   Remove noindex at its source.

   Change the CMS search-visibility option, SEO plugin setting, template rule or HTTP header that created the directive. The finished page can use index, follow, but an explicit index tag is normally unnecessary; the absence of noindex is enough to make an otherwise eligible page indexable.

4. 04
   Make sure crawlers can access the page.

   Return a successful 200 status and check that robots.txt does not block the URL. Search engines need to crawl a page to see that noindex has been removed. Do not add a robots.txt block as a substitute for noindex.

5. 05
   Strengthen the route to the page.

   Link to the page from relevant indexable pages and include it in the XML sitemap when appropriate. These signals help crawlers discover the updated URL, although they do not guarantee indexing.

Verify the fix

Publishing the change is only the first check. Confirm the live rule, then give Google a route to process it again.

• Open the live page and confirm that the robots meta tag does not contain noindex.
• Check the HTTP response and confirm that no X-Robots-Tag: noindex header remains.
• Run another Clearcrawl crawl and verify that the URL is no longer reported as noindex.
• Inspect the URL in Google Search Console and run Test live URL.
• If the live test allows indexing, request indexing for an important page or wait for Google to recrawl it naturally.

The page will not necessarily appear in search immediately. Removing noindex makes it eligible for indexing; it does not guarantee that Google will select or rank the page.

Common mistakes

Frequently asked questions