Crawling & Indexing

Why Is My Website Not Showing Up on Google?

If your website is missing from Google, first find out whether Google has indexed it at all or whether it is indexed but not ranking for the search you tried. Those are different problems with different fixes. This guide shows you how to check the evidence, find technical barriers and give your important pages a clear route into search.

Crawling & IndexingBeginner9 min readPublished

Is the page missing or simply not ranking?

People often say a website is “not on Google” when one of two things is happening:

  1. 01
    The page is not indexed.

    Google has not stored that URL as a page it can show in search results.

  2. 02
    The page is indexed but not visible for the search you tried.

    It may rank much lower than expected, or Google may consider another page more relevant.

This article focuses on the first problem: discovery, crawling and indexing. Ranking depends on relevance, content quality, competition, links and many other signals after a page becomes eligible for search.

A normal search for your company name is not a dependable index test. Results can vary by location, language, device and search history. Use Google Search Console’s URL Inspection tool to check the exact page instead.

Is the problem affecting one page or the whole website?

The scale of the problem helps narrow down the cause.

The whole website appears to be missing

Check the homepage first. If Google does not know about it, the site may be very new, inaccessible to crawlers, protected by a site-wide noindex directive or affected by a wider server, security or manual-action issue.

Also make sure you are checking the correct live hostname. A recent move from HTTP to HTTPS, or from non-www to www, can create confusion when old URLs and the new site are mixed together.

One page or section is missing

A page-specific noindex tag, canonical pointing elsewhere, weak internal linking, duplicate content, an error response or a blocked path may be responsible. If an entire product or article section is missing, look for a shared template or folder rule rather than treating every URL as a separate problem.

The pages were visible and then disappeared

Check what changed. A redesign, migration, CMS setting, plugin update or deployment can alter robots directives, canonical tags, internal links and server responses across many pages at once. Search Console’s Page indexing, Manual actions and Security issues reports can help show whether the change is technical or account-wide.

Why a website may not appear on Google

01 Google has not discovered it yet

A new page needs a route through which crawlers can find it. Google commonly discovers URLs through crawlable links and XML sitemaps. A page with no internal links and no sitemap entry can remain unknown for longer, particularly on a new site with few external references.

Make sure every important page has at least one normal HTML link from another useful page on the site. Include its canonical URL in a working XML sitemap where appropriate.

02 Crawlers cannot access the page

Google needs a stable response from the live URL. Robots.txt rules can prevent crawling, while a firewall, bot filter, login, network failure or rate limit can stop the request before the page is retrieved.

Check the exact protocol, hostname and path. A page that opens for an administrator can still fail for a crawler if the browser has a login cookie or has already passed a security challenge.

03 The page returns an error or redirects

An indexable page normally returns 200 OK. A 404 or 410 says that the page is unavailable. A 5xx response means the server failed to deliver it. A redirect tells crawlers to use another destination rather than indexing the original URL as an independent page.

Soft 404s also matter. These are pages that return 200 OK but look like an error or contain too little useful content to function as the requested page.

04 A noindex directive excludes it

A robots meta tag in the HTML or an X-Robots-Tag HTTP header can tell Google not to index a page. This is useful for private utility pages and unwanted search results, but damaging when it appears on a homepage, product, service or article by mistake.

For Google to see that a noindex directive has been removed, it must be allowed to crawl the page again. Blocking the same URL in robots.txt can delay that discovery.

05 Google selected another canonical URL

Google groups duplicate or very similar pages and chooses a representative canonical URL. If your preferred page points its canonical tag elsewhere, or the wider signals favour another version, the inspected URL may not appear independently.

Check redirects, internal links, sitemap entries and canonical tags together. They should consistently support the URL you want treated as primary.

06 The page is isolated, duplicated or offers little value

Technical eligibility does not guarantee indexing. Google may decide not to index a page that closely repeats another URL, exposes very little original information or is not well connected to the rest of the site.

Ask whether the page has a distinct purpose for visitors. Consolidate unnecessary duplicates, strengthen genuinely useful pages and link to them from relevant navigation or content.

07 Google needs time to recrawl and process the change

New sites and recently repaired pages do not appear immediately. Google needs to discover, crawl, render and process the URL. Requesting indexing can bring an eligible page to Google’s attention, but it does not guarantee inclusion or a particular processing time.

If the page passes the live technical checks, avoid changing it repeatedly while you wait. Monitor the indexed view in Search Console for the updated decision.

Check the live site in Clearcrawl

Clearcrawl shows the technical signals on the live site. This helps you find accidental noindex directives, blocked paths, redirects, failed responses, canonical conflicts and discovery gaps across many URLs at once.

Then inspect priority URLs in Google Search Console. Clearcrawl tells you what the site serves now; Search Console tells you what Google knows from its last processing and lets you test the current URL.

How to understand what you find

Common findings when a page is missing from Google
FindingWhat it meansWhat to do next
URL is unknown to GoogleGoogle has not encountered the URLAdd crawlable internal links, include it in the sitemap and request indexing
Blocked by robots.txtGoogle cannot crawl the page because of an access ruleRemove the block if the page should be crawled
Excluded by noindexThe page tells search engines not to index itRemove the directive only if the exclusion is accidental
Redirect pageThe URL sends crawlers elsewhereCheck that the final destination is the page you want indexed
Not found or server errorThe URL does not return a stable working pageRestore the content or repair the response
Duplicate; Google chose another canonicalGoogle selected a different representative URLAlign canonical tags, links, redirects and sitemap signals
Crawled or discovered, currently not indexedGoogle knows the URL but has not selected it for the indexCheck content value, duplication, internal linking and technical consistency
Live test passes but indexed view still shows an errorThe current page may be fixed, but Google has not processed the changeRequest indexing when appropriate and allow time for recrawling
URL is on Google but absent for your keywordThis is mainly a ranking or relevance problemReview search intent, content, competition and authority

Treat Search Console labels as evidence, not as a list of instructions to make every URL indexable. Duplicate filters, confirmation pages, retired URLs and other utility pages often belong outside the index. Check whether the important pages are indexable and whether a noindex rule is blocking them. A crawl with technical SEO software can help find sitewide access issues.

How to get an important page ready for Google

Work through the signals in order. The goal is not to make every URL appear in search; it is to give each important page a clear, technically consistent route into the index.

  1. 01
    Decide which URL should appear.

    Choose one permanent version of the page. Use HTTPS and the preferred hostname, and decide whether the trailing-slash or non-trailing-slash version is canonical. Redirect unnecessary variations to it.

  2. 02
    Make the page crawlable and available.

    Return 200 OK with the intended content. Remove accidental robots.txt blocks, login requirements and crawler restrictions. Repair intermittent server errors that could make access unreliable.

  3. 03
    Remove unintended indexing barriers.

    Check the robots meta tag and X-Robots-Tag header. Remove noindex only when the page should appear in search. Confirm that the canonical tag supports the intended URL rather than another page.

  4. 04
    Give crawlers a clear discovery route.

    Link to the page from relevant indexable pages using normal HTML links. Include its canonical address in the XML sitemap and make sure the sitemap can be read. Avoid leaving important content accessible only through an internal search box or a script action that does not create a crawlable link.

  5. 05
    Make the page worth indexing.

    Give it a clear purpose and enough original information to satisfy the visitor’s need. Merge or redirect thin duplicate pages when one stronger page would be more useful. Use a descriptive title and headings, but do not mistake metadata changes for a substitute for useful content.

  6. 06
    Test and submit the finished page.

    Run another crawl and inspect the exact URL in Search Console. Use Test live URL to check the current version. If the page is eligible, request indexing for an important individual URL. Submit a working sitemap when many new or updated pages need discovery.

How to verify the fix

Check both the live technical state and Google’s later processed state.

  • Open the exact canonical URL and confirm that it returns the intended page.
  • Verify the response, robots directives and canonical URL in a fresh Clearcrawl crawl.
  • Follow the page’s internal links and sitemap entry to make sure they use the same canonical address.
  • Inspect the URL in Google Search Console and run the live test.
  • Request indexing once after a meaningful repair if the live test succeeds.
  • Return later to see whether Google has updated the indexed result.

The live test reports whether the current page appears eligible; it does not promise that the URL will be indexed. If the technical checks pass but Google still does not select the page, review duplication, content value and how well the page is connected to the rest of the site.

Common mistakes

01Checking only with a Google search

Search results are ranked and personalised. Use URL Inspection for the exact page before assuming it is absent from the index.

02Requesting indexing before fixing the cause

Submitting a blocked, noindexed, broken or duplicate page does not remove the underlying signal. Repair the live page first.

03Removing every exclusion

Not every URL belongs in search. Preserve intentional exclusions for utility, duplicate and private pages while fixing important content.

04Treating one page when the problem comes from a template

If many URLs share the same error, inspect the CMS, page template, plugin or deployment setting. Correcting the common source is safer than editing each page separately.

Frequently asked questions

How long does it take for a new website to appear on Google?

There is no fixed waiting period. Google must discover, crawl and process the site, and a request for indexing does not guarantee inclusion. Make sure the homepage is accessible, linked, indexable and present in a working sitemap, then monitor it in Search Console.

How can I check whether Google has indexed my website?

Open Google Search Console and inspect the full URL of the homepage or another important page. The indexed view shows what Google currently knows. Use Test live URL to check whether the page’s present version appears eligible for indexing.

Why is my homepage indexed but other pages are missing?

Google may not have discovered the other pages, or they may have individual access, noindex, canonical, duplication or quality issues. Check internal links and the XML sitemap, then inspect representative missing URLs rather than relying on the homepage result.

Will submitting a sitemap make my website appear on Google?

A sitemap helps Google discover your preferred URLs, particularly on new or large sites. It does not override robots rules, noindex directives, errors or canonical signals, and it does not guarantee indexing or rankings.

Can my page be indexable and still not appear in Google?

Yes. Indexability means the page has no obvious technical barrier to inclusion. Google can still decide not to index it, select another canonical URL or rank it too low for the search you tried. Technical eligibility is the starting point rather than a guarantee.

Clearcrawl logoClearcrawl
A clearer next step

Find what is keeping important pages out of search

Crawl your website to find blocked paths, noindex directives, failed pages, canonical conflicts, broken discovery routes and sitemap problems. Review every affected URL in one report, repair the source and crawl again to confirm what changed.

7 free days. See what Clearcrawl can find, then choose the plan that fits your site.

Try Clearcrawl free for 7 daysStart a 7-day free trial Start with Clearcrawl. Subscribe when you’re ready.

Learn more from Google