5xx Server Errors: How to Find and Fix Them

What is a 5xx server error?

A 5xx response is an HTTP status code showing that the server could not complete a request. Unlike a 4xx response, which usually describes a problem with the requested URL or the client’s access, a 5xx response points to a failure on the server side of the exchange.

The page address may be correct. The server may still fail because an application crashed, a database became unavailable, an upstream service stopped responding or the system ran out of resources.

Visitors often see a generic message such as “Internal Server Error,” “Bad Gateway” or “Service Unavailable.” Crawlers receive the response code and cannot access the intended page content.

What do the common 5xx response codes mean?

The status narrows the investigation, although the precise root cause still depends on the website’s infrastructure.

Other 5xx codes exist, but these four explain many website incidents. Record the exact response instead of grouping every failure under “the site was down.” The code, time and affected path help the technical team find the responsible layer.

An error page should return the matching 5xx status. A friendly maintenance screen that sends 200 OK hides the failure from monitoring tools and can be mistaken for normal page content.

How do 5xx errors affect SEO?

Search engines cannot crawl the intended content when the server returns an error. Google says server errors can mean that the request timed out, the site was busy or Googlebot had to abandon the request.

The practical effects depend on frequency, duration and scope:

• New or changed pages may not be processed: the crawler receives no usable content.
• Existing pages may not be refreshed: prices, availability, titles and other changes can remain out of date in search.
• Crawl activity may slow: Google adjusts its crawl rate when a host appears unable to serve requests reliably.
• Persistent failures can affect indexing: URLs that repeatedly remain unavailable may eventually be dropped.
• Visitors lose trust and conversions: an inaccessible product, form or checkout fails regardless of search visibility.

A temporary 5xx response does not instantly remove a page from search. Search engines expect servers to fail occasionally and normally retry. The concern grows when errors persist or affect many important URLs.

Prioritise the homepage, robots.txt, XML sitemaps, important landing pages and any template with a sudden error increase. A robots.txt availability failure can have wider crawling consequences than one failed low-priority page.

How to find 5xx errors

Run a website crawl

A crawl records the response code returned by each requested URL. It can reveal pages that consistently fail, redirects ending on server errors and site sections affected by a shared application or template.

One successful crawl does not rule out an intermittent problem. Repeat the test at different times when users or monitoring data report occasional failures.

Review Google Search Console

The Page indexing report can group URLs under Server error (5xx). Inspect examples and note the last crawl time. A live test may now succeed if the incident was temporary, so compare Google’s recorded failure with the current state.

The Crawl Stats report shows host availability and server-error patterns across Googlebot requests. A spike can help connect the problem to a deployment, traffic event or infrastructure incident.

Check server and application logs

Logs often contain the evidence a crawl cannot provide: stack traces, timeout messages, exhausted workers, failed upstream requests and database errors. Match the URL and timestamp from the crawl or Search Console with the corresponding server event.

Use uptime and error monitoring

Scheduled monitoring catches failures between manual crawls. Test important page types and transactions from more than one location when regional infrastructure or a CDN may be involved.

Reproduce more than one request type

The error may affect only mobile requests, uncached visitors, logged-out sessions, specific query parameters or crawler user agents. Test the same URL under conditions that resemble the affected visitor or bot.

Check server errors in Clearcrawl

Clearcrawl helps show the visible shape of the incident. One failed URL may point to a page-specific application problem; hundreds under the same folder may reveal a shared service, template or routing failure.

Use the crawl results alongside logs and infrastructure monitoring. The crawl tells you which public routes fail; the server evidence explains why they fail.

What the error pattern can tell you

Avoid diagnosing only from the visible error page. The same design may be used for several response codes, while the server status and logs contain the useful difference. Separate server failures from genuine missing-page responses, then use a broader website health check to prioritise sitewide patterns.

How to fix 5xx server errors

1. 01
   Confirm the exact response and scope.

   Check the full URL and record the status, timestamp and response headers. Test several affected URLs and several known-good URLs. Determine whether the failure is page-specific, template-wide, regional, intermittent or site-wide.

2. 02
   Match the failure to a recent event.

   Review code deployments, plugin updates, infrastructure changes, certificate work, traffic spikes and scheduled jobs. If the error began immediately after a change, a safe rollback may restore service while the root cause is investigated.

3. 03
   Inspect the failing layer.

   Follow the request path through the CDN, firewall, load balancer, web server, application, database and external services. The status code often suggests where communication stopped, but verify it with logs and health checks.

   Common repairs include correcting configuration, fixing an unhandled exception, restoring a failed connection, increasing appropriate resource limits, optimising a slow query, and repairing DNS, proxy or CDN origin settings.

4. 04
   Restore a meaningful response.

   An available page should return 200 OK with its intended content. A permanently removed page should return 404 or 410, not a 500. A page that has moved should use the appropriate redirect. Choose the status that describes the actual state.

5. 05
   Fix links and dependencies exposed by the failure.

   If an internal route, API call or asset URL caused the incident, repair the source configuration or template. Confirm that canonical tags, sitemaps and navigation no longer reference a route that fails.

6. 06
   Add monitoring for recurrence.

   Create an alert around the failure pattern rather than relying on visitors or search reports to discover it. Monitor representative URLs, error rates, response times and the service dependency that caused the incident.

How to handle planned maintenance

Use 503 Service Unavailable when a site or page is deliberately offline for a short period. It tells crawlers that the interruption is temporary.

Add a Retry-After header when you can estimate when service will return:

HTTP/1.1 503 Service Unavailable
Retry-After: 3600

Keep the maintenance page lightweight and useful. Explain what is unavailable, when you expect it to return and how a visitor can get help.

Continue serving a valid robots.txt file. Returning 503 for robots.txt can stop crawling more broadly because crawlers cannot confirm which paths they may access.

A 503 is suitable for a short interruption, not a long-term holding page. Extended unavailability can cause URLs to be dropped from the index, so restore normal responses as soon as the service is ready.

How to verify the repair

Prove that the site is stable, rather than checking one page once.

• Request each affected URL and confirm the expected response.
• Test several URLs sharing the repaired template or service.
• Check from an uncached session and, where relevant, more than one region.
• Review application, server, proxy and CDN logs for recurring errors.
• Run another Clearcrawl crawl and confirm that 5xx findings have cleared.
• Inspect important examples in Search Console and run a live test.
• Monitor response codes and latency after normal traffic resumes.

Search Console may continue to show the historic error until Google recrawls and updates the report. Use the last crawl time to distinguish an old incident from a current failure.

For a widespread issue, use Validate Fix in the relevant Page indexing report after the live service is stable. Continue watching server-side monitoring because Search Console is not a real-time uptime system.

Common mistakes

Frequently asked questions