The Real Reasons Behind a Surge of 404 Pages
Key takeawayA 404 by itself is not an error, but a large volume of 404s reveals systemic problems in the site's link structure, routing rules, or redesign process. It continuously consumes crawler resources and makes search engines lower the site's crawl frequency.
When this applies
- Sites where logs or GSC show a high proportion of 404s
- Sites where old URLs were not redirected after a redesign
- Stores where filtering, paging, and faceted navigation generate huge numbers of invalid combinations
- Sites with external links pointing to deleted pages
| Source | How to identify | Severity | How to handle |
|---|---|---|---|
| Old URLs left from a redesign | Log paths concentrated in the old directory structure | High | 301 each to the corresponding new page |
| Parameter combinations | Paths with many query strings and no matching content | High | Canonical or noindex |
| Pagination overrun | ?page=999 beyond the actual page count | Medium | Cap pagination depth |
| External links to deleted pages | Referer shows external origins | Medium | Contact the source or build a 301 |
| Case / trailing slash differences | 404 triggered only by case or a slash | High | Normalize at the server level |
| Misspelled internal links | Referer is an internal page | Medium | Fix the links |
| Intentionally deleted pages | No internal or external links, long-term 404 | Low | Switch to returning 410 |
Step-by-step diagnosis
- Group the 404s in your logs by path prefix to find the prefixes contributing the most volume.
- Cross-check against redesign records to confirm which prefixes belong to old paths — that batch usually deserves 301s.
- Check for pseudo-404s caused by case, trailing slash, or URL-encoding differences; these are extremely cheap to fix and usually numerous.
- Add canonicals or noindex to parameter-driven 404s, and fix the in-site entry points that generate those links.
- For pages that are truly removed with no links and no traffic, return 410 instead of 404 — 410 means "permanently removed" and makes crawlers stop probing sooner.
- Measure 301 coverage over the redesign-legacy portion and prioritize URLs that have external links or historical rankings.
- Monitor the weekly 404 ratio as a long-term metric.
Common mistakes
- Believing that more 404s mean an active site — 404s only consume resources; they bring no traffic.
- Redirecting all 404s to the homepage — that is soft 404 behavior and gets judged as low quality.
- Using the 404 page to catch traffic (redirecting to the homepage) instead of returning the correct status code.
- Launching a redesign without a 301 mapping — the most common origin of a 404 explosion.
- Ignoring server-level normalization (case, slashes), so the same resource exists at multiple URLs with only one accessible.
FAQ
What is the difference between returning 410 and 404?
404 means "not found for now"; 410 means "permanently removed." 410 makes crawlers stop retrying sooner and is the correct choice when deleting pages.
What should a 404 page contain?
A search box and the main section entry points, helping users and crawlers find other pages. The precondition is that the status code genuinely is 404.
Why does 404 volume explode right after a redesign?
Because old URLs lose their corresponding pages while internal and external links still point to them. That is exactly why a 301 mapping is mandatory.
Version notes2026-11-20: first release, compiled from official Google documentation and public technical sources; updated as engine rules change. Engine behaviour details follow the latest official Google documentation.
Related reading
Why Google Does Not Index Infinite Scroll Product Lists — and the FixesCommon hreflang Errors on Multilingual Sites: The Full ListCanonical Tag Conflicts Causing Duplicate Indexing: How to Fix ThemXML Sitemap Errors: Types and Fixes ComparedWhy Breadcrumb Schema Errors Make Rich Results DisappearNote: this page is a summary of publicly documented search-engine rules and is for self-check reference only. Engine behaviour is subject to the latest official Google documentation; verify against your own Search Console data.