On-Site Duplicate Content: Judgment Criteria and a Prevention Checklist
Key takeawayDuplicate content is not judged by literal text similarity but by whether search engines can determine which version is the primary one. The real risk is multiple URLs with equivalent content and no normalization signal pointing to a single primary version.
When this applies
- Stores where one product has multiple spec or color variant pages
- Sites where one landing page is reached through many channel parameters
- Sites where listing pages and detail pages show the same body content
- Sites where old and new URLs coexist after a domain or platform change
| Type | Needs action? | Approach | Risk level |
|---|---|---|---|
| Fully identical | Yes | Consolidate with 301 or canonical | High |
| Same body, minor differences | Pick a primary version | Canonical to the primary version plus differentiated content | Medium |
| Same template, different parameters | Yes | Canonical or noindex | High |
| Old vs. new versions after a redesign | Yes | 301 redirects | High |
| Similar-topic pages on different URLs | No | Keep independent and optimize each | Low |
| Quoted or syndicated content | Case by case | Add original value and cite the source | Medium |
Step-by-step diagnosis
- Export all indexable URLs and use content hashing or similarity tooling to find highly duplicated groups.
- Classify each group: fully identical, same body with differences, or same template only — each has a different treatment.
- For fully identical pages, prefer a 301 redirect (it passes signals through); canonical is the second choice.
- For pages with the same body but real differences, designate a primary version with a self-referencing canonical, point the rest to it, and differentiate the content.
- For parameter pages sharing a template, add a canonical or noindex, and stop generating in-site links to them.
- Confirm the site has no case of a canonical pointing to a target that does not exist.
- After fixing, spot-check with URL Inspection in GSC to confirm the normalization signal is recognized.
Common mistakes
- Using canonical as a substitute for 301 — when signals need to pass through, a 301 is more complete.
- Canonicaling every product page that shares a template to the category page.
- Treating "similar topics" as duplicate content and removing pages that could have ranked independently.
- Adding a canonical without changing the in-site links — the site keeps generating links to the duplicate pages.
- Ignoring the 404s left behind by old URLs after a redesign — a large volume of 404s slows down crawling.
FAQ
How similar does content have to be to count as duplicate?
There is no percentage threshold. The practical test: can search engines tell the intent of the two pages apart? If two pages give substantively different answers to the same query, they are not duplicates.
Should variant pages use canonical or noindex?
Variants with independent search value (e.g. separate product terms per color) get a self-referencing canonical and independent optimization; variants that exist only from parameter filtering get noindex.
Can 301 and canonical be used together?
Yes, but stay consistent within a scenario: use 301 for permanent address changes and canonical for multiple URLs at one address. Never attach both conflicting signals to one page.
Version notes2026-11-20: first release, compiled from official Google documentation and public technical sources; updated as engine rules change. Engine behaviour details follow the latest official Google documentation.
Related reading
Why Google Does Not Index Infinite Scroll Product Lists — and the FixesCommon hreflang Errors on Multilingual Sites: The Full ListCanonical Tag Conflicts Causing Duplicate Indexing: How to Fix ThemXML Sitemap Errors: Types and Fixes ComparedWhy Breadcrumb Schema Errors Make Rich Results DisappearNote: this page is a summary of publicly documented search-engine rules and is for self-check reference only. Engine behaviour is subject to the latest official Google documentation; verify against your own Search Console data.