Faceted navigation without tanking your crawl budget
The rules that keep large ecommerce sites indexable.
Filters that generate URLs are the most common reason a large catalogue's organic traffic plateaus. The fix is rarely 'noindex everything' — that's a sledgehammer, and it loses real traffic alongside the bloat. The fix is a decision tree applied consistently across the site, then enforced with monitoring.
The decision tree
- Index facets with measurable search demand (over 50 monthly searches in your market).
- Canonicalise facets that don't change content meaningfully.
- Block in robots.txt only when crawl budget is genuinely capped.
- Apply rel=nofollow on internal links to non-indexed facets.
A sane default
Two-facet combinations indexable, three or more canonicalised to the two-facet parent. Review quarterly against keyword data. Document the rule in your SEO playbook so dev teams don't accidentally regress it on the next platform upgrade.
How to monitor
Pull crawl stats from Search Console weekly. If Googlebot is spending more than 30% of crawl budget on filter URLs, tighten the rules. Most large catalogues sit at 50–60% wasted crawl when they first audit — there's room.
Edge cases
Brand-by-category facets (e.g. 'Nike running shoes') usually deserve indexation regardless of facet depth. Size and colour rarely do unless inventory is unique per variant.
