Crawl budget matters when a large site gives search engines too many low-value URLs to sort through. For multi-location brands, thin city pages are a common cause. We need to know when to improve, consolidate, noindex, or redirect them.
TL;DR
- Crawl budget matters most for large or fast-changing sites, not every multi-location website.
- Thin city pages should be reviewed with Search Console, crawl data, and log files before any cleanup.
- Noindex can help manage low-value pages, but stronger content and cleaner internal links usually matter more.
Does crawl budget actually matter for your site?
Crawl budget is the amount of crawling Google can and wants to do on a site. Google’s crawl budget guidance says most small and medium sites do not need to worry much about it. If important pages are crawled the same day they are published or updated, crawl budget is probably not the main issue.
For multi-location websites, crawl budget becomes more important when the URL footprint grows faster than the value of the content. This can happen when a brand creates pages for every location, city, suburb, service, category, and filter combination.
A site with 200 locations may start with 200 useful location pages. But if it also creates service pages for every city, state pages, nearby-area pages, filtered directory pages, and duplicate landing pages, the site can quickly grow into thousands of URLs. Some of those pages may help users. Others may only repeat the same template with a city name changed.
That is where crawl budget multi-location SEO becomes important. The issue is not just page count. The issue is whether Google is spending time on pages that do not deserve to be indexed.
Crawl budget is worth reviewing when important location pages are slow to appear in search, new pages stay in “Discovered, currently not indexed,” or Googlebot keeps crawling low-value URL patterns. It is also worth reviewing when large groups of city pages receive no impressions, no clicks, and no clear business value.
For enterprise SEO teams, the question is not only “Can Google crawl this site?” The better question is: Are we making it easy for Google to find and prioritize the pages that matter most?
Identifying low-value pages with GSC and log files
Before we noindex location pages or thin city pages, we need to identify the real problem. A cleanup based on guesswork can remove pages that still have value.
A low-value city page is usually a page that targets a local query but does not provide enough useful local information. It may have a unique URL, title, and H1, but the body copy is nearly the same as hundreds of other pages. It may not include a real location, local staff, service details, reviews, photos, FAQs, or anything that helps users make a decision.
Google Search Console is the first place to look. The Page Indexing report can show whether city pages are indexed, crawled but not indexed, discovered but not indexed, or excluded by a noindex tag. The Performance report can show whether those pages receive impressions, clicks, and real queries.
The strongest insights usually come from grouping pages by URL pattern instead of checking one page at a time. For example, we can compare /locations/, /service-areas/, /city/, and /services/{service}/{city}/ pages. This helps show whether one template is creating most of the indexation waste.
Log files add another layer. Search Console shows how pages perform in Google Search. Log files show what crawlers actually request from the server. Tools such as Oncrawl, Botify, and Screaming Frog can help teams review crawl behavior, noindex directives, and large URL patterns without manually checking every page. Screaming Frog’s documentation, for example, explains that noindex directives can be reviewed through the tool’s directives reporting.
A simple audit should compare three signals:
| Source | What it helps us understand |
| Google Search Console | Whether pages are indexed, discovered, crawled, or receive search demand |
| Crawl data | Whether pages are indexable, duplicated, internally linked, or technically weak |
| Log files | Whether Googlebot is spending time on the right URL patterns |
A page becomes a stronger cleanup candidate when all three signals point in the same direction. For example, a city page with no impressions, no clicks, no unique content, no internal value, and regular bot activity may be wasting crawl attention.
Decision tree: improve, consolidate, noindex, redirect
Not every thin page should be noindexed. Some pages should be improved because they have search demand. Some should be consolidated because they overlap with stronger pages. Some should be redirected because they are obsolete. Others should stay live for users, but be removed from the index.
Use this decision tree when reviewing a page type or URL pattern.
Start: Review the city page or location page pattern
│
├── Does the page have real search value?
│ Examples: impressions, clicks, rankings, backlinks, conversions, or clear local demand
│
│ ├── Yes
│ │ └── Improve the page
│ │ Add stronger local content, better internal links, clearer services,
│ │ unique proof, and more useful location details.
│ │
│ └── No
│ │
│ ├── Is there a stronger page that serves the same intent?
│ │
│ │ ├── Yes
│ │ │ └── Consolidate
│ │ │ Merge useful content into the stronger page.
│ │ │ Update canonicals and internal links.
│ │ │
│ │ └── No
│ │ │
│ │ ├── Does the page still need to exist for users?
│ │ │ Examples: paid campaigns, filters, internal navigation,
│ │ │ support journeys, or account-related flows
│ │ │
│ │ │ ├── Yes
│ │ │ │ └── Noindex
│ │ │ │ Keep the page accessible, but remove it from search results.
│ │ │ │
│ │ │ └── No
│ │ │ │
│ │ │ ├── Is there a relevant replacement page?
│ │ │ │
│ │ │ │ ├── Yes
│ │ │ │ │ └── Redirect
│ │ │ │ │ Send users and crawlers to the best-matching page.
│ │ │ │ │
│ │ │ │ └── No
│ │ │ │ └── Remove
│ │ │ │ Return a proper 404 or 410 if the page is truly gone.
The decision should be made by page purpose, not only by traffic. A new location page may have low traffic because it has not had enough time to perform. A city page with low traffic may still be useful if it supports a real service area and can be improved.
When to improve
Improve the page when it has a reason to rank. This may include search impressions, backlinks, leads, local demand, or a clear connection to a real branch or service area.
A stronger city page should give users something they cannot get from a generic template. That may include nearby locations, service availability, local reviews, local photos, pricing notes, booking options, or city-specific FAQs.
When to consolidate
Consolidate when several pages compete for the same intent. For example, a site may have a city page, a service-area page, and several service plus city pages that all say nearly the same thing.
In that case, one strong page is usually better than several weak ones. Google’s canonical guidance explains that when duplicate or similar URLs exist, Google chooses a canonical URL, and site owners can help by sending consistent canonical signals.
Consolidation may involve merging content, updating internal links, choosing a canonical URL, or redirecting old pages when there is a clear replacement.
When to noindex
Noindex is useful when a page still helps users but should not appear in search results. Google’s noindex guidance explains that a noindex rule prevents a page from appearing in Search, but Google must be able to crawl the page to see that directive.
A noindex tag may make sense for thin city pages that support navigation but do not deserve organic rankings. It may also fit filtered location pages, internal search pages, paid landing pages, or duplicate support pages.
A basic noindex tag looks like this:
<meta name=”robots” content=”noindex,follow”>
Do not block the page in robots.txt if Google needs to read the noindex tag. If crawling is blocked, Google may not see the directive.
When to redirect or remove
Redirect a page when it no longer needs to exist, and there is a relevant replacement. A thin city page might redirect to a stronger city page, nearby branch page, or service page if that destination satisfies the same user intent.
Avoid sending every removed page to the homepage. That rarely helps users and can create soft 404 issues.
If there is no relevant replacement and the page is truly obsolete, a proper 404 or 410 may be cleaner.
Internal linking adjustments after pruning
After pruning, internal links need to match the new strategy. If old links keep pointing to noindexed, redirected, or weak pages, the site continues sending mixed signals.
Internal linking should guide users and crawlers toward the best remaining pages. A state page should point to strong location pages. A service page should link to relevant service-area or branch pages. A location page should link to nearby locations only when those links are useful.
This is also where XML sitemaps need cleanup. A sitemap should include URLs we want indexed. It should not keep noindexed pages, redirected URLs, blocked pages, duplicate variants, or thin URLs that were already removed from the organic strategy.
The same logic applies to navigation, footer links, breadcrumbs, related-location modules, and HTML sitemaps. After a cleanup, those links should point to the preferred indexable URLs.
Google’s canonical documentation also recommends linking consistently to the preferred canonical URL, which supports a cleaner consolidation strategy.
Internal linking cleanup is often where crawl budget gains become more realistic. Noindex alone can remove a page from search results, but strong internal links help show which pages deserve attention.
Monitoring impact post-cleanup
A cleanup is not finished when noindex tags or redirects go live. We need to monitor how Google responds.
Start with technical checks. Confirm that noindex tags appear in the rendered HTML or HTTP headers. Check that important pages were not accidentally noindexed. Make sure redirects point to the relevant pages and that canonical tags point to the correct preferred URLs.
Then review Google Search Console. The Page Indexing report should show whether Google detected noindex tags. Google’s noindex guidance also recommends using URL Inspection to test what Googlebot received when crawling the page.
The most useful monitoring compares page groups before and after cleanup.
| What to monitor | Why it matters |
| Indexed pages by URL pattern | Shows whether the index is becoming cleaner |
| Crawl stats and log files | Shows whether crawler attention is shifting |
| Impressions and clicks | Shows whether stronger pages are gaining visibility |
| Sitemap indexation | Shows whether submitted URLs are being indexed |
| Conversions from location pages | Shows whether SEO cleanup supports the business |
We should not expect instant results. Google needs time to recrawl noindexed pages, process redirects, update canonical signals, and understand the new internal linking structure.
The goal is not simply to reduce page count. The goal is to create a stronger set of pages that are easier to crawl, easier to understand, and more useful to searchers.
For multi-location SEO, that usually means fewer thin city pages, stronger branch pages, cleaner directory pages, and more intentional internal linking.
FAQ
Does crawl budget matter for every multi-location site?
No. Crawl budget usually matters most for large sites, fast-changing sites, or sites with many low-value URLs. Smaller sites should still keep sitemaps clean and monitor indexing, but crawl budget may not be the main issue.
When should we noindex location pages?
Noindex a page when it still needs to exist for users, but should not appear in search results. If the page has real local demand, links, conversions, or useful content, improving or consolidating it may be better.
Should thin city pages be deleted or noindexed?
It depends on the purpose of the page. If it is useful for users but not for search, noindex may fit. If it has a strong replacement, redirect it. If it has no value and no replacement, a proper 404 or 410 may be cleaner.
Can noindex save crawl budget?
Noindex helps control indexing, but it does not fully stop crawling. Google still needs to crawl the page to see the noindex directive. For crawl budget, noindex works best with cleaner internal links, cleaner sitemaps, and better URL management.
Should noindexed pages stay in the XML sitemap?
Usually, no. XML sitemaps should focus on URLs we want indexed. Noindexed, redirected, blocked, and duplicate URLs should generally be removed.
Is robots.txt better than noindex?
They solve different problems. Robots.txt blocks crawling. Noindex blocks indexing after the page is crawled. Do not use robots.txt to block a page if Google needs to see the noindex directive.
How many thin pages are too many?
There is no fixed number. The concern grows when low-value pages make up a large share of crawlable URLs, receive little organic value, and compete with stronger pages for crawl attention.
What should we monitor after pruning city pages?
Monitor indexing, crawl stats, log files, impressions, clicks, sitemap health, and conversions by URL pattern. Also, check that important location pages were not accidentally noindexed or redirected.
Sources
- Google Search Central, Crawl Budget Management for Large Sites
- Google Search Central, Block Search indexing with noindex
- Google Search Central, Consolidate duplicate URLs
- Google Search Central, Robots meta tag and X-Robots-Tag specifications
- Screaming Frog, Noindex directive issue
- Botify, Evaluating Noindex Pages
- Oncrawl, Log Analyzer
Paul Warren is the co-founder and Head of SEO at the Local Agency and has over 15 years of enterprise SEO experience.