Crawl Budget

The number of pages Googlebot will crawl on your site within a given time period.

Crawl budget is determined by two factors: crawl rate limit (how fast Googlebot can crawl without overloading your server) and crawl demand (how many pages Google wants to crawl based on PageRank and URL freshness). For most small and medium sites, crawl budget is not a concern — Google will crawl all important pages. It becomes critical for large sites with thousands or millions of URLs.

To improve crawl budget efficiency: block low-value URLs in robots.txt (faceted navigation, filtered listings, thin pages), fix crawl errors that waste crawl budget, improve internal linking to important pages, and use XML sitemaps to signal priority pages. Orphan pages (no internal links pointing to them) are rarely crawled regardless of their quality.

Google Search Console → Crawl Stats report shows daily crawl requests, average response time, and how crawl budget is being spent across your site.