All terms

What is Crawl Budget?

Crawl budget is the number of URLs a search engine will fetch from a site in a given period, determined by the site's authority, health, and demand for its content.

Deep dive: automated site audits

Small sites rarely exhaust crawl budget, but new and low-authority domains experience its cousin: crawl reluctance. Google discovers URLs (via sitemaps and links) but declines to fetch them — surfacing in Search Console as 'Discovered – currently not indexed'. The cure is demand-side, not technical: fewer, deeper pages; internal links from pages Google already values; and external links that justify the crawl.

Wasting budget still hurts at any size: redirect chains, parameter duplicates, thin near-duplicate pages, and orphaned URLs all spend fetches that deeper pages needed.

Related terms

See how your site scores on this

The free Website Verdict scan measures your SEO and AI visibility in under a minute.

Run a free scan