Products SEO & GEO Agent Paid Ads Agent Website Agent Conversion Agent
How it works Pricing FAQ Blog Book a call

Screaming Frog technical SEO audit, step by step

Updated · 8 min read

Short answer Install the SEO Spider, enter your homepage and click Start. On a small site the defaults are fine; the free version crawls 500 URLs. When it finishes, open the Issues tab, then rank fixes by what Google does with each: blocked or noindexed key pages first, then server errors, broken links, redirect loops and canonicals. Alt text comes last.

Want help applying this to your business? Talk it through on a free 30-minute call.

Book a call

Screaming Frog technical SEO audit: the steps

Crawl, read, rank, fix, re-check. Screaming Frog's SEO Spider is a desktop crawler: it follows every link on your site, as Google's crawler does, and records what each address returns. These steps follow its user guide as of November 2026.

StepWhere in Screaming FrogFree version?
1. Install and enter your homepage"Enter URL to spider" box, then StartYes
2. Change settings only if your site needs itConfiguration menu; File > SettingsMost settings need a licence
3. Run crawl analysis when the crawl endsCrawl Analysis > StartYes, for the filters you have
4. Open the Issues tabRight-hand paneYes
5. Rank the problems by what Google does with themThe table further downYour judgment
6. Export a fix listExport button, Bulk Export, ReportsYes
7. Re-crawl the fixed pagesList mode: paste the addressesYes

The software needs a 64-bit computer with at least 4 GB of RAM and runs on Windows, macOS and Linux (Screaming Frog user guide: General). For the general audit method, tool aside, see technical SEO audit.

Do you need the paid licence for an audit?

Not for a typical small business site. The free version crawls up to 500 URLs at a time. The licence costs £199 a year per user; the US-dollar price shown in November 2026 was $279. There's no Canadian-dollar price (Screaming Frog: pricing).

Audit taskFreePaid
Broken links, errors and redirectsYesYes
Titles, meta descriptions, meta robots (noindex)YesYes
Exact duplicate pagesYesYes
More than 500 URLsNoLimited by your memory and disk
Changing crawl settings, including checking your XML sitemapNoYes
JavaScript renderingNoYes
Near-duplicate pages, structured data checksNoYes
Search Console, Analytics and PageSpeed dataNoYes
Saving crawls, comparing crawls, scheduled crawlsNoYes

If the free crawl stops at 500 before your pages run out, Google Search Console covers much of the rest at no cost: indexing, sitemaps and server errors, as Google sees them. See Search Console technical SEO audit.

Which settings to change before you crawl

On a small site, usually none. The defaults already save crawl data to disk with 2 GB of memory, obey your robots.txt (the file that tells crawlers what to skip) and send 5 requests at a time "to not overload servers" (user guide: General and Configuration).

SettingDefaultChange it when
Storage and memory (File > Settings)Database storage, 2 GB RAMYou see the "running out of memory" warning. The guide suggests 4 GB for crawls up to 2 million URLs
Rendering (Configuration > Spider > Rendering)Text Only: reads the raw HTMLYour pages build their text or links with JavaScript, so the crawl shows them nearly empty
robots.txt (Configuration > robots.txt)Respect robots.txtYou want to see what your robots.txt blocks. "Ignore robots.txt but report status" crawls those pages and still marks them
XML sitemaps (Configuration > Spider > Crawl)OffYou want the Sitemaps tab: tick "Crawl Linked XML Sitemaps"
Search Console (Configuration > API Access)OffYou want Google's index status per page: up to 2,000 URLs per property a day
Crawl analysis (Crawl Analysis > Configure)Check the box thereTick "Auto Analyse At End of Crawl" so you don't forget

Leave JavaScript rendering off unless you need it. If your text shows in the page's source code, the default is enough; Google notes that crawling the HTML "works well for classical websites or server-side rendered pages" (Google Search Central: JavaScript SEO basics). If you suspect a problem, see JavaScript SEO audit.

Run the crawl and know when it's done

Type your homepage address, click Start, and wait for the progress bar to reach 100%.

Then run Crawl Analysis > Start. Most tabs fill in during the crawl, but some filters only appear after this step: the sitemap filters, Link Score (a 0–100 estimate of how much internal linking a page gets) and near duplicates. Filters still waiting say "Crawl Analysis Required" (user guide: Configuration).

How to read the Issues tab

The Issues tab is a to-do list, not a ranking of what hurts you. It sorts findings into issues, warnings and opportunities, each with High, Medium or Low priority. Screaming Frog itself says those priorities come from general best practice:

"They are not hard rules for what should be prioritised in your SEO strategy, or must be 'fixed' in your SEO audit as they lack context." (Screaming Frog: SEO Spider issues)

Click each issue and ask: does it touch a page that brings in customers? A missing description on an old tag page can wait; a noindex tag on your main service page can't.

What to fix first

Rank by what Google does with each problem, not by how many rows it has.

OrderProblemWhere to find itWhy it matters
1Key page blocked or set to noindexResponse Codes > Blocked by Robots.txt; Directives > NoindexDrops a noindex page from results
2Server errors (5xx)Response Codes > Server Error (5xx)Slows crawling; indexed pages are "eventually dropped"
3Internal links to missing pages (4xx)Response Codes > Client Error (4xx); Bulk Export: inlinks to 4xx pagesRemoves 4xx pages from the index
4Redirect loops and long chainsResponse Codes > Redirect Loop, Redirect Chain; Reports > RedirectsFollows up to 10 hops, then gives up
5Key page canonicalised elsewhere, or no canonicalCanonicals > Canonicalised, MissingPicks its own version when you don't
6Sitemap lists pages that can't be indexedSitemaps > Non-Indexable URLs in Sitemap (licence)Treats a sitemap as a weak hint
7Missing or duplicate titles on key pagesPage Titles > Missing, DuplicateWeaker search listings for that page; Screaming Frog calls unique titles "critical"
8Meta descriptions, alt text, multiple H1sMeta Description; Images > Missing Alt Text; H1Low risk; fix on key pages when you next edit them

Google's own words behind rows 1 to 6:

"For the noindex rule to be effective, the page or resource must not be blocked by a robots.txt file, and it has to be otherwise accessible to the crawler." (Google Search Central: Block search indexing with noindex)

So a page you want gone needs noindex and no robots.txt block; a page you want found needs neither.

"Google's crawlers follow up to 10 redirect hops." (Google Search Central: HTTP status codes and network errors)

A chain of two or three redirects still works; collapse it to one hop when you touch it. A loop never resolves, so fix it now. See 301 redirect problems.

"[I]f you don't specify a canonical URL, Google will identify which version of the URL is objectively the best version to show to users in Search." (Google Search Central: How to specify a canonical URL)

On meta descriptions, Google says snippets come "primarily" from the page content and it uses the description only sometimes (Google Search Central: snippets). A missing one is rarely urgent.

Turn the crawl into a fix list

Three ways to get data out (user guide: General):

  • Export button (top left): saves the tab and filter you're viewing.
  • Bulk Export: for a broken link, the pages that link to it, which is what you edit.
  • Reports: summaries such as Redirect Chains. Exports can go straight to Google Sheets.

Keep one sheet with five columns: URL, problem, fix, who does it, date done. Put customer-facing pages at the top. When fixes are live, switch to List mode, paste only those addresses and crawl again to confirm. Then re-crawl the whole site after redesigns, plugin changes or a drop in traffic; see how often to do an SEO audit.

Example

Illustrative cases with invented numbers.

A 60-page electrician site (US dollars)

The free crawl finds 410 URLs. The Issues tab shows 140 rows. Two matter: the "Commercial wiring" page carries noindex, left over from the build, and 9 pages link to a deleted coupon page. Removing the tag and repointing the links takes an hour. Tool cost: $0.

A Toronto law office with 1,800 URLs (Canadian dollars)

The free crawl stops at 500. Rather than buy a licence for one audit, the office reads Search Console's page indexing report and crawls only its 40 practice-area pages in List mode. It finds a redirect loop on the old "contact-us" address.

A dental group re-checking monthly

With a licence, it schedules a crawl on the 1st of each month and compares it with last month's. A plugin update adds noindex to the blog. The next comparison shows it, weeks before traffic would.

FAQ

Is the free version enough for a technical SEO audit?

Under 500 URLs, yes: errors, redirects, noindex, titles and exact duplicates. Sitemap checks, JavaScript rendering and Search Console data need the licence.

Will crawling my own site slow it down?

Rarely. The default is 5 requests at a time. If your host struggles, a licence lets you slow it to one URL per second.

Does Screaming Frog tell me which pages Google indexed?

Not alone. With a licence and Search Console connected, it pulls Google's index status for 2,000 URLs a day. Otherwise, use Search Console's indexing report.

Different for you?

Keep learning

NextSE Ranking SEO audit: a quick 30-minute checkRun an SE Ranking website audit and turn it into a fix list in 30 minutes: what to read first, what to ignore, what it costs, and the free alternatives.

Ask us one question

One line is enough. We reply by email.

Sources

  1. Screaming Frog: SEO Spider user guide, General
  2. Screaming Frog: SEO Spider user guide, Configuration
  3. Screaming Frog: SEO Spider user guide, Tabs
  4. Screaming Frog: SEO Spider issues
  5. Screaming Frog: SEO Spider pricing
  6. Google Search Central: How HTTP status codes and network errors affect Google Search
  7. Google Search Central: Block search indexing with noindex
  8. Google Search Central: How to specify a canonical URL
  9. Google Search Central: Control your snippets in search results
  10. Google Search Central: Understand the JavaScript SEO basics