Screaming Frog for on-page SEO: crawl and fix every page
Short answer Install Screaming Frog's free SEO Spider, enter your homepage and click Start. For a site under 500 URLs, the free version lists every page's title, meta description, headings and word count. Filter each tab for Missing and Duplicate first: those are real problems. Treat its length and word-count limits as hints, not Google rules.
Want help applying this to your business? Talk it through on a free 30-minute call.
Book a callWhat to fix first after a Screaming Frog crawl
Fix what is missing or duplicated. Leave what is merely "too long" or "too short" for later, if at all.
Screaming Frog's SEO Spider is a crawler: a desktop program that follows the links on your site, page by page, the way Google's crawler does, and puts every page's on-page details in one table. On-page SEO means the parts of each page you control: its title, its description, its headings and its text.
| Flag (tab > filter) | Real problem? | Fix |
|---|---|---|
| Page Titles > Missing | Yes. Google asks you to "make sure every page on your site has a title" | Write one per page |
| Page Titles > Duplicate | Yes, on pages you want found | Give each page its own title: service, then town or detail |
| Meta Description > Missing or Duplicate | Worth fixing on key pages | One short, specific summary per page |
| H1 > Missing | Usually: the page has no visible main heading | Add one that says what the page is |
| Content > Exact Duplicates | Yes, if both pages are meant to exist | Merge them, or redirect one to the other |
| Content > Low Content Pages | Maybe. Check whether the page answers its question | Add what customers ask, or merge into a fuller page |
| Images > Missing Alt Text | Worth fixing on photos that carry meaning | Describe the picture in a few words |
| Over 60 characters, Over 155 characters, Multiple H1 | Hints only | Change only if the visible result reads badly |
The tool's own guide says its Issues tab priorities are "not hard rules" and that "no tool can provide that as they lack context" (Screaming Frog SEO Spider user guide: Tabs).
Do you need it, and what does it cost?
For a small business site, the free version is usually enough. It crawls up to 500 URLs (web addresses) per crawl (Screaming Frog: SEO Spider pricing).
"For £199 a year you can buy a licence, which removes the 500 URL crawl limit." (Screaming Frog SEO Spider user guide: General)
| Free | Licence | |
|---|---|---|
| Price | $0 | £199 a year per user (bulk discounts from 5 users); the pricing page also shows dollar and euro options |
| URLs per crawl | 500 | Limited by your computer's memory and disk |
| Titles, descriptions, headings, broken links, exact duplicates | Yes | Yes |
| Near duplicates (pages mostly the same) | No | Yes |
| Spelling and grammar checks | No | Yes |
| Change settings, save and reopen crawls | No | Yes |
| Search Console and Analytics data beside each page | No | Yes |
Buy the licence if your site has more than 500 URLs, or you want near-duplicate checks on a site with many similar pages, such as town pages. Otherwise the free crawl, plus Google Search Console for which pages Google actually shows, covers it.
How to crawl your own site, step by step
As of November 2026, version 25.1, free version:
- Install. Download the SEO Spider from Screaming Frog's website. It runs on Windows, macOS and Ubuntu, and the user guide says it needs "a 64-bit OS with at least 4gb of RAM available".
- Start. Type your homepage, such as https://www.yoursite.com, in the "Enter URL to spider" box at the top and click Start.
- Wait for 100%. The Overview tab on the right fills in counts for each filter as it goes.
- Export before you close. Saving a crawl needs a licence. Pick a tab and filter, then click Export at the top left; it saves only the rows that filter shows (user guide: General). Do this for each list you want to work on.
- Read the Issues tab on the right. It lists every problem found, each marked Issue, Opportunity or Warning, and High, Medium or Low priority.
Finding missing and duplicate titles
Open Page Titles and use the filter drop-down. "Missing" and "Duplicate" are the two to clear. The tool explains why duplicates hurt: if every page has the same title, "it can make it more challenging for users and the search engines to understand one page from another" (user guide: Tabs).
A common cause: a theme or plugin gives every page one title, such as the business name. Two SEO plugins at once show up under "Multiple".
The "Over 60 characters" filter is the tool's limit, not Google's. Google sets no maximum. It says a long title "is truncated in Google Search results as needed, typically to fit the device width" (Google Search Central: Influencing your title links).
So a 68-character title that puts the service and town first is fine. How to word them: SEO titles.
Meta descriptions: fix the gaps, don't chase the count
The Meta Description tab has the same Missing and Duplicate filters, plus Over 155 characters and Below 70 characters.
A description is the summary under your page in search results. Google writes its own snippet much of the time:
"Google primarily uses the content on the page to automatically determine the appropriate snippet." (Google Search Central: Control your snippets in search results)
It still asks for one per page: "Identical or similar descriptions on every page of a site aren't helpful when individual pages appear in search results." Start with your service pages and homepage. More in meta descriptions.
Weak headings: what the H1 and H2 tabs show
A heading is a title inside the page. The H1 is the main one, H2s split the page into sections. The H1 tab filters are Missing, Duplicate, Over 70 characters, Multiple, Alt Text in h1 and Non-sequential.
What matters:
- Missing H1. Often the page's main heading is styled text, or an image, so nothing tells a reader or a machine what the page is about.
- Duplicate H1. Ten service pages all headed "Our Services" is the weak heading to fix. Name the service.
- Your H2 list. Click the H2 tab and read down a service page's headings. Do they cover price, process, area served and common questions? Gaps here are what to write next.
What doesn't, according to Google's SEO Starter Guide:
"Having your headings in semantic order is fantastic for screen readers, but from Google Search perspective, it doesn't matter if you're using them out of order."
Screaming Frog itself calls multiple H1s "not strictly an issue", since web standards allow them. Fix them only if they confuse readers. More in H1 tags.
Thin pages: using the word count
Open Content > Low Content Pages. By default it lists HTML pages under 200 words (user guide: Tabs). The tool warns that "there isn't a minimum word count for pages in reality". Google agrees:
"The length of the content alone doesn't matter for ranking purposes (there's no magical word count target, minimum or maximum, though you probably want to have at least one word)." (Google Search Central: SEO Starter Guide)
Use the list as a reading list, not a to-do list. Open each page and ask: would a customer get their answer here? A 150-word contact page is fine. A 120-word "Water heater repair" page with no price range, area or process is thin. Add what callers ask on the phone. If two thin pages cover one service, merge them. See content length for SEO.
Duplicate pages: exact and near
Content > Exact Duplicates lists pages that are identical, character for character; one character's difference and they don't show (user guide: Tabs). The usual small-site cause is the same page at two addresses, such as with and without a trailing slash, or a test copy left live.
Near duplicates, such as 30 town pages with only the town name swapped, need a licence: you enable them under Config > Content > Duplicates, and the default threshold is 90% similar.
Google's view on the first kind:
"Having duplicate content on your site is not a violation of our spam policies, but it can be a bad user experience and search engines might waste crawling resources on URLs that you don't even care about." (Google Search Central: SEO Starter Guide)
So it's housekeeping, not a penalty. How to pick which copy stays: duplicate content.
Example
Illustrative cases with invented numbers.
A plumber with 40 pages (US dollars)
The free crawl finds 112 URLs. Page Titles > Duplicate shows 14 service pages titled "Smith Plumbing". He rewrites them as "Water Heater Repair in Tulsa | Smith Plumbing" and so on. Low Content Pages lists 9 pages; 6 are contact and thank-you pages he leaves alone. Tool cost: $0.
A dental clinic in Ottawa (licence priced in pounds)
The site has 1,300 URLs, mostly blog posts, so the free crawl stops at 500. The owner buys a licence (£199 a year) to crawl it all and run near-duplicate checks on 25 treatment pages, and finds 8 sharing over 90% of their text.
A landscaper whose H1 is a logo
H1 > Missing lists every page. The theme puts his logo in the heading slot. One change in the theme settings gives each page a text heading.
FAQ
Is the free version of Screaming Frog enough for on-page SEO?
For sites under 500 URLs, mostly yes. It shows titles, descriptions, headings, word counts and exact duplicates; near duplicates, spelling checks and saved crawls need the licence.
What does the Issues tab's "High" priority mean?
It's the tool's estimate of potential impact. Screaming Frog says priorities are "not hard rules" and the tab "does not substitute expertise".
Should I fix every title over 60 characters?
No. Google sets no title length limit; it shortens titles to fit the screen. Fix missing and duplicate titles first.
How often should I re-crawl?
After a redesign, a theme or plugin change, or adding many pages. A theme update can reset titles or headings on every page at once.
Different for you?
- You also want broken links, redirects and blocked pages: do an SEO audit yourself.
- You want to crawl a rival's site instead: Screaming Frog competitor analysis.
- You're rewriting a service page from scratch: on-page SEO for service pages.
- Pages aren't showing in Google at all: Google Search Console for small business.
- You'd rather have the crawl and the fixes done for you: SEO & GEO Agent.