Screaming Frog for content marketing: run a content audit
Short answer Crawl your site with Screaming Frog's free SEO Spider (up to 500 URLs), then read the Content tab: Exact Duplicates and Low Content Pages show copies and short pages. Near duplicates and Search Console data need the £199 (US$279) yearly licence. Treat each flag as a question, not a verdict: fix pages that serve a customer, merge overlaps, delete the rest.
Want help applying this to your business? Talk it through on a free 30-minute call.
Book a callWhat a content audit with Screaming Frog finds
It gives you a list of every page on your site with its word count, a fingerprint of its code, and its title. That is enough to spot three kinds of weak pages. It can't judge whether a page is useful; you do that.
| Problem | Where it shows | Free version? |
|---|---|---|
| Exact copies (same page at two addresses) | Content tab > Exact Duplicates | Yes |
| Short pages | Content tab > Low Content Pages (under 200 words by default) | Yes |
| Pages that say "not found" but load as normal | Content tab > Soft 404 Pages | Yes |
| Leftover placeholder text | Content tab > Lorem Ipsum Placeholder | Yes |
| Near copies (90% the same, e.g. town pages) | Content tab > Near Duplicates | No, needs a licence |
| Clicks and visits per page | Search Console and GA4 tabs | No, needs a licence |
| Pages with an old date | No filter; see below | Partly |
The free version crawls up to 500 URLs per crawl. Screaming Frog's user guide says a licence "removes the 500 URL crawl limit" and "also provides access to the configuration" (the settings), plus Analytics and Search Console integration. In November 2026 the licence was £199 or US$279 a year per user, with no Canadian-dollar price listed (Screaming Frog: SEO Spider pricing).
Do you need it? For a site under a few hundred pages, the free crawl plus a free Search Console export covers a content audit. Buy the licence if you have many town or service pages that might be near copies, or more than 500 URLs.
How to run the content audit, step by step
These steps follow Screaming Frog's user guide as of November 2026.
- Install the SEO Spider from Screaming Frog's site (Windows, macOS or Linux).
- Crawl. Type your homepage into the box at the top and click Start.
- Open the Content tab. Click each filter in the right-hand Overview pane: Exact Duplicates, Low Content Pages, Soft 404 Pages.
- Licence only: switch on near duplicates. Go to Config > Content > Duplicates, tick Enable Near Duplicates, crawl, then run Crawl Analysis > Start. The Duplicate Details tab at the bottom shows each match and its similarity.
- Export. Use Export on each filter, or Bulk Export > Content. Save before you close: free users can't save and reopen a crawl.
- Add traffic. Free: export the Pages list from Search Console's Performance report and match it to your sheet by address. Licensed: Configuration > API Access > Google Search Console pulls clicks, impressions and position into the crawl (Search Console basics).
- Mark a decision per page in one sheet: keep, update, merge, delete.
Two settings change what the free crawl counts. Word count skips the menu and footer by default, and duplicate checks look only at pages Google may index, so a copy already pointed at its original with a canonical tag (a tag that names the main version) won't show.
Thin pages: what "Low Content" really means
It means under 200 words, not "bad". Screaming Frog calls the filter "a rough guide" and says there "isn't a minimum word count for pages in reality". Google agrees:
"The length of the content alone doesn't matter for ranking purposes" (Google Search Central: SEO Starter Guide)
So a contact page, a booking page or a photo gallery can be short and fine. A page is thin when a customer who lands on it still has to search again. Google lists that among its quality questions:
"Does your content leave readers feeling like they need to search again to get better information from other sources?" (Google Search Central: Creating helpful, reliable, people-first content)
Read each flagged page and ask: does it answer one customer question well? If not, add what the customer needs (prices, steps, photos, what's included), fold it into a stronger page, or remove it. Padding it to 200 words fixes nothing. More on length: how long a page should be.
Duplicate pages: exact and near
Exact duplicates are the same page at two addresses, often with and without a trailing slash, or http and https. These are technical: pick one address, 301-redirect (permanently forward) the other to it, and link only to the main one. Screaming Frog's guide says the other versions "should be 301 redirected to the canonical version".
Near duplicates are pages 90% alike by default, such as 30 town pages with only the town name swapped. Google's view:
"Having duplicate content on your site is not a violation of our spam policies, but it can be a bad user experience" (Google Search Central: SEO Starter Guide)
Copies on their own won't get you penalized. But pages built mainly to rank, in bulk, can fall under Google's "scaled content abuse" policy: "many pages are generated for the primary purpose of manipulating search rankings and not helping users" (Google Search Central: Spam policies). Keep the town pages where you have real jobs, photos or reviews; merge the rest into one service-area page. Full playbook: duplicate content.
Outdated pages: what a crawl can't tell you
Screaming Frog doesn't know when you wrote a post. Three ways to find old pages:
| Method | Cost | Limit |
|---|---|---|
| Last-Modified column in the Internal tab | Free | Empty if your server doesn't send that date; often shows the last code change, not the last real edit |
| Custom Extraction (Config > Custom) to pull the date printed on each post | Licence | You must point it at where your site shows the date |
| Search Console: pages whose clicks fell versus last year | Free | Shows decline, not age |
Old isn't the problem; wrong is. Check prices, rules, staff, hours and years in the text. Don't just change the date: Google's quality questions ask, "Are you changing the date of pages to make them seem fresh when the content has not substantially changed?" To choose between updating and rewriting, see content decay: update, merge or delete.
Fix, merge or delete: the decision
| The page… | Do this |
|---|---|
| Gets clicks or answers a real question, but is short or dated | Update it at the same address |
| Covers the same topic as a stronger page | Merge the useful parts in, then 301-redirect it there |
| Gets no clicks, has no links pointing at it, and serves no customer | Delete it and let it return 404 (not found) |
| Is an exact copy at a second address | 301-redirect it to the main address |
Merging with a redirect is safe if the target really covers the old page. Google says that when you have "consolidated content previously hosted on multiple pages to a new single page, you can redirect the older URLs to that new, consolidated page" (Google Search Central: Site moves with URL changes). Don't send everything to the homepage; Google says that "might be treated as a soft 404 error". A deleted page returning 404 is removed from Google's index (Google Search Central: HTTP status codes). When cutting helps and when it doesn't: content pruning.
After the fixes, crawl again. Redirected pages should show as 3xx in Response Codes, and nothing on your site should still link to them.
Example
Illustrative cases with invented numbers.
A dental clinic blog (US dollars)
340 URLs, so the free crawl covers it. Low Content flags 22 posts under 200 words. Search Console shows 15 had no clicks in a year and repeat longer posts; she merges them into four guides and redirects. Seven get a few clicks each; she adds prices and FAQs. Tool cost: $0.
A roofer with 60 town pages (Canadian dollars)
Exact Duplicates finds nothing, because each page names a different town. He buys a licence (US$279; Screaming Frog lists no Canadian-dollar price) and switches on near duplicates: 52 pages match at 95% or more. He keeps eight towns where he has real jobs and photos, and merges the rest into one service-area page.
An accountant's site with old tax posts
Last-Modified is empty for every page. She sorts the Search Console export by clicks, then reads the top 30 posts for outdated limits and deadlines, and updates those first.
FAQ
Is Screaming Frog free for a content audit?
Yes, for up to 500 URLs. Exact duplicates, low-content pages, soft 404s and word counts work free. Near duplicates, saved crawls and Search Console or GA4 data need the licence.
What word count counts as thin content?
None. Screaming Frog's 200-word filter is a default you can change, and Google says it has no preferred word count. Judge whether the page answers its question.
How often should I run a content audit?
When traffic drops, after a redesign, or before planning new content. A small site rarely needs more than a yearly crawl.
Different for you?
- You found copies and need to decide which to keep: duplicate content.
- Pages are losing traffic over time: content decay: update, merge or delete.
- You're thinking of cutting many pages at once: content pruning.
- You want to compare your pages to a rival's: Screaming Frog competitor analysis.
- Deleted pages now show as 404 errors: 404 errors in Search Console.
- You'd rather have the audit and fixes done for you: SEO & GEO Agent.