Products SEO & GEO Agent Paid Ads Agent Website Agent Conversion Agent
How it works Pricing FAQ Blog Book a call

Screaming Frog for content marketing: run a content audit

Updated · 7 min read

Short answer Crawl your site with Screaming Frog's free SEO Spider (up to 500 URLs), then read the Content tab: Exact Duplicates and Low Content Pages show copies and short pages. Near duplicates and Search Console data need the £199 (US$279) yearly licence. Treat each flag as a question, not a verdict: fix pages that serve a customer, merge overlaps, delete the rest.

Want help applying this to your business? Talk it through on a free 30-minute call.

Book a call

What a content audit with Screaming Frog finds

It gives you a list of every page on your site with its word count, a fingerprint of its code, and its title. That is enough to spot three kinds of weak pages. It can't judge whether a page is useful; you do that.

ProblemWhere it showsFree version?
Exact copies (same page at two addresses)Content tab > Exact DuplicatesYes
Short pagesContent tab > Low Content Pages (under 200 words by default)Yes
Pages that say "not found" but load as normalContent tab > Soft 404 PagesYes
Leftover placeholder textContent tab > Lorem Ipsum PlaceholderYes
Near copies (90% the same, e.g. town pages)Content tab > Near DuplicatesNo, needs a licence
Clicks and visits per pageSearch Console and GA4 tabsNo, needs a licence
Pages with an old dateNo filter; see belowPartly

The free version crawls up to 500 URLs per crawl. Screaming Frog's user guide says a licence "removes the 500 URL crawl limit" and "also provides access to the configuration" (the settings), plus Analytics and Search Console integration. In November 2026 the licence was £199 or US$279 a year per user, with no Canadian-dollar price listed (Screaming Frog: SEO Spider pricing).

Do you need it? For a site under a few hundred pages, the free crawl plus a free Search Console export covers a content audit. Buy the licence if you have many town or service pages that might be near copies, or more than 500 URLs.

How to run the content audit, step by step

These steps follow Screaming Frog's user guide as of November 2026.

  1. Install the SEO Spider from Screaming Frog's site (Windows, macOS or Linux).
  2. Crawl. Type your homepage into the box at the top and click Start.
  3. Open the Content tab. Click each filter in the right-hand Overview pane: Exact Duplicates, Low Content Pages, Soft 404 Pages.
  4. Licence only: switch on near duplicates. Go to Config > Content > Duplicates, tick Enable Near Duplicates, crawl, then run Crawl Analysis > Start. The Duplicate Details tab at the bottom shows each match and its similarity.
  5. Export. Use Export on each filter, or Bulk Export > Content. Save before you close: free users can't save and reopen a crawl.
  6. Add traffic. Free: export the Pages list from Search Console's Performance report and match it to your sheet by address. Licensed: Configuration > API Access > Google Search Console pulls clicks, impressions and position into the crawl (Search Console basics).
  7. Mark a decision per page in one sheet: keep, update, merge, delete.

Two settings change what the free crawl counts. Word count skips the menu and footer by default, and duplicate checks look only at pages Google may index, so a copy already pointed at its original with a canonical tag (a tag that names the main version) won't show.

Thin pages: what "Low Content" really means

It means under 200 words, not "bad". Screaming Frog calls the filter "a rough guide" and says there "isn't a minimum word count for pages in reality". Google agrees:

"The length of the content alone doesn't matter for ranking purposes" (Google Search Central: SEO Starter Guide)

So a contact page, a booking page or a photo gallery can be short and fine. A page is thin when a customer who lands on it still has to search again. Google lists that among its quality questions:

"Does your content leave readers feeling like they need to search again to get better information from other sources?" (Google Search Central: Creating helpful, reliable, people-first content)

Read each flagged page and ask: does it answer one customer question well? If not, add what the customer needs (prices, steps, photos, what's included), fold it into a stronger page, or remove it. Padding it to 200 words fixes nothing. More on length: how long a page should be.

Duplicate pages: exact and near

Exact duplicates are the same page at two addresses, often with and without a trailing slash, or http and https. These are technical: pick one address, 301-redirect (permanently forward) the other to it, and link only to the main one. Screaming Frog's guide says the other versions "should be 301 redirected to the canonical version".

Near duplicates are pages 90% alike by default, such as 30 town pages with only the town name swapped. Google's view:

"Having duplicate content on your site is not a violation of our spam policies, but it can be a bad user experience" (Google Search Central: SEO Starter Guide)

Copies on their own won't get you penalized. But pages built mainly to rank, in bulk, can fall under Google's "scaled content abuse" policy: "many pages are generated for the primary purpose of manipulating search rankings and not helping users" (Google Search Central: Spam policies). Keep the town pages where you have real jobs, photos or reviews; merge the rest into one service-area page. Full playbook: duplicate content.

Outdated pages: what a crawl can't tell you

Screaming Frog doesn't know when you wrote a post. Three ways to find old pages:

MethodCostLimit
Last-Modified column in the Internal tabFreeEmpty if your server doesn't send that date; often shows the last code change, not the last real edit
Custom Extraction (Config > Custom) to pull the date printed on each postLicenceYou must point it at where your site shows the date
Search Console: pages whose clicks fell versus last yearFreeShows decline, not age

Old isn't the problem; wrong is. Check prices, rules, staff, hours and years in the text. Don't just change the date: Google's quality questions ask, "Are you changing the date of pages to make them seem fresh when the content has not substantially changed?" To choose between updating and rewriting, see content decay: update, merge or delete.

Fix, merge or delete: the decision

The page…Do this
Gets clicks or answers a real question, but is short or datedUpdate it at the same address
Covers the same topic as a stronger pageMerge the useful parts in, then 301-redirect it there
Gets no clicks, has no links pointing at it, and serves no customerDelete it and let it return 404 (not found)
Is an exact copy at a second address301-redirect it to the main address

Merging with a redirect is safe if the target really covers the old page. Google says that when you have "consolidated content previously hosted on multiple pages to a new single page, you can redirect the older URLs to that new, consolidated page" (Google Search Central: Site moves with URL changes). Don't send everything to the homepage; Google says that "might be treated as a soft 404 error". A deleted page returning 404 is removed from Google's index (Google Search Central: HTTP status codes). When cutting helps and when it doesn't: content pruning.

After the fixes, crawl again. Redirected pages should show as 3xx in Response Codes, and nothing on your site should still link to them.

Example

Illustrative cases with invented numbers.

A dental clinic blog (US dollars)

340 URLs, so the free crawl covers it. Low Content flags 22 posts under 200 words. Search Console shows 15 had no clicks in a year and repeat longer posts; she merges them into four guides and redirects. Seven get a few clicks each; she adds prices and FAQs. Tool cost: $0.

A roofer with 60 town pages (Canadian dollars)

Exact Duplicates finds nothing, because each page names a different town. He buys a licence (US$279; Screaming Frog lists no Canadian-dollar price) and switches on near duplicates: 52 pages match at 95% or more. He keeps eight towns where he has real jobs and photos, and merges the rest into one service-area page.

An accountant's site with old tax posts

Last-Modified is empty for every page. She sorts the Search Console export by clicks, then reads the top 30 posts for outdated limits and deadlines, and updates those first.

FAQ

Is Screaming Frog free for a content audit?

Yes, for up to 500 URLs. Exact duplicates, low-content pages, soft 404s and word counts work free. Near duplicates, saved crawls and Search Console or GA4 data need the licence.

What word count counts as thin content?

None. Screaming Frog's 200-word filter is a default you can change, and Google says it has no preferred word count. Judge whether the page answers its question.

How often should I run a content audit?

When traffic drops, after a redesign, or before planning new content. A small site rarely needs more than a yearly crawl.

Different for you?

Keep learning

NextGoogle Search Console for content strategy: find the gapsUse Google Search Console to plan content: spot posts losing clicks, find new topics in your own searches, and catch two pages competing for one search.

Ask us one question

One line is enough. We reply by email.

Sources

  1. Screaming Frog: SEO Spider user guide, General
  2. Screaming Frog: SEO Spider pricing
  3. Screaming Frog: SEO Spider user guide, Tabs
  4. Screaming Frog: SEO Spider user guide, Configuration
  5. Screaming Frog: How to check for duplicate content
  6. Google Search Central: SEO Starter Guide
  7. Google Search Central: Creating helpful, reliable, people-first content
  8. Google Search Central: Spam policies for Google web search
  9. Google Search Central: Redirects and Google Search
  10. Google Search Central: Site moves with URL changes
  11. Google Search Central: HTTP status codes and network errors