Technical SEO Audit with Screaming Frog + GSC

Direct answer: export or connect Search Console first, run a scoped Screaming Frog crawl, normalize both datasets to final canonical URLs, and compare the sets. Prioritize pages that already earn impressions or clicks, important pages Google knows but the crawl missed, and crawlable indexable pages with conflicting status, canonical, directive, or sitemap signals. Re-crawl after implementation, then use Search Console to confirm Google's stored view catches up.

The tools answer different questions. Screaming Frog reports what its configured crawler can reach and extract from the live site. Search Console reports Google's processed search and indexing evidence, with reporting limits and delays. Agreement raises confidence. A disagreement is a diagnosis lead, not an automatic error.

This guide starts after you have chosen the crawler and obtained safe viewing access. If you are comparing the free and paid versions, limitations, or whether the software suits you, read the separate Screaming Frog review. If someone else will access the property, define the minimum role using the SEO audit access guide.

Know what each dataset can prove

Evidence	Useful for	Does not prove
Screaming Frog crawl	Live response, internal discovery, directives, canonicals, links, metadata, and crawl depth under the chosen configuration	That Google discovered, rendered, selected, indexed, or ranked the same URL
Performance report	Pages and queries receiving reported Google Search impressions and clicks during the selected period	A complete URL inventory, current live status, or index status for every page
Page indexing report	Google-known URL totals, indexed and non-indexed groups, and representative examples by reason	That every non-indexed URL is an error or that every example can be exported
URL Inspection	Google's stored index view for one URL and a separate live fetch test	A guarantee that the page will appear or rank in search results

Google's Performance report documentation calls this canonical URL aggregation: most performance data is assigned to the page's canonical URL, not to a duplicate URL. That means a row may not use the exact URL variant that received the original request. Preserve the raw URL, but add a normalized join key rather than assuming a missing exact-string match means missing search evidence.

1. Define the scope and protect the baseline

Record the audit boundary before changing any crawler setting. Include the public host, allowed subdomains, staging exclusions, important folders, locale rules, parameter policy, expected canonical host, and the date range for Search Console. Save the raw exports and crawl file. You need the unchanged evidence when a filter or fix later changes the counts.

Property: https://example.com/
Audit date: 2026-08-04
Search Console period: last 90 days
Included hosts: example.com
Excluded areas: /account/, /checkout/, staging.example.com
Rendering: HTML / JavaScript
Canonical policy: HTTPS, non-www, no trailing slash
Expected indexable templates: service, category, product, guide

Do not aim an unrestricted crawl at infrastructure you do not own or have permission to test. Exclude private, transactional, faceted, calendar, and infinite URL spaces before starting. Use an appropriate crawl speed, especially on small or fragile hosting.

2. Export the Google evidence before the crawl

In Search Console, choose the correct property and save these three views:

Performance, Pages. Use a meaningful comparison window, normally 90 days for a stable audit and longer when the business is seasonal. Export clicks, impressions, CTR, and position by page.
Page indexing. Record indexed and non-indexed totals, reasons, trend changes, and examples for the groups that affect intended pages.
Sitemaps. Record submitted files, last read state, and the relationship between submitted and discovered pages.

Search Console's ordinary report export is limited to the rows shown, and Google documents a maximum of 1,000 representative examples for several reports. Performance tables also omit some data for privacy and internal limits. Treat the export as valuable evidence, not a complete census. Larger properties may need the Search Console API or bulk export for fuller performance analysis.

Do not label every “Not indexed” group as a defect. Google explicitly notes that duplicates, alternates, redirects, blocked pages, and intentionally noindexed URLs can be legitimate. The audit question is whether the affected URL was intended to be a separate canonical search result.

3. Configure and run a reproducible crawl

Screaming Frog's default Spider mode is designed as a quick start, but the configuration still determines the evidence. Confirm these settings before pressing Start:

Start URL and host boundary. Use the preferred canonical origin and decide whether subdomains or external CDN hosts belong in scope.
Robots behavior. A normal crawl should respect the intended rules. If you test a second configuration that ignores them, label it clearly because it answers a different question.
Rendering. Start with HTML for a conventional server-rendered site. Use JavaScript rendering when important links, content, canonicals, or directives depend on client rendering.
Includes and excludes. Prevent session URLs, internal search, calendar loops, filter explosions, and private routes from consuming the crawl.
Sitemaps and external URL sources. Add the XML sitemap and Google-derived URLs when you need to find pages that are known elsewhere but not linked by the crawl.
Storage and speed. Choose settings the machine and server can sustain, then preserve them for the validation crawl.

After the crawl finishes, run Crawl Analysis so filters that depend on post-crawl calculations populate. Export the internal HTML inventory and the reports needed for redirect chains, canonicals, directives, response codes, crawl depth, inlinks, and sitemap membership. An issue count on the right side of the interface is not the deliverable. The affected URLs, source URLs, intended behavior, and fix owner are.

4. Connect Search Console or join the exports

The licensed integration can pull Search Analytics and optional URL Inspection data into the crawl. As verified on August 4, 2026, Screaming Frog documents the interface path as Configuration > API Access > Google Search Console. Focus URL Inspection on important templates because the API has a daily property quota and the crawler queries discovered URLs in crawl order unless you narrow the scope.

A manual join is equally valid for a small audit. Keep one row per observed URL, preserve each raw value, and create a consistent normalized key:

raw_url
normalized_url
crawl_status
indexability
declared_canonical
crawl_depth
internal_inlinks
in_sitemap
gsc_clicks
gsc_impressions
gsc_index_group
google_selected_canonical
decision
owner

Normalize only known presentation variants such as the canonical host, protocol, default port, and a documented trailing-slash rule. Do not strip meaningful parameters or merge paths because they look similar. Keep redirects and canonical targets in separate columns so the audit can reveal where variants converge.

5. Reconcile the URL sets

Observed state	Likely question	First check
Crawlable 200 + indexable + GSC performance	Is a valuable page technically stable?	Protect status, canonical, content, links, and search intent before changing it
GSC performance + missing from crawl	Is the page orphaned, redirected, blocked, outside scope, or represented by another variant?	Test the raw GSC URL, final response, inlinks, canonical, and crawl configuration
Crawlable indexable page + no performance row	Is it new, low demand, weakly linked, excluded from the export, or not selected for search?	Check launch date, Page indexing, intent, internal links, and a URL Inspection sample
Non-200 or non-indexable + GSC performance	Did a valuable URL change, or is historical/canonical data still being reported?	Inspect the date trend, redirect target, selected canonical, and replacement page
In sitemap + not internally discovered	Is the sitemap compensating for an orphan?	Find the intended hub, category, or contextual link path
Internally discovered + canonical points elsewhere	Are links promoting a duplicate?	Compare content, canonical intent, sitemap membership, and internal destinations

The most useful bucket is often “GSC evidence but missing from the crawl.” It can expose orphan pages, old URLs that still receive search activity, pages reachable only through a sitemap, or a crawler configuration that excluded an important section. The opposite bucket is not automatically bad. A live page with no Performance row may simply have no reported impressions during the selected period.

6. Prioritize impact before issue count

Do not sort the final backlog by the number of warnings. Score each finding across four dimensions:

Business and search value. Does the URL generate leads, sales, impressions, clicks, or support an important user journey?
Technical severity. Does the defect block fetching, indexing, canonical selection, navigation, or a final destination?
Pattern reach. Is this one stale page or a shared template, navigation component, routing rule, or sitemap generator?
Confidence. Do the crawl, Search Console, live page, and intended policy agree on the diagnosis?

Priority = value x severity x pattern reach x confidence

Critical: valuable URL unavailable, blocked, wrongly canonicalized, or removed without a valid replacement
High: repeatable template or navigation defect affecting intended pages
Investigate: evidence conflicts or the intended page job is unclear
Ignore: intentional state with no harmful discovery, index, or user impact

A missing meta description across 2,000 archive pages may be less urgent than one accidental noindex on the primary service page. A large redirect count may be expected history, while five redirecting links in the global navigation deserve a quick component fix.

Turn findings into fixable tickets

Each ticket should name the evidence, rule, shared source, sample URLs, owner, risk, rollback path, and validation method. “Fix 404s” is not enough. “Update the footer component so its four old routes link directly to final 200 destinations, then re-crawl every template using that footer” is testable.

Use the template-level technical SEO fixes when the diagnosis points to shared code. Use the on-page audit when one representative page needs content, heading, media, and internal-link review rather than a sitewide crawl fix.

What not to do

Do not export every issue and call it an audit. Filters report conditions. They do not know the page's job or business value.
Do not assume Screaming Frog is Google. A desktop crawl and Google's stored view differ in timing, discovery sources, rendering, and processing.
Do not treat missing Performance data as zero demand. The selected period, canonical aggregation, export limits, privacy filtering, and a new page can all affect the row set.
Do not force 100% index coverage. Legitimate duplicate, redirected, noindexed, and utility URLs should not all become canonical search results.
Do not change crawl settings between baseline and validation without recording it. Different scope can create a false improvement.
Do not request indexing before the fix is live. First verify the response, rendered output, directives, canonical, and internal links.

Validate the implementation in two stages

Stage one is immediate technical validation. Repeat the same scoped crawl, compare it with the saved baseline, and test representative affected URLs. Confirm the first response, final response, rendered directive, canonical, inlinks, sitemap membership, and template behavior. Make sure a falling issue count was caused by the fix rather than an exclusion or failed crawl.

Stage two is Google validation. Use URL Inspection on a small set of critical representatives. Compare Google's indexed view with the live test, then request indexing only when appropriate. Monitor the relevant Page indexing group and Performance pages over the following crawls. Search Console data can lag, so deployment success and Google processing are separate checkpoints.

Finding:
Affected pattern:
Baseline crawl file:
Search Console evidence:
Expected behavior:
Implementation owner:
Deployment date:
Same-config re-crawl result:
URL Inspection live result:
Google indexed-view follow-up:
Final decision: RESOLVED / MONITOR / REOPEN

Sources checked

Screaming Frog crawl configuration, Search Console integration, crawl analysis, and Google reporting and inspection behavior were verified against official documentation on August 4, 2026.

Need the crawl and Google evidence reconciled for you?

I can turn the URL sets into a prioritized implementation plan with affected patterns, owners, and validation checks, without handing over an unfiltered tool export.

Request an SEO audit