Menu
Scan my site freeWhatsApp Usman
+92 340 8747609contact@malikusman.me

Guide covering the whole topic

Technical SEO checklist: what I check first on any site

Technical SEO makes sure Google can reach, read and index the pages you want found, and only those. I check it in a fixed order: can Google crawl the site, which pages get indexed, which ones duplicate each other, how redirects behave, how fast pages load on a phone, and whether structured data and internal links say the right things.

By Published 13 min read
On this page

Technical SEO is the part of search work that makes sure Google can find your pages, read them, and decide to show them. When it is broken, nothing else works: great content on a page Google cannot index brings no visitors at all.

This is the checklist I run on every site, in the order I run it. The order matters. A crawling problem can make it look like you have an indexing problem, and an indexing problem can make it look like your content is weak. Start at the top and work down, and you fix causes instead of chasing symptoms.

What to take from this

  • Check in order. Crawling problems hide indexing problems, and indexing problems hide everything else.
  • Search Console's Pages report is the first place to look. It shows what Google decided about each URL and why.
  • Most sites lose more to duplicate and junk URLs than to slow pages.
  • Fix site-wide causes before page-level ones. One wrong setting in a template can affect thousands of pages.

In this guide

  1. 01How to get a new website or page indexed by Google
  2. 02What an SEO audit finds, and what you get at the end
  3. 03Crawled or discovered, currently not indexed: causes and fixes

Before you start: the tools you need

You need three things, and two are free.

Google Search Console. It shows Google's own view of your site: which pages are indexed, which are not and why, how you appear in search, and any manual actions or security problems. If you do not have it set up, do that before anything else. My guide to the Search Console reports that matter explains which reports to open first.

A crawler. A crawler moves through your site the way a search engine bot does, following links and recording what it finds on each page. Screaming Frog has a free version for up to 500 URLs. Sitebulb and others work too. The crawl shows you what is linked, what returns errors, and what each page says about itself.

PageSpeed Insights. It shows how fast a page loads, using real visitor data where Google has enough of it.

With those three open, work through the sections below.

1. Can Google crawl the site?

Crawling is how Google discovers pages. If a page cannot be crawled, it cannot be indexed, unless other sites link to it, and even then Google knows almost nothing about it.

Every URL your site can produceURLs Googlebot discoversURLs it chooses to crawlURLs it indexesURLs that earn impressions
Each layer is smaller than the one above it. Duplicate and filter URLs waste crawling at the top. Diagram, not client data.

Check these first:

robots.txt. Open yoursite.com/robots.txt. Look for any Disallow line that blocks pages you want found. A single Disallow: / left over from a staging site blocks everything. Google explains how the file works in its robots.txt introduction.

Server responses. In the crawl, sort by status code. Pages you want found should return 200. Look for 5xx server errors, which tell Google your site is unreliable, and for 4xx errors on pages that are still linked.

Links Google can follow. Google follows normal links written as <a href="...">. Menus and buttons built only with JavaScript click handlers may not be followed at all. If a section of the site is missing from the crawl, check how the menu links to it.

Rendering. Many sites now build their content with JavaScript. Use the URL Inspection tool in Search Console and look at the rendered HTML. If the main content or the links are missing from it, Google may not see them either.

Crawl blocks are among the most serious problems, and among the easiest to fix once found. The common ones, including the robots.txt lines that block whole sections by accident, are in robots.txt and noindex mistakes.

2. Which pages are indexed, and why are the others not?

Being crawled is not the same as being indexed. Google crawls many pages it then decides not to keep. The Pages report in Search Console tells you which pages are indexed and gives a reason for every page that is not.

Discoverlinks, sitemapsCrawlGooglebot fetchesRenderJavaScript runsIndexstored, canonical setServeshown for searches
The order Google describes for every page. A page that fails an early step never reaches the later ones. Diagram, not client data.

Open the report and read the reasons, not just the totals. The reasons that need action most often are:

  • Excluded by noindex tag. Fine for pages you meant to hide, such as thank you pages. A problem if it lists service pages or products.
  • Crawled, currently not indexed. Google saw the page and chose not to index it, usually because it looks thin, duplicated or low value.
  • Discovered, currently not indexed. Google knows the URL but has not crawled it yet. On a big site this often means Google is spending its time on junk URLs.
  • Duplicate without user-selected canonical. Google found several versions of the page and picked one itself.
  • Not found (404). A page that used to exist, or a broken internal link.

The two "not indexed" reasons cause the most confusion. I wrote a full fix guide for them in crawled or discovered, currently not indexed. If you have new pages that Google has not picked up at all, the steps to get them in are in getting new pages indexed.

Compare the indexed count with what you expect. A site with 200 real pages and 9,000 indexed URLs has a junk problem. A site with 200 real pages and 40 indexed has a quality or access problem.

3. Duplicate URLs and canonicals

Most sites produce more URLs than they have pages. The same product appears under two categories. Tracking parameters create new addresses. Filters on a store create thousands of combinations. HTTP and HTTPS, www and non-www versions all respond.

Each extra URL splits Google's attention. Google has to choose which version to show, and it may choose the wrong one.

Check these in your crawl:

  • Does every page have a canonical tag, and does it point to itself or to the version you want indexed?
  • Do the www, non-www, HTTP and HTTPS versions all redirect to one version?
  • Do URLs with and without a trailing slash both return a page, or does one redirect?
  • Do tracking parameters such as ?utm_source= or ?fbclid= create separate indexed pages?
  • On a store, do filter and sort URLs get indexed?

Google's guide to consolidating duplicate URLs explains the signals it uses. In short: redirects are strongest, canonicals are a strong hint, and sitemaps and internal links support them. If your signals disagree, Google picks its own winner.

I wrote up the duplicate cases I see most, and how to fix each, in canonical tags explained. For stores, filter URLs are a topic of their own, covered in faceted navigation without junk pages.

One project shows how much this can matter. An online store grew organic clicks by over a three month comparison after duplicate collection paths were merged, filter combinations stopped creating indexable URLs and catalogue pages got self referencing canonicals. Several changes went live together, so the result cannot be split between them.

5. XML sitemaps

A sitemap lists the URLs you want indexed. It helps Google find pages, especially new ones or ones with few links.

Check that:

  • The sitemap is submitted in Search Console and referenced in robots.txt.
  • It lists only pages that return 200, are canonical and are not set to noindex.
  • It updates when you add or remove pages.
  • Large sites split it into several files by type, such as products, categories and articles, so problems are easy to spot.

A sitemap full of redirected, blocked or noindexed URLs tells Google your signals cannot be trusted. Clean it so every URL in it is one you want indexed.

6. Speed and Core Web Vitals

Speed matters for visitors first and rankings second. Google measures three things it calls Core Web Vitals: how fast the main content appears, how fast the page responds to a tap or click, and how much the layout jumps while it loads. The definitions and thresholds are on web.dev's Web Vitals page.

Open PageSpeed Insights for your main templates, not just the home page: one service page, one category, one product, one article. Look at the field data at the top first. It comes from real visitors on real phones. The lab test below it is useful for finding causes.

The usual causes on small and medium sites are large images that were never compressed, too many plugins or apps loading scripts on every page, web fonts that block text from showing, and cheap hosting that takes a long time to answer. Fixes worth doing first are the ones that affect every page through the template.

Speed work can swallow a lot of budget for little return, so I wrote a separate guide on which Core Web Vitals fixes are worth it.

7. Mobile rendering

Google indexes the mobile version of your site. If something is missing on mobile, such as text hidden behind tabs that never load, or links only present on desktop, Google may never see it.

Check on a real phone, not only a narrow browser window. Look for text that is too small to read, buttons too close together to tap, content wider than the screen, and pop-ups that cover the page as soon as it opens. Then compare the mobile and desktop versions of one page in the crawl: the main content, headings, links and structured data should be the same.

8. Structured data

Structured data is code that describes the page in a vocabulary search engines understand: this is a local business with these hours, this is a product at this price, these are the steps of a guide. It does not raise rankings by itself, but it helps Google understand the page and can make it eligible for richer results.

Check that each template uses the right type, that the data matches what is visible on the page, and that the Rich Results Test shows no errors. Common problems are review stars added to pages that show no reviews, prices in the markup that do not match the page, and the same Organization data repeated with different details on different pages.

For service businesses, the types that matter are covered in schema for local businesses.

10. Security and site health

Finally, the checks that are quick but can be serious:

  • HTTPS on every page, with no mixed content warnings.
  • The Security Issues and Manual Actions reports in Search Console both empty.
  • No spam pages added by a hack. Search site:yoursite.com together with words like casino or pharmacy. Hacked sites often hide pages that only search engines see.
  • Contact details and business information correct and consistent.

Platform specific checks

Each platform has its own common problems. On WordPress, the settings that cause the most trouble are the "discourage search engines" box left ticked after launch, SEO plugins that conflict with each other, and tag and author archives that create thin pages. I list them in WordPress SEO settings to check.

On Shopify, the common issues are duplicate product URLs through collections, thin tag pages, and apps that add scripts to every page. On custom built sites, the problems are usually rendering, missing canonical tags and URLs that change with every deploy.

International and multi-language sites

If your site serves more than one country or language, add one more group of checks. Each language version should have its own URL, not a version that changes based on the visitor's location. Hreflang tags should tell Google which version is for which language and country, and each version should point back to the others. The canonical tag on each version should point to itself, not to the English page.

The common mistakes are hreflang tags that point to redirected or broken URLs, versions that do not link back to each other, and a site that redirects visitors by IP address so Google, crawling mostly from the US, never sees the other versions. For a Pakistani business with an Urdu and an English version, keep both on clear separate URLs, such as /ur/ and the main site, and let the visitor choose.

How long a technical check takes

For a small business site, working through this list with Search Console and a free crawler takes a day, perhaps two if you need to look up how to read some reports. For a store with thousands of products, the crawl alone can take hours, and making sense of the results takes days.

Do the check in one sitting where you can. Write each problem down with the URL, what you saw, and the fix you think it needs. A list written as you go is far more useful than memory a week later.

What to fix first

A checklist this long can be overwhelming. Fix in this order:

  1. Anything that blocks crawling or indexing of important pages: robots.txt blocks, noindex on money pages, server errors.
  2. Site-wide duplicate problems: www and HTTP versions, parameters, filters.
  3. Redirect chains and broken internal links.
  4. Template-level speed problems.
  5. Structured data errors.
  6. Everything else, page by page.

The first two groups usually affect the most pages, so they move the most. Leave cosmetic warnings from audit tools until last. Many of them make no measurable difference.

When technical problems show up as a traffic drop

Many technical problems are found only after traffic falls. A redesign removes redirects, a plugin update adds noindex to a template, a developer blocks a folder in robots.txt. If your traffic fell suddenly, the first job is to find the date and match it to a change. I wrote the full method in diagnosing a traffic drop.

When to hand it over

You can find most of the problems in this list yourself, with Search Console and a free crawler. Fixing them is another matter. Template changes, server settings and redirect maps need someone who knows the platform, and a wrong fix can make things worse.

If you want a quick first look, the free one-page scan checks a page for crawl blocks, indexing settings and the basics of structured data. If the problems are site-wide, or you want someone to own the fixes, my technical SEO service covers the full list above. And if you want everything reviewed at once, including content and links, read what a full SEO audit covers to see what you should expect from one.

Questions people ask

What is included in technical SEO?

Crawling, indexing, duplicate URLs and canonicals, redirects, site speed and Core Web Vitals, mobile rendering, structured data, internal linking, sitemaps and robots.txt. Content and links are separate areas, though an audit usually looks at all three.

How often should I run a technical SEO check?

A full check once a year, and a quick one after every redesign, theme change, plugin update or migration. Large stores and news sites should watch the Pages report weekly.

Can I do technical SEO without a developer?

You can find most problems yourself with Search Console and a crawler. Fixing them often needs someone who can edit templates, server settings or the theme.

Which tool should I use for a technical audit?

Start with Google Search Console, which is free and shows Google's own view. Add a crawler such as Screaming Frog or Sitebulb to see the site the way a bot moves through it. Use PageSpeed Insights for speed.

, independent SEO consultant

I work alone from Pakpattan, Punjab, on technical SEO, local search, ecommerce structure and content clean ups. I write and check every post on this blog myself.

Last reviewed 3 November 2026. How I write and check these posts