Noindex and indexability checker

Check whether anything on a page tells search engines to stay away: a noindex meta tag, a noindex X-Robots-Tag header, or a robots.txt rule that blocks the URL. Enter a page URL to see what each one says and whether the page can be indexed, in seconds.

We’ll scan up to 10 pages of this site and email your results. By continuing you confirm you own this website or are allowed to have it analysed. See our Terms and Privacy Policy.

What is indexability?

Before a page can appear in Google, Google has to crawl it (fetch it) and then index it (store it as a candidate for search results). Three things can stop that:

  • A robots meta tag in the page’s <head>, such as <meta name="robots" content="noindex">, which asks search engines not to index the page.
  • An X-Robots-Tag header sent by the server, such as X-Robots-Tag: noindex. It does the same job but is invisible in the page source.
  • A robots.txt rule that disallows the URL, which stops crawlers fetching the page at all.

Why indexability matters for SEO

A noindex tag on the wrong page is one of the most expensive mistakes in SEO, because nothing looks broken. The page loads fine for visitors. It just never shows up in search, and can disappear from results it used to rank for.

It happens more often than you’d think: a staging site’s “hide from search engines” setting carried over at launch, an SEO plugin option that noindexes a whole post type, or a header added by a security plugin or server configuration.

How to use this checker

  1. Enter the full address of a page you want in Google, especially one that has dropped out of search results.
  2. Read each result. The checker shows the page’s robots meta tag, its X-Robots-Tag header and whether robots.txt allows the URL.
  3. If robots.txt blocks the URL, Evisio doesn’t fetch the page, so the meta tag and header show as “couldn’t check”. Fix robots.txt first, then check again.

What Evisio checks

Not blocked in meta tags

Fails when the page’s robots meta tag contains noindex. The result shows the tag’s full content, or “None found”, which is fine: no tag means search engines may index the page.

Not blocked in response headers

Fails when the server sends an X-Robots-Tag header containing noindex. This one is easy to miss because you can’t see it by viewing the page source.

URL blocked by robots.txt

Evisio reads your robots.txt and tests the exact URL you entered against its rules, showing “Allowed” or “Disallowed”.

Blocked from crawling in robots.txt

Fails when robots.txt blocks the whole site for all crawlers. The result lists every path disallowed for all crawlers.

How to fix indexability issues

  • Noindex meta tag: on WordPress, first check Settings > Reading and untick “Discourage search engines from indexing this site”. Then check the page’s own setting in Yoast SEO (Advanced > “Allow search engines to show this page in search results?”) and the content type defaults under Yoast SEO > Settings.
  • X-Robots-Tag header: look in your server configuration (.htaccess or Nginx), your CDN, and security or caching plugins. Your host can tell you where the header comes from.
  • Blocked by robots.txt: remove or narrow the Disallow rule that matches the URL. The robots.txt checker shows the whole file’s rules.

Once fixed, ask Google to recrawl the page with the URL Inspection tool in Search Console. More detail: using the X-Robots-Tag noindex header and pages not being crawled.

Check every page on your site

A plugin setting can noindex dozens of pages at once. A free Evisio SEO audit report checks indexability on every page it scans, along with more than 80 other checks, with step-by-step instructions for each issue.

Questions

Which pages should be noindex?

Pages with no value in search: thank-you pages, cart and checkout, login and account pages, internal search results, and thin tag or archive pages.

Can I use robots.txt and noindex together?

Not on the same page. If robots.txt blocks a page, Google never fetches it, so it never sees the noindex tag, and the URL can still appear in results. To remove a page from search, allow crawling and use noindex.

The page passes. Why isn’t it in Google yet?

Passing means nothing blocks the page. Google still has to find it, through links or your sitemap, and decide it is worth indexing. New pages can take days or weeks. Search Console’s URL Inspection tool shows Google’s view.

More free SEO tools