Skip to main content

Guides / Fix it: guides for the errors people actually hit

Site disappeared from Google: what to check

If your site vanished from Google, work through the causes in order of how quickly they can be ruled out: manual action, noindex, robots.txt, canonical, a hack or blacklisting, server errors, then the domain itself. Six of the seven are visible from outside the site in a few minutes, and the order below is chosen so that the cheapest checks come first.

First, confirm it is actually gone

Search for your exact homepage URL, then run site:example.com. No results at all for either points at a site-wide cause and the list below applies. Results for the homepage but not for a section points at one template or one path, which narrows the search considerably. Results that are all present but ranking worse than last month is a different problem: that is a ranking change, not a deindexing, and none of these steps address it.

1. Check for a manual action

Search Console, Security and Manual Actions, Manual actions. A manual action means a human reviewer applied a penalty, and it names the reason and the scope. Nothing else on this list will fix a site under one. The path back is to correct what the notice describes and submit a reconsideration request, which takes days to weeks. This is first on the list because it is one click and it makes the other six irrelevant.

2. Check for noindex

The most common cause of a whole site disappearing without anything appearing broken:

curl -sI https://example.com/ | grep -i x-robots-tag
curl -s https://example.com/ | grep -i 'name="robots"'

Either command printing noindex is your answer. Check a page from each major template, not just the homepage, because the leak is often confined to one route. The mechanism and the usual ways it ships are covered in what noindex does.

3. Check robots.txt

Open https://example.com/robots.txt. A Disallow: / under User-agent: * blocks the entire site from crawling. Confirm the file itself returns 200 as well: a persistent 5xx on robots.txt can make Google pause crawling the host even though every page on it is fine. The rules and the traps are in how to check robots.txt.

4. Check the canonical

curl -s https://example.com/ | grep -i 'rel="canonical"'

If the canonical names a staging host, a development domain or any URL that is not the page you fetched, every page is asking to be folded into somewhere else. That is a canonical pointing to staging, and it produces exactly this symptom while the site keeps returning 200.

5. Check whether the site is hacked or blacklisted

A compromised site is often removed from results before its owner notices anything. Look at Search Console's Security Issues report first, then check the reputation lists from outside: the free Web Risk check reports whether Google's own malware and phishing lists currently flag your URL, and the free blacklist check covers the DNS blacklists that also affect your mail delivery. A flag on either means finding and removing the injected content comes before anything else, since a review will not clear while the site is still compromised.

6. Check for server errors

Pages that return 5xx or time out for the crawler get dropped, and a server that fails only under crawler load looks perfectly healthy in a browser. Check the Crawl Stats report in Search Console for a spike in failed requests, and confirm the status code your site returns from more than one network, since bot-protection rules that answer 403 to data-center ranges are invisible from a normal browser.

7. Check that the domain has not expired

The failure nobody expects, because it takes the site down completely and quietly: a domain that lapsed at renewal, or a DNS record that was removed with it. The free WHOIS lookup shows the registration and expiry dates along with the current name servers. An expiry date in the past, or name servers that are not yours, explains everything above it on this list.

The reason this list is long

Every cause here except the manual action produces a site that loads correctly, returns 200 and passes an ordinary uptime check, which is why the first signal is usually a traffic graph a week after the change shipped. Running these checks by hand catches it once. The free site crawl tool covers the on-page half of the list in a single pass across every page rather than the two or three you happened to test, and keeping the rest under continuous watch is what a website monitoring service is for.

Check it now

Run the free check against your own site - no account needed.

Crawl site

Monitor this permanently

Get alerted the moment it breaks: HostTracker checks from 300+ locations and notifies you by email, SMS, Slack, Telegram and more.

HostTracker features

More in this section: Fix it: guides for the errors people actually hit