Pages Disappearing From Google's Index: Causes Ranked by Likelihood (2026)

When pages fall out of Google's index, the cause is almost always an instruction the site gave Google — usually by accident, usually in one line of code or one checkbox in a CMS. It is very rarely a penalty, and on a site with a few dozen or a few hundred pages, it is essentially never crawl budget. The fastest way to a fix is to stop theorising about what Google might be thinking and go read the report where Google states, per URL, exactly why the page isn't in the index.

It's worth separating two problems that feel identical from the outside. A ranking drop means the page is still indexed but sitting further down the results, which is a different diagnostic entirely. Disappearing from the index means the page isn't eligible to rank at all. The second is usually easier to solve, because Google will tell you the reason in plain language if you ask it the right way.

First, confirm the page is actually gone

Before diagnosing anything, verify the premise. Open Google Search Console, paste the URL into the inspection bar at the top, and read what comes back. Google's own URL Inspection tool documentation is explicit that the default result shows the most recently indexed version of a page, not the live version on the web — so a page you fixed yesterday can still show as broken until Google re-crawls. That's what the "Test live URL" button is for: it fetches the page as it stands right now and tells you whether the current version would be indexable.

Run both. The gap between them is the most useful thing you'll learn all afternoon. If the indexed version says "noindex" and the live test says the page is indexable, the fault is already fixed and you're waiting on a re-crawl. If both say the same thing, the fault is still live.

Cause 1: An accidental noindex tag

This is the most common cause by a wide margin, and it's the most common because it takes one action to create. A noindex directive is a single meta tag or HTTP header that tells Google not to include the page in search results, and Google honours it reliably and quickly. Search Console labels these pages "URL marked 'noindex'" in the Page Indexing report.

The ways it gets there are mundane. A WordPress install ships with a "discourage search engines" setting that stays ticked after launch. An SEO plugin applies a site-wide rule to a post type — tags, categories, a custom "services" type — and the pages built on that type quietly leave the index a fortnight later. A staging build gets promoted to production with its blocking headers intact, or a maintenance mode goes on for a redesign and never comes off. In every case the site works perfectly for humans, which is exactly why nobody notices for weeks.

There is a trap inside this one worth knowing about. Google's guidance on blocking indexing states that "for the noindex rule to be effective, the page or resource must not be blocked by a robots.txt file, and it has to be otherwise accessible to the crawler." Blocking a page in robots.txt to remove it from search does the opposite of what people expect — Google can no longer read the noindex tag, and the URL can still appear in results if other pages link to it. If you want a page out of the index, it needs to be crawlable and carry the noindex directive. Those two things sound contradictory and they are the reason a lot of DIY deindexing attempts fail.

Cause 2: A canonical tag pointing somewhere else

Second most likely, and the one that confuses people most, because the page isn't really missing. It's been folded into another page.

Google groups pages it considers duplicates and picks one to represent the cluster. Its canonicalisation documentation describes the explicit signals it weighs — redirects, rel="canonical" annotations, and sitemap inclusion — alongside implicit ones such as preferring the HTTPS version and the relationships declared in hreflang clusters. Google notes that these signals stack and become more effective when combined, which is the polite way of saying that when your signals conflict, Google resolves the conflict for you and you may not like the answer.

Three Search Console labels come out of this, and they mean different things. "Alternate page with proper canonical tag" means the page correctly points at another page and that other page is indexed — usually intentional, usually fine. "Duplicate without user-selected canonical" means Google found near-identical pages, you never nominated one, and Google picked. "Duplicate, Google chose different canonical than user" means you did nominate one and Google overruled you, which is a clear signal your pages are too similar to justify separate entries. Sorting out which of your pages genuinely deserve separate entries is its own piece of work, and we've written it up in what duplicate content actually does to Australian business sites.

Cause 3: A redesign that changed URLs

Third, and heavily concentrated in the weeks after a site launch. Old URLs stop resolving, new URLs exist at different addresses, and nothing connects the two. Search Console shows this as "Not found (404)", "Page with redirect", or "Soft 404" depending on how the new platform handles unknown paths. Some builders return a friendly "page not found" screen with a 200 OK status, which reads to Google as a real page containing nothing.

If pages vanished within a month or two of a rebuild, this is almost certainly your answer, and the recovery path is a redirect map rather than a content project. We've written the full recovery sequence for a redesign that cost you rankings, including the order to work in when hundreds of URLs moved at once.

Cause 4: Google crawled the page and decided against it

Fourth, and the uncomfortable one. "Crawled — currently not indexed" means, in Google's own words, that "the page was crawled by Google but not indexed. It may or may not be indexed in the future." No error, no directive, no fault. Google looked at the page and judged it not worth including.

This is not a bug to be fixed with a resubmission. Google's guide to how Search works states plainly that "Google doesn't guarantee that it will crawl, index, or serve your page, even if your page follows the Google Search Essentials." Indexing has always been discretionary, and the discretion has visibly tightened over the last few years.

The pages that usually attract this label are the ones generated in bulk: forty suburb pages that differ by a place name, service pages assembled from the same three paragraphs, tag archives, thin blog posts written to hit a keyword. On a small business site, a cluster of these disappearing at once is usually a content problem wearing a technical costume. The fix is fewer pages that each say something, not more pages that say the same thing — a point that comes up constantly in the broader question of why a site isn't showing up on Google in Australia.

Cause 5: Robots.txt, server errors, and bot protection

Fifth is the group of access faults. A robots.txt disallow rule that's too broad will stop crawling outright — and it's worth knowing that Google's robots.txt specification supports only four fields: user-agent, allow, disallow and sitemap. A noindex: line in robots.txt is not a supported rule and does nothing, despite still appearing in older advice.

Alongside that sit the server-side faults: 5xx errors when Googlebot happens to call, 401 and 403 responses from a password wall that reaches further than intended, and bot-protection rules that treat Googlebot as an attacker. Cheap shared hosting under load produces a version of this that's genuinely hard to see, because the site loads fine when you check it and times out when the crawler arrives at 3am.

Cause 6: Someone used the Removals tool

Rare, but it takes ten seconds to rule out and it explains an otherwise inexplicable disappearance. Search Console has a Removals tool that hides URLs from results, and Google's documentation notes that "a successful request lasts only about six months." One prefix request from a previous developer can take out a whole directory. Open the Removals report and check the list before you go any further.

The crawl budget myth

Crawl budget gets blamed for indexing problems on small sites constantly, and it almost never applies. Google's large site owner's guide to crawl budget opens by telling most readers not to bother: "if your site doesn't have a large number of pages that change rapidly, or if your pages seem to be crawled the same day that they are published, you don't need to read this guide."

The rough thresholds Google gives are large sites with over a million unique pages where content changes weekly, and medium sites with over 10,000 unique pages where content changes daily. An Australian services business with 60 pages and a blog is not in that conversation. The one legitimate exception is a site showing a large share of URLs as "Discovered — currently not indexed", which Google explicitly names as a crawl-scheduling signal — typically caused by slow server responses rather than by page count.

The Search Console walkthrough, in order

Open the Page Indexing report under the Indexing section. Ignore the graph and go straight to the "Why pages aren't indexed" table underneath it, which groups every excluded URL by reason. Sort by page count and start at the top, because one bad rule usually explains one large group. Be aware that a fair share of what sits in that table is Google confirming an instruction you meant to give — we've sorted the statuses that need action from the ones best left alone in Search Console errors in plain English, which is the better starting point if the report as a whole looks alarming rather than a specific page having vanished.

For each group, click through to the URL list and inspect two or three representative URLs individually. You're looking for a pattern in what they have in common: a post type, a directory, a date range, a template. Once you've found the shared cause, fix it at the source rather than page by page. Then use "Test live URL" on one of them to confirm the fix has actually landed, and only then hit "Request indexing" on a handful of the most important pages. Requesting indexing on two hundred URLs accelerates nothing — once the underlying rule has changed, Google re-crawls the rest on its own schedule, and the report moves over weeks rather than hours.

The thing worth resisting during that wait is the urge to change something else. If you fix the noindex tag on Monday and rewrite forty pages on Tuesday, you've lost the ability to tell which one worked.

If you'd like to rule out the platform side before you go digging, run the free audit — it checks crawl readiness including robots.txt and your sitemap, the certificate, HTML structure, page weight and Core Web Vitals in about a minute. It won't tell you whether a specific page is indexed, because only Search Console can answer that. It will tell you whether you're chasing one stray directive or a site that's making Google's job harder than it needs to be.