The index coverage states for your pages can reverse as Google picks up more signals over time.
In this article, we'll explain why your pages index coverage states can reverse over time as Google picks up more signals. And show you examples of indexed pages that have had there index coverage states reverse.
Let's dive in!
Every page URL has a coverage state in the Page Indexing report.
A page's coverage state indicates the specific reason it's Not Indexed. This information is available through the URL Inspection API, which provides additional details.
These index coverage states can be grouped into 3 categories:
Pages regularly shift between these three categories. Contrary to popular belief among SEO teams, technical and duplication issues aren't the primary drivers of these changes.
Based on our data, page quality issues are the primary cause of index coverage changes.
A page's index coverage state can change over time.
When pages are actively removed from Google’s index, the index coverage state changes based on the URL's priority in the Googlebot web crawling process.
Tracking changes over millions of URLs for our alpha testers led us to a surprising conclusion.
A page’s indexing coverage state can go backwards from being submitted and indexed to Google, telling you that the URL is “unknown”.
For example, a typical page usually goes through the following steps:
But over time, the same page coverage state can go backwards from:
It can take between 130 and 190-days for an Indexed page to reverse.
Our 190-Day Indexing Rule research found that Google actively forgets pages that haven't been recrawled in 190 days. An Indexed page that hasn't been crawled in 190-days can have its coverage state changed to 'URL is Unknown to Google'.
If we combine this research with the 130-Day Indexing Rule research, we can see that over time index coverage states it takes 130 days for an Indexed page to be deindexed.
Let’s look at some examples to understand page coverage states going backwards in the wild.
Below are 3 examples of a page’s coverage state reversing:
The first example is from The SEO Sprint.
The current coverage state in the URL Inspection tool is ‘URL is unknown to Google’.
However, using the URL report in Indexing Insight, we can see that this particular URL was indexed and had its Google-selected canonical URL.
The Google-selected canonical change happened on the 15th of August, 2024. We even provide a link to a historic URL Inspection tool in Google Search Console.
The change in the canonical links indicates that Google dropped the data from its index.
When you open the link, you can see that historically, this page had a ‘crawled - currently not indexed’ coverage state.
If we look at the page's search performance, we can see that it had 18 impressions in March 2024. This indicates that this page was indexed and served in search results.
This page URL reverse index state went from:
This page URL indexing state reversed, and the URL coverage state “went backwards”.
The second example is a programmatic SEO website.
In this example you can see the /cities/banbury/ URL is shown as URL is not on Google.
However, when checking the page's history in the URL report, we can see historic changes. The page was indexed (13th June 2024), and there were changes to the canonical URL (14th October 2024).
If we check the link to Search Console, we can see that the page has the coverage state of ‘crawled - currently not indexed’.
If we check the Search Performance of the page URL over the last 16 months we can see that this page was indexed. And it was served to users (which is why it has impressions).
Again, this page URL reverse index state went from:
Like the previous example, this page's URL indexing state reversed, and the URL coverage state “went backwards”.
The final example is a niche cricket website.
If we input the page /players/niaz-khan/ into the URL Inspection tool, the current coverage state is ‘Discovered - currently not indexed’.
However, using Indexing Insights, we track any changes in a page's index state. As you can see, this particular URL was indexed on May 31st, 2024, and the canonical URL changed on October 9th, 2024.
If we open the historic URL Inspection report for when the canonical URL changed (9th October 2024), this page will have a ‘crawled - currently not indexed’ coverage state.
Finally, if we look at the performance over the last 16 months, we can see that the page has had impressions and clicks.
This indicates that Google indexed this page and showed it in its search results.
To recap, this cricket niche website page URL index state reversed and went from:
Like the other examples this page URL indexing state reversed, and the URL coverage state “went backwards”.
A page’s index coverage state can change because of page quality.
Gary Illyes confirmed on LinkedIn, Google collects signals over time that cause a page's index coverage state to change. And the signals surrounding a page can tell Google that can cause it to "forget" a page exists.
Google recently published a video Help! Google Search isn’t indexing my pages on 20 Aug 2024. The video focuses on the ‘Discovered - currently not indexed’ state in Search Console.
In the video, Martin explains that one of the most common reasons pages are in the ‘Discovered - currently not indexed’ category is that they have been actively removed from its index.
To quote the video:
“The other far more common reason for pages staying in "Discovered-- currently not indexed" is quality, though. When Google Search notices a pattern of low-quality or thin content on pages, they might be removed from the index and might stay in Discovered.”
- Martin Splitt, Help! Google Search isn’t indexing my pages, 20 Aug 2024
It confirms Google actively removes indexed content from its index AND when it is removed the coverage state changes.
It happens more often than you think.
For one alpha tester, 32% of their 1 million submitted URLs have experienced reverse indexing over the last 90 days.
They can see that historical URLs are slowly being removed from Google’s serving index. And as more time passes the state is changing to be moved to other indexing states…
…which reflects the crawl priority in Google’s crawling system.
They can see the slow increase of coverage states ‘discovered - currently not indexed’ as the number of ‘crawled - previously indexed’ declines.
They are also seeing an increase in ‘URL is unknown to Google’ as important URLs become less of a crawl priority to Googlebot.
They are watching Googlebots crawl system deprioritising URLs to be crawled.
There are two ways to detect if a page's index coverage state is reversed:
At Indexing Insight, we have reports that help your team automatically track changes in your index coverage state.
The 'crawled - previously indexed' report helps you track pages that were previously indexed but have since been removed from Google's index.
The 'URL is Known to Google' report helps to identify pages which have moved from 'discovered - currently not indexed' to 'URL is known to Google'.
The Crawl Coverage report helps you track how frequently Googlebot crawls your pages and identify those that may be getting deindexed or deprioritised based on crawl frequency.
The coverage states in the page indexing report indicate levels of crawl prioritisation.
SEO teams can use the Search Console index coverage report to identify which important pages they want to rank are being actively removed by Google.
How can you find out?
Filter on the important pages you want to rank in Google Search.
In Google Search Console, go to Page Indexing > All Submitted Pages. This will provide you with a clearer picture of the pages that you want to rank in Google Search.
Do you see ‘crawled - currently not indexed’ or ‘discovered - currently not indexed’?
Then, it’s highly likely your pages are experiencing reverse index coverage states.
If you find ‘crawled - currently not indexed’ or ‘discovered - currently not indexed’ with this filter, it means:
If nearly all your pages submitted are in these two categories, then it's a strong indication that your website is less of a priority to crawl, index, and rank in Google.
The coverage states of your pages in the Page Indexing report are not static.
In this article, we’ve provided evidence that Google will actively remove pages from its search results. AND that the coverage states of your pages can go backwards.
This reverse in index coverage state indicates low crawl priority in Googlebot’s system.
If we map the index coverage states to Googlebot’s crawl, render and index process we can see that pages in the ‘URL is Unknown to Google’ and ‘discovered - currently not indexed’ are less of a priority to crawl.
This is a problem because crawl priority indicates a page's ability to rank in Google.
Hopefully, this article has inspired you to look at your page indexing report with a new understanding. And identify which pages are less of a priority for Google to crawl and index.
Adam Gent
SEO Product Manager and Technical SEO. I’m currently an independent consultant who works with organisations to plan, scope and execute SEO projects that drive results.
