After 190 days since last crawl, Google "forgets" a Not Indexed page even exists.
At Indexing Insight, a study has uncovered the 190-Day Indexing rule.
This rule is based on a study of 1.4 million pages across 18 different websites (see methodology for more details).
Our study focused on combing the Days Since Last Crawl (based on Last Crawl Time) and the index coverage states from the URL Inspection API.
The indexing data pulled in this study is from Indexing Insight. Here are a few more things to keep in mind when looking at the results:
Google's crawling and indexing system forgets or has forgotten pages that have not been crawled in 190 days.
Our data from 1.4 million pages across multiple websites shows that if a page has not been crawled in 190 days then there is a 90% chance the page will be either start to be forgetten or forgotten by Google Search.
A page is forgotten when its indexing state changes to 'URL is unknown to Google' and has not been crawled in 190 days.
Any historically crawled and indexed page can be forgotten over time. If it is not crawled in 190 days then eventually the page's indexing state changes to 'URL is unknown to Google'.
In Google Search Console, the indexing state indicates crawl priority.
At Indexing Insight, our research has identified that the index coverage states of pages can be mapped to the crawl priority in Googlebot.
Gary Illyes confirmed that page URLs can move between indexing states and that Google's systems can forget historically crawled and indexed pages.
“Those have no priority (URL is known to Google); they are not known to Google (Search) so inherently they have no priority whatsoever. URLs move between states as we collect signals for them, and in this particular case the signals told a story that made our systems "forget" that URL exists. I guess you could say it actually fell out the barrel altogether.” - LinkedIn
Gary mentions that page URLs can move between “states” as Google’s system picked up signals over time. And that historically crawled and indexed pages can eventually move to ‘URL is unknown to Google’.
To quote Gary, Google’s systems will eventually “forget” a page exists.
We've incorporated the 190-day rule into Indexing Insight.
Based on this research, the Indexing Insight team has created reports that help customers quickly identify pages not crawled in 190+ days.
To view pages not crawled in 190+ days go to the Crawl Coverage > 130+ Days Report. It's in this report you'll find all the forgotten pages.
To view the pages not crawled in 190 days, you need to sort the table by Days Since Last Crawl. This will show you pages with the longest crawl time. Then apply any filters to segment the data.
Now you can see which pages haven't been crawled in 190 days and have been actively 'forgotten' by Google's Search index.
Check our pricing page to see which plans include Crawl Coverage reports.
Google can forget crawled and indexed pages exist after 190 days.
SEO teams can use Google Search Console data to identify which pages are being actively forgotten. By combining the last crawl time with index coverage state you can quickly identify forgotten pages.
At Indexing Insight, we've incorporated the 190-day rule into our product so customers get actionable data from day one.
Adam Gent
SEO Product Manager and Technical SEO. I’m currently an independent consultant who works with organisations to plan, scope and execute SEO projects that drive results.
