Google is taking aggressive action on SERP data collection. Over the past year, that has changed how we collect data, and it will change how you should read your own rankings.
This post is based on Ray’s talk at Tech SEO Connect. The slides and the recording are linked at the end.
Why this is happening
Search results are one of the most valuable sources of behavioral data on the web. They show what people want and shape what they do next.
Two forces are now pulling on that data. AI assistants split one prompt into several searches and pull the results through proxies or SERP APIs. The person gets an answer and never visits Google. Google still serves the results, and nobody sees an ad.
An index is the prize for an AI lab. Building one takes years of work on crawling, rendering and ranking. Licensing one is possible, but the Bing APIs were retired on August 11, 2025, and the replacement keeps the raw results private. The SERP is the way in, because it sits on top of hundreds of billions of ranked, spam-filtered pages.
How it got here
The changes have come in steps:
| When | What changed |
|---|---|
| January 2025 | “SERPpocalypse.” Plain HTTP requests stopped returning results in the HTML. |
| September 2025 | The num=100 parameter was removed. Covering the top 100 now takes 10 requests. |
| December 2025 | Google took legal action against SerpApi’s scraping. |
| May 2026 | Drop-and-pops appeared in our data, as domains dropped out of results and came back. |
| July 2026 | The NetNut residential proxy platform was taken down, with Google among the partners involved. |
| August 2026 | “Goto links.” Result links began routing through google.com before reaching the site. |
| September 2026 | Aggressive blocking hit our collection. Google’s September spam update ran from September 24 to October 8. |
| October 2026 | Poisoned SERPs appeared: complete, normal-looking pages with the usual sites swapped out. Goto links returned, with waiting pages before some redirects. |
For the full story of these changes, see Google’s Summer of Love.
What you can do about it
- Check drops before you act. For a top 3 keyword, check the change in Search Console first. Some drops come from the data, not the search results.
- Ask your vendor what their pipeline does with bad samples. Ask how they capture samples and what happens to the anomalies. Vet the process as well as the output.
- Decide what part of the stack you own. If data is about you or affects you, it’s your data. Think about a warehouse strategy, and keep your own copy of the data that matters most.
- Track SERP features, not only position. A page can hold position 2 and still miss the AI Overview citation. The features around the results often matter as much as the rank.
- Read trends over weeks. A single day rarely tells you much. Look for patterns across a long run of samples.
- Triangulate. Compare your tracking with Search Console and other sources. Treat any single source as one input.
- Label outages and errors, and keep stakeholders updated. When collection has a bad day, mark it in your reports and tell the people who rely on the numbers. Trusting the data comes down to trusting the process.
How we respond
We now focus collection on the top of the SERP: the first 2 to 3 pages of results.
Every SERP we collect is compared with the last 30 days for the same keyword, domains and URLs. If a result falls outside that baseline, we fetch it again. If it still doesn’t match, we label it as spurious and store it with the label, so it stays out of your trends.
Every ranking gets a label:
| Label | What it means |
|---|---|
| Poisoned | The SERP falls outside the 30-day baseline even after a refetch. It is set aside. |
| Drop-and-pop | The keyword drops out and returns within 7 days. |
| Stable | The keyword ranks through the period. Short drops came back within 7 days. |
| Lost, unresolved | The keyword ends the period not ranking, for 8 days or more. |
These labels sit inside Observability, our product for navigating poisoned SERPs. It shows the spurious result rate, sampling, drop-and-pops, fetch error rates and labeled outages for every keyword you track. It’s data about your data, so you can see how far to trust each number. Read more about how we detect poisoned SERPs.
Impact at DemandSphere
In early October, 11% of the SERPs we collected each day came back spurious. As of November 2026, under 2% remained spurious after detection and retries. The 11% was our first measurement, taken while collection was under heavy blocking, so treat it as a snapshot of a changing situation. For the full story, see Spurious data in Google SERPs.
Accuracy is the wrong goal
As Mike King of iPullRank puts it, accuracy is the wrong goal in longitudinal monitoring. “99% accurate” tells you little without an answer to accurate against what. Google serves different pages to different clients, and one sample says little about the trend. The better question is whether your data pipeline captures samples reliably and labels what it can’t trust.
Where this goes next
Search results are getting more generative, interactive and multi-turn. More of the traffic will come from agents, which are harder to block and never see an ad. That leaves Google with an open question: what does its ad model look like when most SERP traffic is agentic?
Keep an eye on new AI models, especially the open ones. Our AI Frontier Model Tracker follows them.
And see how this fits your stack: Observability.
Let us know if you have questions or comments, and we’ll address them in future posts.