Observability is a word from IT Land. We don’t usually hear about it much in the world of SEO and AI search, although we should and I think we’re going to hear about it a lot more.

As we’ve been discussing recently, the landscape of search data is changing in many ways, but one key change is that it is becoming an increasingly volatile data set to work with.

This volatility is, in our view, directly correlated with the value it provides AI and data teams. It’s a scarce resource, it’s hard to get at scale, and it provides a look at the forces that reflect, influence, and shape consumer behavior on a scale that cannot be found elsewhere.

It is also our data, like yours and mine. This goes back to our belief that if there is data about you or something that affects you, then it is your data. Same thing goes for your websites.

The key players who claim to “own” this data thrive on information asymmetry and, as a result, aren’t particularly inclined to make it easy to get.

Hence the volatility.

As this volatility has increased, especially over the last year, it has become increasingly important to build data about this data to answer questions like: what is the error rate, sampling rate, number of pages being fetched, and so forth, on a daily basis.

And the shape of these questions and answers differs depending on the engine in question. If we’re talking about Google SERPs, then it has one set of characteristics, and data from ChatGPT has another. And so on.

To date, as far as we are aware, there isn’t much out there from vendors such as DemandSphere (or anyone in our market) providing answers to these questions as a part of their offering.

We feel that it is time to change that and, as a result, we are announcing a new feature within our platform called Observability. We’ve been working on this for a few months. It’s not what we would consider “done” as it will be evolving every time there is a new big change and there is still a lot more that we want to do with it.

First, let’s establish a formal definition of what Observability is in the IT world.

What is Observability in IT and data?

The goal of Observability is to understand a system’s internal state and the key characteristics that shape how data within the system manifests itself. All data has a shape. Observability describes that shape and the forces that act upon it. Traditionally, there are a number of key aspects to Observability, including:

  • Logs: the raw data, typically row-level records of individual events
  • Metrics: aggregated measurements taken over time, such as counts, rates, and distributions
  • Traces: the end-to-end record of a single request as it moves through the system, assembled from timed spans, so you can follow one operation across every stage it passed through

Traces are the one that needs translating for our world. In a web application, a trace follows a single request across services and shows you where the time went or where it broke. In search data collection, the equivalent is following a single fetch: which region and route it was requested from, when it ran, what came back, how it was parsed, which imputation steps touched it, and how all of that became the position number sitting on a chart.

That is the part that matters when a number looks wrong. Without it you are staring at an aggregate and guessing. With it you can take one suspicious data point, walk it back to the collection event that produced it, and find out whether the SERP moved or the pipeline did.

Observability vs. Monitoring

They may sound similar, but Observability and Monitoring are two different things. Traditionally Monitoring focuses on predefined alerts and looking for certain patterns. Observability is designed to provide more ad hoc levels of flexibility to analyze the data.

In the context of SEO and AI search and, specifically for our feature, we are adding an additional facet to this definition: first-order vs. second-order.

First-order is the data that everybody pays for: SERP data, LLM data, etc.

Second-order is the data about this data. Again, this will be data points like error rate, sampling rate, etc.

In a perfect world, there wouldn’t be a need for much second-order data about the first-order data but, as of October 2026, we’re in a new world.

We can confidently predict that we are entering a new era of volatility and that it will be key for customers to understand what is happening within their data systems on a regular basis.

This does add a large new burden on platform vendors that did not exist before and it is something that will take time to build up. But we’ve decided to do it and we’re announcing this new feature set today.

Observability in DemandSphere

There have been a number of volatile changes with data, particularly from Google, over the last year. We log the confirmed ones on the Google Algorithm and AI Search Update Tracker.

The newest view in Observability covers the problem we wrote about in how we detect poisoned SERPs: result pages that come back looking complete but with the usual results swapped out. This panel shows how many delivered ranks were read from a SERP the fetcher flagged as poisoned, after re-fetching.

Poisoned SERPs in delivered ranks
For each delivery from 2026-05-08 to 2026-11-03, Google desktop and mobile: how many delivered ranks came from a SERP the fetcher flagged as poisoned, after re-fetching. A rank is judged by the SERP it was read from, including ranks carried over from an earlier fetch.
Clean from a SERP that was checked and not poisoned. Poisoned from a SERP flagged as poisoned. Older SERP read from a SERP fetched before the poisoned check was added on 15 October, such as a rank carried over from an earlier fetch. That share falls to zero as those SERPs age out.
Latest delivery · % poisoned
0.46%
2026-11-03 · 8 of 1,722 ranks
Latest delivery · % clean
99.30%
0.23% from older SERPs
Worst delivery · % poisoned
2.67%
2026-10-26 · 46 of 1,722 ranks
All deliveries · % poisoned
0.07%
225 ranks over 180 deliveries
Delivered Google ranks per delivery · line: % poisoned, 0 to 100 scale
0800160024000%25%50%75%100%2026-05-082026-05-232026-06-072026-06-222026-07-072026-07-222026-08-062026-08-212026-09-052026-09-202026-10-052026-10-20
Older SERP Clean Poisoned % poisoned
DeliveryDeliveredCleanPoisonedPoisoned (ranking)Older SERP% poisoned% of checked
2026-11-031,7221,7108740.46%0.47%
2026-11-021,7221,7108840.46%0.47%
2026-11-011,7221,70215950.87%0.87%
2026-10-311,7221,687302751.74%1.75%
Reading this. The right axis runs 0 to 100, not autoscaled, so the poisoned share is shown against the whole of what could have gone wrong rather than stretched to fill the panel. The worst delivery in the period reached 2.67%. Bars are grey until 15 October because the check did not exist yet and every rank was read from an unchecked SERP.

The scale is deliberate. Plotted against the full 0 to 100 range, the poisoned share is small, and it should be. The point of measuring it is to know that, and to know the day it stops being true.

One that we haven’t had a chance to talk about too much publicly yet, and it hasn’t affected everybody, is what we call “drop-and-pops” (“DnPs” for short).

A drop-and-pop occurs when Google temporarily tanks a given domain’s rankings for a few days and then brings them back to where they were. This is something that has occurred, to varying degrees, for years. But clients started to notice an increase in these in May. We needed a way to show the extent, impact, and volatility levels of these events. We also needed to be able to show when some percentage of results happened to recover.

We set a 7-day window for this, on a rolling basis, and have developed our first version of Observability around this phenomenon.

Here are some views from REI (not yet a client but we’d love to talk to you), a domain we use a lot for demos.

End-state by tier, with the drop-and-pop episode counts alongside:

Tier × end-state (volume-weighted)
Where each tier ended the period. These six states are mutually exclusive and sum to 100% of tier volume. Each cell shows % of tier volume, with keyword count.
TierStableTentativeErodedLost (rec.)Eroded→LostLost (unres.)
Top 1–3
1059 kws · 9,592,500 mo. searches
99.8%
1044 kws
0.0%
1 kws
0.1%
4 kws
0.1%
7 kws
0.0%
0 kws
0.0%
3 kws
Top 4–10
469 kws · 1,842,280 mo. searches
99.3%
450 kws
0.0%
2 kws
0.7%
7 kws
0.0%
5 kws
0.0%
0 kws
0.0%
5 kws
Top 11–20
75 kws · 327,350 mo. searches
99.9%
65 kws
0.0%
0 kws
0.0%
0 kws
0.1%
7 kws
0.0%
0 kws
0.0%
3 kws
Drop & Pop episodes
A separate measurement from the table above, and not part of that 100%. A keyword can end the period stable and still have dropped and popped repeatedly along the way.
Top 1–3
64.8%
of keywords had at least one episode, 1.6 ep/kw
1.3%
inside an episode on an average day
Top 4–10
80.6%
of keywords had at least one episode, 3.6 ep/kw
3.4%
inside an episode on an average day
Top 11–20
98.7%
of keywords had at least one episode, 7.4 ep/kw
7.2%
inside an episode on an average day
Reading this. Stable = operationally fine after imputation. Tentative = ends the period NR but for <8 days. Eroded = still ranking, median position dropped ≥10. Lost (recovered) = a real 8+ day NR episode but ends ranking. Eroded→Lost = drifted ≥10 down before going NR. Lost (unresolved) = ends NR for 8+ sustained days. Drop & Pop = ≥1 resolved ≤7-day NR episode, pre-drop position ≤20. The two Drop & Pop figures differ because one counts keywords ever affected across the whole period and the other counts keywords affected on a given day.

Two numbers above look like they disagree and do not. 64.8% of Top 1-3 keywords had at least one drop-and-pop across the period; on any given day only about 1.3% are sitting inside one. Episodes are short and spread out, so a tier can be almost entirely stable at any moment and still have most of its keywords affected at some point. The chart below is the daily view.

Trended view:

Daily drop-and-pop · count-weightedpre-drop ≤ 20
Prevalence is the share of keywords currently inside a DnP episode. The amber dashed line is a Theil-Sen trend fit on the pre-boundary days. The right-edge boundary zone, the last 7 days, is shaded because imputation loses accuracy there.
0%25%50%75%100%MayJunJulAugSepOct2026-04-092026-04-272026-05-152026-06-022026-06-202026-07-082026-07-262026-08-132026-08-312026-09-18
DnP prevalenceDiagnostic trend
Prevalence by tier · monthly average
boundary zone excluded · partial months annotated
TierAPR 2026MAY 2026JUN 2026JUL 2026AUG 2026SEP 2026OCT 2026
partial · through 2026-10-05
Period avg
Top 1–30.4%2.3%2.1%1.3%0.4%1.1%–1.3%
Top 4–103.8%5.0%3.6%4.2%1.5%2.3%–3.4%
Top 11–208.9%9.6%6.7%7.7%4.4%6.0%–7.2%
Avg of tiers4.4%5.6%4.1%4.4%2.1%3.1%–3.9%
All primary tiers (weighted)1.8%3.4%2.7%2.4%0.9%1.7%–2.2%

Volume-weighted trends:

Daily drop-and-pop · volume-weightedpre-drop ≤ 20
Same episode qualification, weighted by monthly search volume. Higher-volume keywords contribute proportionally more, which reveals head-term concentration.
0%25%50%75%100%MayJunJulAugSepOct2026-04-092026-04-272026-05-152026-06-022026-06-202026-07-082026-07-262026-08-132026-08-312026-09-18
DnP prevalenceDiagnostic trend
Prevalence by tier · monthly average
boundary zone excluded · partial months annotated
TierAPR 2026MAY 2026JUN 2026JUL 2026AUG 2026SEP 2026OCT 2026
partial · through 2026-10-05
Period avg
Top 1–30.1%0.5%1.4%2.6%1.3%0.3%–1.1%
Top 4–102.6%3.0%2.1%1.4%0.7%3.0%–2.1%
Top 11–209.4%11.1%6.4%6.0%2.1%4.6%–6.5%
Avg of tiers4.0%4.8%3.3%3.4%1.4%2.7%–3.2%
All primary tiers (weighted)0.7%1.2%1.7%2.5%1.2%0.9%–1.4%

Position trends by tier, which show whether damage is concentrated or spread evenly:

Top 1–31059 kws
trend +0.04 pos/wk
01020MayAprJunAugOct
Top 4–10469 kws
trend -0.04 pos/wk
01020MayAprJunAugOct
Top 11–2075 kws
trend -0.16 pos/wk
02550MayAprJunAugOct
Top 21–5026 kws
trend -0.10 pos/wk
02550MayAprJunAugOct
Average positionMedian position
Read the tier panels left to right. Damage concentrated in the premium tier, with the Top 1–3 trend pointing up, is genuine trouble; similar trends across every tier is tracker noise. Slope values are positions per week for average position, or percentage points per week for DnP.

An additional view on confirmed lost keywords (DnP did not “pop” again within 7 days):

Confirmed lost keywords · top by volume
Keywords classified lost_unresolved (sudden) or eroded_into_loss (gradual) in primary tiers, sorted by search volume.
Customer impact
QueryMo. searchesTierPatternBaseline med posDrop dateDnP history
day packing2,900Top 1–3lost_unresolved1.02026-08-219 ≤20 · 9 any
what age is a 24-inch bike for210Top 4–10lost_unresolved4.02026-08-2215 ≤20 · 15 any
how tall should you be for a 16-inch bike30Top 4–10lost_unresolved5.02026-08-1210 ≤20 · 10 any
what age can ride a 26 inch bike30Top 4–10lost_unresolved7.52026-07-195 ≤20 · 5 any
do running shoes make a difference in running0Top 11–20lost_unresolved19.02026-08-1010 ≤20 · 10 any
plug types in africa0Top 4–10lost_unresolved5.02026-09-2712 ≤20 · 12 any
should you buy walking shoes one size bigger0Top 1–3lost_unresolved3.02026-06-193 ≤20 · 3 any
what age is good for 24-inch bike0Top 11–20lost_unresolved10.52026-07-103 ≤20 · 3 any
what happens when you take a geocache0Top 11–20lost_unresolved11.02026-08-127 ≤20 · 7 any
what outfits to wear in winter0Top 4–10lost_unresolved10.02026-08-047 ≤20 · 8 any

We have a similar one for recovered keywords.

We are, this week, adding some additional metrics, one of which we will discuss in our next blog post about Google serving spurious results.

These metrics will cover both the spurious results rate and also the overall sampling percentage, along with details around which keywords have sampled data on a day-to-day basis.

Conclusion

Given the world we’re heading into, we believe strongly that transparency and Observability are critical aspects of any data strategy and that is doubly true when it comes to the world of search. Let me know any questions or feedback!