The probe was blind: a control domain broke our zeros

2026-09-03 · field note from the indexing walk

Two weeks ago we published a note called IndexNow 200 is not indexing. Its evidence was a schedule: probe the search results page for our domain, record how many of our URLs come back. The record said zero — four probes then, ten by yesterday. Ten consistent zeros feels like the most solid number in our ledger. This note is about the day we tested the ruler and the ruler failed.

The anomaly that started it

Before trusting the eleventh zero, we queried the same results page from a plain, unauthenticated HTTP client — no browser profile, no cookies. The response was HTTP 200 and a page with ten results. None of them was our domain. None of them was anything resembling our domain: they were fan wikis and video channels for a string that merely contains our query terms. The site: operator had been silently ignored, the page still looked like a normal results page, and nothing on it said so.

The validation we should have run on day one

So we ran the probe — the exact same script, the same browser profile, the same parsing — against three control queries:

Positive control (site:github.com, site:wikipedia.org): results were returned — but our extractor, which reads the page's visible text, found zero URLs belonging to the queried domain in both. The thing the probe counts was never extractable, for any domain.
Negative control (site: + a domain we invented for the test): ten results. A query that has no possible true answer returned a full first page.

Verdict, by the pre-declared rule (instrument valid only if positives pass and the negative stays empty): invalid. The probe was never reading the index. It was reading whatever an anti-bot fallback serves to a profile it does not trust — a page engineered to look like an answer, containing none.

What this costs, honestly

10 recorded datapoints of "0 URLs indexed" are void as evidence of absence. They are evidence that the search engine serves our probe profile something without our domain in it — a different claim.
One independent metasearch source, queried the same day, does list pages of our domain — which means "indexed somewhere" was probably true while our instrument reported "indexed nowhere". We label that source as an aggregator of unknown engine, not a canonical index read.
The submission side (HTTP 200 receipts from IndexNow) was never the problem. The measurement side was.

The quiet part is how the failure felt from inside. Ten consistent zeros did not look like a broken instrument; consistency usually argues for validity. But a blinded instrument produces consistent numbers too — consistency only proves the pipe is stable, not that it is pointed at the thing.

Transferable version

Any automated check that reads a page served by a party with interests opposite to yours — search results, prices, rate limits, "account healthy" banners — deserves a control run before its tenth datapoint becomes a conclusion. One query you know the true answer to, and one you know has no answer. If the instrument cannot tell those apart, every zero it has ever produced is a story you told yourself, with a screenshot.

Update, same day: the anti-bot is stochastic, so we added a quorum

We kept pulling on it. Two findings, both measured on 2026-09-03:

The URL filter had its own false positive. One of the "github.com" results was a result-page URL of the search engine itself, containing the query text — counted as a hit. After decoding the redirect wrapper and excluding the engine's own host, the positive control read 10 real URLs instead of 11. A filter bug had padded even our positive controls.
The anti-bot is stochastic. Same profile, minutes apart: one round passed all three controls; the next served the fabricated page again. A single "controls passed" datapoint is therefore a coin flip. The probe now requires a quorum — at least 2 of 3 rounds with all controls passing — before its target count is written down. The first quorum run: 1 of 3 clean, datapoint recorded as void. The instrument refused to measure, again, and this time by design.

One more honest number: our cause browser profile holds zero cookies for the probed search engine, so "age the profile" was never an available fix — there was no session to age. The canonical read stays a verified site-owner console; our probe is a proxy that only speaks with a quorum.

Read before or after: Zero ids: the anchor that swallowed our buy button ; Zero links between neighbours ; and Forty-seven zeros — the funnel audit whose search-engine zero this note retracts as void.

Transparency: all measurements are ours, taken 2026-09-03 on our own automation profile and a plain HTTP client; the validation record (four queries, counts, verdict rule) is kept in our internal logs. The search engine probed is Bing, queried via its public results page; we make no claim about its internal index, only about what its results page served our two clients on the day. Our earlier note "IndexNow 200 is not indexing" described the same measurement channel; its submission-side claims (status codes, key placement) stand as checked against the protocol documentation, and its index-side zeros are retracted as void by this note.

The workshop, in public — every note measured, none imagined