The evidence
A question a buyer would ask is put to Claude, ChatGPT, Gemini and Perplexity, cold: a fresh conversation, live search on, nothing supplied about the company. Each answer is kept word for word. Every page an agent opened or cited to write it is fetched at that moment and kept, so what the agent read is on record beside what it said. Every sentence of every answer is split out and tagged: for, against or mixed on the company, a fact or an opinion, which competitor it names, what it hedges on.
The company’s own site is read the way an agent reads it: a plain fetch, no JavaScript, and about forty checks straight off the domain, from robots.txt and the sitemap to what a page hands back as text and whether it carries the markup that says what it is.
The test
A test is one sentence an agent said, one change to one page, and a prediction. The change is the smallest thing the company controls that would change what the agent read: a fact stated on the page the agents already open, a hedge replaced by the control behind it, a page that hands back nothing revived, a claim retracted. The prediction says which way the answers should move and why: if the page states X, the agent should be less likely to say Y, because today it reads only Z.
- The change is named, with the page it lands on and the sentence the page should carry.
- The company makes it. A prompt carries the question, the sentences, the pages the agents opened and the whole answers, so an assistant editing the site fixes the sentence and not the theme.
- The page is read again, the way an agent would find it: from the homepage and the sitemap. What is new on it is judged against the sentence under test: what changed, whether it speaks to what the agent said, and whether it is likely, partly likely or unlikely to move the answer. A change that speaks to none of it is not run.
- The same question goes to the same four agents, cold, again.
The before and after
Each agent’s new answer is set against its answer before: where it stands (for, hedging, against), whether the watched sentence was said again, and whether the agent opened the page that changed. A test ends supported when more answers stand better than worse, rejected when the reverse, and inconclusive when they do not move on balance. AI answers vary from one run to the next, so a prediction says fewer agents will say something, never that one will, and a result is confirmed only when a later round says the same.
What it is not
It is not a split test: the web cannot be halved. It is not proof of cause from one draw: four agents answering once is a thin basis, and the test says so until a second round agrees. And an answer opened from memory, with no page fetched, is a different problem with a different fix.