62% of AI Brand Suggestions Collapse with Just One Buyer Question—What’s Really Going Wrong?
Ever stumbled upon a report so detailed you swear it could practically whisper the secrets of AI… only to find out a rogue zero was playing hide-and-seek in the data? That’s exactly what happened with Clovion AI’s latest study, “Surviving the AI Funnel.” When Zahir Hasan, the COO, casually dropped that their headline figure of 33 contradictions was actually 330 due to a layout slip-up, it wasn’t just a typo—it was a wake-up call. You see, in the wild world of AI brand recommendations, missing a decimal point can flip your entire strategy on its head. Are these AI assistants really disagreeing 15% of the time—as first reported—or is the story way more nuanced? As someone who’s danced through decades of SEO shifts and algorithm puzzles, I can tell you this: trusting AI outputs without questioning the numbers is like building a castle on sand. So, what does this colossal correction mean for marketers and brand strategists trying to decode AI’s brand bias? And more intriguingly, how do we keep our finger on the pulse when the numbers themselves might fool us? Let’s unravel this AI funnel maze and see why vigilance beats blind faith every single time. LEARN MORE
Zahir Hasan didn’t have to tell me his company’s numbers were wrong.
I’d sent Hasan, COO of the Oslo-based research firm Clovion AI, a list of methodology questions about “Surviving the AI Funnel,” Clovion’s new study of how Claude, ChatGPT, and Gemini recommend brands across a conversation. Question ten was routine, the kind of thing you ask every research team. The report says the three AI assistants flatly contradict each other on brand facts 15% of the time, based on 33 verified contradictions. Was 33 really enough to support a claim about which model tends to undersell a brand’s features and which tends to oversell them?
Hasan’s answer wasn’t a defense of the number. It was a correction. “The real number is 330,” he wrote back. “A designer dropped a zero in layout.” The same slipped decimal, he said, had also turned 2,040 brands into “204” on page seven of the PDF that I’d been sent in advance of its publication. A revised version is coming out this week. So, I got the corrected figures first.
That’s a strange way to start a column about an AI research report, admitting before anything else that the draft report had an error in it. But it’s the most honest way in, because the correction says something the study’s headline stats never could. Reading AI answers correctly, whether you’re a marketer trying to figure out if ChatGPT is recommending your product or a researcher building a study about it, comes down to catching the decimal point before you build a strategy on it.
The Funnel, Recapped
Set the typo aside for a moment and the underlying research holds up. Clovion ran 69,120 multi-turn conversations across the three assistants in 36 B2B software and fintech categories, asking an opening question like “best CRM tools?” and then a single realistic follow-up. Re-asking the same question kept 90% of the recommended list intact. Adding one ordinary buyer detail, something as plain as “for a small team,” kept only 28%. Sixty-two percent of the brands that made the first answer were gone by the second one.
I asked Hasan whether “small team” was cherry-picked to produce that drop. It wasn’t. His team also tested “for a large enterprise” and got almost identical churn, around 72% either way, against roughly 10% when the question was simply repeated. The list isn’t unstable. It’s responsive, and mostly to whether the model has decided who a brand is actually for.
That’s the part worth sitting with if you do SEO or brand strategy for a living. Being named in an AI answer is not the same thing as being trusted by it. A model that puts you in its first CRM list can still cut you the moment a buyer gets specific, and Clovion’s data says that happens most of the time, not some of the time.
The Correction Changes the Shape of the Smallest, Most-Cited Number
Here’s where the fixed decimal actually matters for how you should read this study. The old figure, 33 verified contradictions, was small enough that any per-model claim built on it was standing on thin ice. Corrected, it’s 330, and the per-model breakdown Hasan shared is far more telling than the aggregate 15% figure the draft report leads with: Claude underclaims a brand’s own features 160 times against 10 overclaims. ChatGPT underclaims 70 times and never overclaims. Gemini runs the other way, overclaiming 80 times against 30 underclaims.
Hasan’s working theory, drawn from a separate, not-yet-published Clovion study on where each model sources its answers, is that Gemini leans more heavily on marketing material and video, so it tends to credit a brand with whatever it’s hyping. Claude and ChatGPT lean more on documentation and product pages, describe the core product accurately, and hedge toward “doesn’t have it” when a newer feature isn’t well documented. If that holds up under the study Clovion hasn’t released yet, it means the direction of an AI assistant’s error about your product is a function of what kind of content you’ve put in front of it, and where that content lives.
I’ve spent more than 20 years telling clients that ranking well and being described accurately are two different problems. This is the clearest evidence I’ve seen that they’re now the same problem, playing out inside a single conversation, and that the fix depends on which assistant is doing the misdescribing.
Why Nobody Catches the Missing Zero
Frederick Vallaeys has a story in his book “The AI-Amplified Marketer” that explains exactly why a dropped decimal survives all the way to publication. An automated report once flagged “great performance” on a keyword because its cost per acquisition was running much higher than the target. Somewhere in the system, high had gotten swapped for good, when a high CPA is bad news, not good news. Anyone skimming the summary would have nodded along, because the sentence read smoothly even though its meaning had flipped.
Vallaeys ties this to research on predictive processing, the idea that fluent readers aren’t decoding every word, they’re predicting what comes next based on context and moving on. That’s how “teh” reads as “the” and a missing “not” slides right past you. As Vallaeys puts it, our mental model of the sentence overrules the text in front of us. A confident, well-formatted PDF is the easiest place in the world for that to happen, and a dropped zero in a layout file is a much smaller, much more forgivable version of the same failure.
It’s also why the fix isn’t “trust the report less.” It’s “keep a human pilot in the loop who checks the number instead of the vibe of the paragraph around it.” Thirty-three contradictions and 330 contradictions don’t just differ by a factor of ten. They support entirely different confidence levels about whether a per-model pattern is real. Two hundred four brands and 2,040 brands aren’t the same study. If Clovion hadn’t caught it, and if I hadn’t asked, the smaller, shakier numbers would have kept circulating as fact, cited by exactly the kind of trade press that’s supposed to catch this.
What Clovion Isn’t Claiming, and Why That’s the Honest Part
The report is careful to say the link between how a model perceives your fit and whether it recommends you is “a strong, consistent coupling, not a proven causal law.” I pushed Hasan on what a real causal test would look like. His answer: change one thing, a brand’s public positioning content, leave everything else alone, and see whether the models’ behavior moves relative to brands nobody touched. Clovion hasn’t run that test yet. He also conceded the more uncomfortable possibility directly, that a brand’s actual real-world positioning is probably driving both how the model describes it and whether it gets recommended, which would make positioning the real lever and the model’s “perception” just a symptom, not a cause.
That’s an unusually candid answer from a company selling AI visibility monitoring, and it’s exactly why I trust the rest of what Hasan told me. He also had no data on how fast an AI’s perception of a brand shifts after that brand changes its own content. “We didn’t do a before-and-after test,” he said. “Treat it as worth testing, not guaranteed in X weeks.” Anyone telling you they can promise a specific timeline for moving Claude’s or Gemini’s opinion of your brand is guessing, by Clovion’s own admission.
What To Actually Do About It
There are three things that you should do, based on what Hasan told me and what the corrected data supports.
First, track the whole conversation, not the first answer. If you’re monitoring AI visibility with a single-prompt check, you’re measuring the top of a funnel that loses 62% of its contents one sentence later. Build your monitoring around the follow-up questions your real buyers actually ask.
Second, fix the assistants one at a time, in order. Hasan was direct that a single content change won’t move all three models at once, because they pull from different sources. His suggested order: correct flat factual errors first, since those are cheap wins, then go after the segment-fit combinations that matter most to your pipeline, checking each assistant across several runs rather than trusting any single answer.
Third, don’t cite a stat you haven’t traced to its source, including this one. Clovion’s own report needed a correction on its most technical, most citable number. Before you build a column, a client deck, or a content brief around any AI research percentage, ask where the underlying count came from and whether anyone’s checked the math since it left the design software.
I’ve watched SEO go through a few of these moments, from Panda to mobile-first indexing to the slow bleed of zero-click search. Each one rewarded the practitioners who checked the primary source instead of repeating the headline number. AI visibility is shaping up the same way. The brands that win the disappearing act Clovion documented won’t be the ones with the best press release about their AI Overviews strategy. They’ll be the ones who read the report closely enough to ask what a “33” really meant, and who keep asking that question after this one.
Zahir Hasan is COO of Clovion AI, based in Oslo, Norway. Clovion’s corrected version of “Surviving the AI Funnel,” reflecting the figures in this column, is expected this week.
Post Comment