There Is No #1 in an AI Answer
Somebody is going to offer to get you ranked first in ChatGPT. They may show you a dashboard with a position number on it, tracked over time, trending in a satisfying direction.
The number is not measuring what the dashboard implies it is measuring. Not because the vendor is dishonest — many believe it — but because the thing being counted does not work the way a search ranking works.
What a search ranking is
When you rank third on Google for a phrase, that three is a real property of a real system. There is an index. There is an ordering. Two people searching the same phrase in the same place at the same moment get substantially the same list. The position is stable enough to measure on Tuesday and compare to Thursday.
That stability is what makes rank tracking meaningful. You are sampling something that holds still.
What a generated answer is
A language model does not look up an answer and return it. It produces text one piece at a time, choosing each next piece from a probability distribution. Unless the sampling is pinned to always take the most likely option — and in consumer products it generally is not, because that makes for flat, repetitive writing — the same question asked twice produces two different answers.
Layer on top of that: which sources were retrieved for this particular question, how the question happened to be phrased, what else is in the conversation, what the system knows about the person asking, and which version of the model answered. Every one of those varies.
So "position one" has nothing to attach to. There is no list being ordered. There is a sentence being written, and your business is either mentioned in it or not, this time.
Try it yourself
Ask an assistant the same commercial question about your industry five times, in five fresh conversations. Not five variations — the identical question, five times.
You will usually get five different answers. Often different businesses named. Sometimes a different number of them. Occasionally one that names nobody at all and tells you how to choose instead.
That is not a malfunction, and it is not the model being unreliable. It is what generation is. Any metric that reports a single stable position across that is smoothing away the actual behavior to produce a number that looks like the SEO metric you already understand.
What you can honestly measure
The variance is the problem, so the answer is to measure across it rather than pretend it away.
Frequency of mention. Take thirty or forty questions a real customer might ask. Ask each several times. Count how often you are named. That gives you something like a share of voice — a percentage with a real margin of error, not a rank.
Whether you are cited or merely mentioned. Being named in prose and being linked as a source are different outcomes with different value. Only one of them can send you a visitor.
What the answer says about you when it does name you. This is the one most people skip and it is frequently the most useful. If assistants consistently describe your business with a detail that is out of date, or omit the service you most want to sell, that is a content problem you can actually fix — and you would never see it in a position number.
Direction over time, coarsely. Run the same battery quarterly. Look for meaningful movement, not weekly wiggle. Most week-to-week change in this space is sampling noise wearing a suit.
Why the fake metric is worse than no metric
A position number invites you to optimize toward it. And because it is mostly noise, it will move on its own — which means it will occasionally reward whatever you happened to do last week, and occasionally punish it. You end up learning superstitions.
Frequency measured across many samples is less satisfying and considerably more honest. It also degrades gracefully: if a model version changes underneath you, a share-of-voice figure shifts and you can see it shifted, rather than a rank inexplicably jumping from 2 to 9.
What this does not mean
It does not mean AI visibility is unmeasurable or that nothing you do matters. Being genuinely well known, clearly described, widely referenced and technically readable makes you more likely to be retrieved and named. That is real, and it is worth working on.
What it means is that the work should be judged by a metric that matches the mechanism. Ask any vendor selling you an AI rank how the number is produced: how many samples, how much variance, what happens when the model version changes. A good answer exists. Sellers who have one will be glad you asked.
Next steps
If you want to know how assistants currently describe your business — and, more usefully, what they get wrong about it — that is a concrete thing to look at. It is the substance behind our AI recommendation work.
Ask us what the assistants are saying about you. We will tell you what we can measure and what we cannot, which is most of the value.
