Why does ChatGPT give a different answer every time?

It doesn't look up one fixed answer. It generates one from many it could give, and the wording of the question, the engine and your location all change the result. That's why one check tells you very little.

What is a good mention rate?

There's no universal benchmark. Compare yourself with the competitors named for the same questions, and watch whether your own rate goes up month over month.

How many times should I run each prompt?

At least 3 to 5 runs per question, in the same setup, to see a pattern. Trackers run more, which gives a steadier number.

Why doesn't AI traffic show up in Google Analytics?

Some AI platforms don't pass referral data, so those visits land as direct traffic. Treat AI referrals in analytics as an undercount.

blog / Tracking

How to measure AI visibility when the same prompt never gives the same answer twice

Updated October 29, 2026 · 7 min read · By the Selka research team

short answer

Measure a rate, not a rank. Ask the same buyer questions several times in the same setup and count how often you're named (your mention rate) and where you appear when you are. SparkToro and Gumshoe found less than a 1 in 100 chance that two runs return the same list of brands, so a single screenshot proves very little.

Two runs of the same prompt almost never return the same brand list: under 1 in 100, per SparkToro and Gumshoe.
The number worth tracking is your mention rate: how often you're named across many runs of your buyers' questions.
Fix the words, the setup and the location, run each question 3 to 5 times, and repeat monthly.
Mention rate doesn't show clicks. Add AI referrals, AI bot visits and a "how did you hear about us?" question.

Why AI never gives the same answer twice

Ask ChatGPT the same question 100 times and you'll almost never get the same list of brands twice. In SparkToro's study with Gumshoe, the chance of an identical list was less than 1 in 100. The same list in the same order was closer to 1 in 1,000. Three things drive it:

The model.It doesn't look up one fixed answer. It picks from many it could give.
The wording. Change one word in the question and you can vanish from the answer.
The engine. ChatGPT, Perplexity, Gemini and Google answer the same question differently.

So “we rank #1 in ChatGPT” usually means someone checked once. AI visibility isn't a position. It's a rate.

The only number worth tracking

The useful number is your mention rate: out of every time a buyer question gets asked, how often are you named? Add your average position when you are. Across all your buyers' questions, that's your share of voice. It's keyword ranking for AI, measured over many runs instead of one.

SparkToro's study points the same way: the exact lists changed constantly, but how often certain brands appeared was much steadier. That steadier number is the one to watch.

Five runs, five different lists. Count how often you appear, not where you landed once.

Our method, step by step

Step	What to do	Why
1. Same words	Copy each question exactly, every time	Change one word and it's a different test
2. Same setup	Logged out, web search on, note the date and model	Settings change answers
3. Same location	Fix your location, or run it from each market you sell in	Logged-out ChatGPT answers based on where you are
4. 3 to 5 runs	Score brands by how many runs they appear in	One run is one roll of the dice
5. Record everything	Brands in order, labels like "best value", prices, sources	Sources tell you why a brand shows up
6. Repeat monthly	Same questions, same setup	One month is noise. Three months is a trend.

Step 3 caught us out. Run from India, our budget questions came back in rupees, with Indian tools first. Step 5 matters most, because the sources are the part you can act on: they're the pages to get onto. We used this method for our test of 11 buyer questions.

What mention rate can't tell you

Mention rate shows whether AI talks about you. It doesn't show whether anyone clicks. Three signals help, and none is complete on its own:

AI referrals.They show up in GA4 or Ahrefs Web Analytics. But some AI platforms don't pass referral data, so that traffic lands as direct. Treat referrals as an undercount, and check which pages AI sends people to.
AI bot visits.AI crawlers read your pages far more often than people visit them. Bot analytics, like Ahrefs' on Cloudflare, shows which pages they read.
“How did you hear about us?” People often hear about you in AI, then search your name or type your URL later. That never shows up as AI traffic, so ask them in your signup or checkout.

Together these get close. Alone, each one misleads.

The question to ask instead

Stop asking whether you rank in ChatGPT. Ask how often you show up across your buyers' questions, and whether that number is going up. Selka's AI tracker is built around this: every number shows how many runs it's based on. Read our methodology for the details.

Sources

Facts checked October 29, 2026. See all of our findings on the research page.