You ask ChatGPT for the best roofer in your city and your company is on the list. A colleague tries the same question an hour later and you're gone. Is something broken?
No. Variation is normal, and it's one of the most important things to understand about AI recommendations. This guide explains what we saw when we measured it, why it happens, and how to check your own standing without being fooled by one lucky or unlucky answer.
What we measured
In November 2026 we asked ChatGPT three customer-style questions about each of 390 trade-and-city combinations in Texas, each question twice. That's 2,238 answers (six per market in 356 markets, three in the other 34). We read the public snapshots behind our Who AI Recommends pages. The engine was ChatGPT's search through OpenAI's API, so the consumer app may differ.
What we found:
- ChatGPT usually has a favorite. On average the top business appeared in 86.8% of answers in its market, and in 48% of markets it appeared in all six.
- The rest rotates. On average only 1.4 businesses per market were named in at least five of six answers. In about 22% of markets, no business was named that reliably.
- Many names appear once. At least 35% of the named businesses showed up in only one of the six answers (a floor, because our stored lists stop at 12 names).
- It differs by trade. Painters and pest control were the most settled (the top business was named in all six answers in about 82% of their markets). General contractors were the least (13%). Plumbers (26%) and HVAC (35%) were among the more variable.
So "named in some answers" is a real and common state, especially for the businesses just below the favorite.
Why answers change
Nobody outside OpenAI can give the full list, but several causes are well understood:
- Wording. "Who's the best plumber in Garland?" and "Can you recommend a good plumber in Garland? Give me a short list" can lead to different searches and different pages being read.
- Searching the live web. ChatGPT search looks things up each time, and results shift as pages, listings and rankings change.
- Location. OpenAI says ChatGPT can share your location with trusted providers to supply local information (OpenAI help). Someone in a different suburb may see a different list.
- Chat context. If you're signed in, earlier conversations may shape an answer.
- Model updates. The model behind ChatGPT changes over time.
- Chance. Language models produce answers with some randomness, so two runs of the same question can differ even with the same sources.
Other tools show the same pattern. Only about 5% of citations overlap between ChatGPT, Perplexity and Gemini in Yext's research (Yext).
Why this matters for you
A single answer can mislead you in either direction.
- If you're named once, you might be in the rotation, not the favorite.
- If you're missing once, you might be named most of the time.
- If you fix something and check once, you can't tell whether it worked.
It also explains why a customer may say "ChatGPT told me about your competitor" even though ChatGPT names you at other times.
How to measure it fairly
Use the same method every time and count, instead of judging one answer:
- Pick three questions customers ask, such as "Who is the best [trade] in [city]?"
- Ask each at least twice in a fresh chat each time (three runs is better).
- Count: "named in X of N answers." That is your number.
- Record the sources cited each time.
- Repeat monthly on the same day, with the same wording.
The full steps are in the five-minute test. Researchers face the same problem. SOCi, for example, counted a business as "likely recommended" if it appeared in an AI tool's top five for a market (SOCi), because single answers are too noisy.
What the pattern suggests (carefully)
- Moving from "sometimes" to "usually" is the realistic goal. The businesses named in five of six answers were few. Being in that group seems to take a stable, consistent presence across the sources AI reads. We can't promise any specific step will get you there.
- Consistency of facts helps. Wrong phone numbers, old addresses and mixed-up names across sites can make an AI tool less sure. See what to do if ChatGPT shows the wrong phone number or hours.
- Freshness appears to matter. Businesses with recent reviews were named far more often in one study. See how many reviews and how recent.
- Sources matter more than a single answer. What ChatGPT cites tells you where to look. See the directories ChatGPT reads.
A worked example
Say you run an HVAC company in Frisco. You ask three questions, twice each, and get six answers. You're named in two. That's "named in 2 of 6 answers." Next month you fix your listings, send a steady stream of review requests, and ask the same six questions again. This time you're named in four. That is a better sign than any single answer, though it still isn't proof: the season, the wording or a model update could account for some of it. If it holds for another month, you can be more confident. If it slips back, you've learned that too. Either way, you have a number to track instead of a feeling.
What not to do
- Don't panic about one missing answer, and don't celebrate one lucky one.
- Don't keep re-asking until you get the answer you want. Count every run.
- Don't pay for a service that promises a fixed position. Nobody controls it, and we never guarantee rankings or recommendations.
- Don't change five things at once and then credit one. Change a few, then re-check.
FAQ
How many times should I ask? At least two per question, ideally three, across three or more questions. More runs give a steadier number.
Do the other AI tools vary too? Yes. Google's AI, Perplexity and Gemini each use different sources, so check them separately.
Is it worse in some cities? In our data, larger cities were slightly less settled: in the five largest Texas cities, 10.4% of markets had no business named in most answers, versus 2.7% in the smaller cities. That is suggestive, not conclusive.
Skip the guesswork. Run a free scan at localpickai.com. It asks the questions several times, shows how often you're named, who's named instead, which websites AI read, and what to fix first. Free, no signup, about a minute.
