AI answers change every week. Here is why one check is not enough
The same question can get a different answer every time you ask, so a tourism board needs a steady weekly count, not a screenshot.

Ask ChatGPT for the best beach towns for a family weekend. Then ask again. There is a good chance the second list is different. Ask next week and it may change again.
For a tourism board, this matters. If you check once, take a screenshot and put it in a board report, you may be reporting a good day or a bad one. One answer is a snapshot. It is not a measure.
How much answers move
Here is a sample, with made-up numbers, for Port Juniper (a town we invented for examples). It shows how often AI named Port Juniper on discovery questions, the ones that describe a trip and never name the town:
| Week | Named in |
|---|---|
| Week 1 | 31% of answers |
| Week 2 | 26% of answers |
| Week 3 | 34% of answers |
| Week 4 | 30% of answers |
Nothing changed in Port Juniper over those four weeks. No campaign, no new page, no press. The number still moved eight points from its low to its high. If the marketing team had checked only in week 2, it would have looked like a slump. In week 3 it would have looked like a win.
Why AI answers change
There are a few reasons, and most of them have nothing to do with you.
AI does not give one fixed answer. Assistants write each answer fresh, with some chance built in. The same question can produce a list of five towns one time and a different five the next, especially when several places are a close call.
Assistants search the web as they answer. Many of them look things up before replying. The pages they find can differ from one day to the next, and the answer follows the pages.
The models change. The companies behind these assistants update them often, sometimes without notice. A new version can favor different sources or different places.
The web changes. A new magazine roundup, a viral forum thread or a review site's updated list can shift answers within weeks.
The seasons change. Ask about beach weekends in January and in June and you may get different towns, even with the same wording.
What one check misses
Because answers move this much, a single look can mislead you in both directions:
- You can miss a real problem. If a rival has been gaining for a month, one good week can hide it.
- You can chase a false one. A dip that comes back by itself next week does not need a new campaign.
- You cannot tell if your work paid off. If you rewrite a page in March, you need March, April and May on the same footing to see whether answers changed.
How to tell a real change from noise
You do not need statistics training. You need a few habits.
- Ask the same questions, worded the same way, every time. If the wording changes, you cannot tell whether the answer changed or the question did.
- Ask each question more than once. A count across several answers is steadier than any single answer.
- Ask more than one assistant. They read different sites and move at different times.
- Compare like with like. If an assistant was missing from one week's check, leave it out of the comparison.
- Wait for a trend. A change that holds for three or four checks in a row is worth acting on. A change that lasts one week usually is not.
How Radar handles it
Radar by SPLASHY is built around these habits, so the numbers your team sees are ones you can act on.
- The same questions every week. Your questions stay fixed. If you reword one, Radar starts it fresh as a new question, so a change in wording never looks like a change in what AI says.
- Several answers per question. ChatGPT, Claude, Gemini and Perplexity are each asked every question three times per check. Google AI Overviews is asked once.
- Small moves are labelled as small. On the Overview, a change of less than two points is shown as about the same, because answers wobble that much on their own.
- Fair comparisons. When an assistant fails or is added, Radar compares only the assistants that were checked both times, and says so.
- History you can see. On the Prompts tab, every question shows its last six checks, so you can see at a glance whether a question is steady, climbing or slipping.
The result is a weekly number you can put in a report and defend, and a record that shows whether your work moved it.
Start with a baseline
The best time to start counting is before you need the number. A few weeks of steady data gives you a baseline, and every campaign after that has something to be measured against.
You can run a free sample sweep to see what AI says about your destination today: ten questions, five assistants, in a few minutes.