9. Can we trust an AI visibility checker or a single visibility score?

Trust a score only to the extent that you can inspect its sample, calculation and underlying answers. A useful checker should help reproduce and investigate a finding, not merely display a number.

OpenAI’s documentation on search-query rewriting, memory and location illustrates why the collection environment matters. Our inference is that an API-based test should not be assumed to reproduce a consumer-interface experience without validation. (help.openai.com)

Ask the supplier to demonstrate:

• The exact prompts behind a chart.
• Whether answers come from consumer interfaces, APIs or another collection route.
• The account, search, model and location settings it can actually control.
• How often prompts run and whether cached results are reused.
• How failed collections differ from valid answers with no brand mention.
• How it resolves product aliases and similarly named companies.
• How it distinguishes neutral mentions, recommendations and warnings.
• Whether you can export historical raw answers and citations.
• What happens to the time series when a model or collection method changes.

Run a small, time-matched acceptance test using your own questions. Expect some disagreement because answers vary; investigate systematic differences in surface, wording, location or classification.

Reject false precision: a score of 72.4 is not inherently more decision-useful than a transparent count of relevant recommendations with an uncertainty statement.