5. Are we measuring the consumer experience or an API approximation?

Require the exact collection surface and configuration; an engine name is insufficient.

Put to: Mike King and Patrick Stox, with each shortlisted supplier.

Why it matters to buyers: OpenAI documents location and memory-related context in ChatGPT search, while its API documentation separately describes configurable web-search location settings. This supports treating collection configuration as a material part of the evidence—not assuming two products are equivalent because they share a provider. (help.openai.com)

Ask: “When your dashboard says ChatGPT, Google AI Mode or Perplexity, which product did you actually query, under what conditions, and how have you validated that observation against the experience we care about?”

A good answer should cover:

• Consumer interface, developer API, third-party data feed or another collection route.
• Model or mode where disclosed, search enabled or disabled, account state and session history.
• US localisation, language, device assumptions and unavailable settings.
• Fresh-session testing versus realistic multi-turn purchasing conversations.
• Configuration changes that require a reporting break rather than a continuous trend line.

Press for: a small matched comparison with saved outputs, a written explanation of discrepancies and disclosure of any dimensions the supplier cannot control.

Do not accept “human-like” or “real-time” as substitutes for this specification.

Expert quote — pending: ____________________