13. How should we compare AI visibility software without comparing incompatible scores?

Run a buyer-owned acceptance test before choosing a dashboard.

Put to: Patrick Stox and the technical leads of shortlisted vendors.

Why it matters to buyers: Export claims require inspection. Profound documents prompt and topic exports, while its product page separately advertises raw-data CSV exports. Buyers should verify the fields, coverage and contractual rights available in their actual plan rather than assume every export contains the complete evidence record. (help.tryprofound.com)

Ask: “For a Profound vs Scrunch, Peec AI vs OtterlyAI or Semrush vs Ahrefs evaluation, which tests would establish fit without assuming their visibility scores are directly comparable?”

A good answer should cover:

• The same buyer-approved questions and named surfaces where comparable collection is possible.
• Clear disclosure where methods differ.
• Raw-answer retrieval, timestamps, citations, aliases, failure states and historical exports.
• Manual review of sampled recommendation and accuracy classifications.
• Prompt-edit history, model changes and backfill policies.
• Workflow fit, implementation ownership and full contractual cost.

Press for: a live export that an independent analyst can use without the vendor’s dashboard.

Proposed rejection rule: if the supplier cannot explain a metric, reproduce its calculation from retained evidence or disclose a material limitation, do not use that metric to judge agency performance.

Expert quote — pending: ____________________