How should you score mentions, citations and recommendations?

Use binary, inspectable rules before considering a composite score. The table below is the proposed scoring contract for this pilot.

Illustrative distinction: “Acme is another vendor in this market” is a mention. “For your integration requirement, include Acme in the shortlist” is a recommendation. These are invented examples demonstrating the scoring rule.

What happens when no answer appears?

Keep these outcomes distinct:

• Answer captured: a substantive response is available to score.
• Valid search, no AI feature: for example, the search completed but no AI Overview appeared.
• Refusal or unsupported request: the product responded, but not with a substantive buying answer.
• Technical collection failure: blocked request, timeout, missing capture or unavailable account.

For the main answer-content metrics, use substantive captured answers as the denominator and report all other statuses alongside them. For Google AI Overviews, also show AI-feature appearance rate. If you calculate recommendation exposure across valid searches, count no-AI-feature searches as zero exposure—but name that different denominator explicitly.

A technical failure remains unknown. Do not silently score it as brand absence or retry it indefinitely until a preferred answer appears.

Which metrics should leadership see?

For each surface and prompt cohort, show:

• Mention rate: answers mentioning the brand ÷ scorable answers.
• Owned-citation rate: answers citing an owned domain ÷ scorable answers.
• Recommendation rate: answers recommending the brand ÷ scorable answers.
• Prompt coverage: fully tested prompts with at least one recommendation ÷ fully tested prompts.
• Recurrence: prompts recommended in zero, one, two or all three repetitions.
• Accuracy exceptions: the number and nature of verified discrepancies.
• Collection completeness: successful, no-feature, refusal and failed observations.

Illustrative calculation—not an observed result: across 16 unbranded prompts with three completed answers each, 12 recommendations produce a 25% observation-level recommendation rate. If those recommendations occur across six prompts, prompt coverage is 37.5%. These measures answer different questions.

Keep repetition counts fixed: “at least one recommendation” becomes easier to achieve when you run more tests.

If leadership requests share of voice, define a bounded sampled recommendation share: recommendations for your brand divided by recommendations across the predeclared monitored brands, counting each brand at most once per answer. Name the competitor set. Do not call it US market share.

How do you control scoring errors?

Have a second reviewer inspect all ambiguous recommendations, important inaccuracies and a proposed 20% sample of ordinary results. Resolve disagreements before publishing the scorecard.

An automated classifier may suggest labels, but require an evidence excerpt for each recommendation and keep human overrides. Freeze the rubric and classifier version where possible.

Research on generative-search measurement supports repeated observations rather than treating one response as a definitive result. It does not validate any particular supplier’s proprietary visibility score. (arxiv.org)