Scorecard · No. 06 / 17
What are the 12 weighted criteria?
Last updated
Measurement relevance and reliability receive 64 of the 100 available points. Implementation and commercial evaluation receive a further 24; accuracy, access and purchasing safeguards receive 12. The mandatory gates prevent those lower numerical weights from making serious risks tradable.
The table gives criterion-specific anchors at 1, 3 and 5. Use 2 and 4 only when the evidence meets the corresponding general standard above.
| Criterion | Weight | Weak: 1/5 | Credible: 3/5 | Excellent: 5/5 | Evidence to request |
|---|---|---|---|---|---|
| 1. Commercial decision and purchase fit | 10 | “Increase AI visibility” is the entire objective | Named buying journey, customer segment, decision, budget owner and reason this purchase is needed | Buyer verifies that the proposed scope resolves an important decision better than existing tools or a smaller diagnostic | One-page decision brief; existing-capability assessment; exclusions |
| 2. Buying-prompt relevance and provenance | 12 | Generic generated prompts; branded queries inflate discovery results | Prompts mapped to customer needs and journey stages; source and inferred status recorded; branded tests separated | Buyer audits provenance, redundant intents and missing requirements; priorities and wording are version-controlled | Prompt register; anonymised sales evidence; customer validation; change history |
| 3. Surface, geography and session fidelity | 8 | “Tracks ChatGPT and Google” without specifying interfaces or settings | Each interface, access route, geography, language, session state and observable model identifier disclosed | Buyer checks comparable examples and documented limitations, including personalisation and unavailable settings | Surface specification; collection logs; side-by-side inspection |
| 4. Sampling, variability and comparability | 14 | Single screenshots or unexplained daily scores | Repeated collection, failures, retry rules, uncertainty and method changes documented | Buyer verifies stability tests, wording sensitivity and an analysis that respects the sample’s limitations | Run schedule; variance analysis; failure ledger; frozen-panel comparison |
| 5. Metric definitions and denominators | 10 | Mentions, citations and recommendations blended into a mystery score | Separate measures, explicit denominators and competitor rules; no invented population exposure | Buyer recalculates metrics, checks edge cases and confirms aggregates do not conceal weak buying-intent segments | Metric dictionary; calculations; classification examples |
| 6. Raw evidence, export and scoring QA | 10 | Dashboard screenshots only; classifications cannot be challenged | Full answer-level evidence and usable export; brand disambiguation and human-review procedure | Buyer audits a mixed sample, resolves disagreements and reproduces reported totals from exported records | CSV/JSON sample; answer archive; annotation rules; QA log |
| 7. Diagnosis and intervention logic | 10 | Generic schema, content-volume or backlink package regardless of the observed gap | Each proposed change links an observation to a plausible mechanism, evidence and test | Buyer verifies source inspection and technical diagnosis; alternatives, dependencies and disconfirming results are documented | Prioritised backlog; exact pages; intervention hypotheses; test plan |
| 8. Delivery capability and ownership | 8 | Senior sales team, unnamed delivery staff; recommendations stop at a slide deck | Named delivery team and buyer owners; capacity, permissions, milestones and acceptance criteria | Relevant work sample is checked from diagnosis through deployment and post-release QA | Team allocation; deployment example; responsibility matrix; escalation route |
| 9. Commercial measurement and attribution discipline | 6 | Citations are translated directly into pipeline or revenue | Visibility, referrals, customer-reported discovery and qualified outcomes are reported separately | Buyer validates instrumentation, duplicate handling and a credible comparison strategy where feasible | Analytics specification; CRM fields; intervention log; reporting example |
| 10. Brand accuracy and correction workflow | 4 | Positive sentiment substitutes for factual checking | Dated product facts, claim-level checks, severity and correction owners defined | Buyer verifies a correction exercise, unresolved-claim handling and recurrence checks | Accuracy register; approved facts; review and escalation evidence |
| 11. Access, privacy, security and continuity | 4 | “Public data only” or “enterprise secure” without supporting detail | Collection basis, data flows, permissions, retention, deletion and outage response documented | Relevant controls and an export/deletion or fallback exercise are buyer-verified | Data-flow map; subprocessor list; security evidence; continuity plan |
| 12. Total cost, contract flexibility and independence | 4 | Low headline fee with unclear scope, add-ons, exit rights or referral incentives | Full pilot cost, usage rules, export rights, cancellation and conflicts disclosed | Buyer verifies the quote against expected usage and tests handover; evaluation cannot be rewritten unilaterally | Cost model; statement of work; exit terms; conflict declaration |
| Total | 100 |