Next steps · No. 09 / 11
Stage 6: How will you know whether to continue, change or stop?
Last updated
Judge implementation, visibility and commercial evidence separately. A completed project can be valuable without proving AI recommendation lift; an improved recommendation rate can exist without proving incremental revenue.
Review checklist
- Were the agreed changes deployed and independently checked?
- Are the pre- and post-intervention prompts, surfaces and scoring comparable?
- Are failures and missing observations disclosed?
- Does the apparent change recur across collection dates rather than depend on one run?
- Were prompt groups and surfaces reported separately before aggregation?
- Were product inaccuracies corrected without creating unsupported claims?
- Could model changes, wider demand changes or other marketing activity explain the movement?
- Is further measurement likely to change the funding decision?
Where the decision warrants statistical inference, ask an analyst to account for repeated observations within prompts and collection periods. Do not assume that 144 observations represent 144 independent buyers.
How should visibility connect to pipeline?
Use three parallel evidence streams:
- Observed answers: what appeared in the disclosed benchmark.
- Recorded visits and key events: what analytics actually captured.
- Customer evidence: what prospects report using during research, stored with appropriate permissions and qualification context.
For GA4, retain source, medium and channel information rather than filtering only for the historical referral medium: recognised assistant visits can now receive ai-assistant. Google also documents that Direct traffic lacks a clear referral source; do not reclassify it as AI traffic without additional evidence. (support.google.com)
Ask an open question such as, “Which resources helped you identify or evaluate suppliers?” Record AI-assisted research as one possible influence, not necessarily the origin of the opportunity. Avoid adding self-reported and analytics-attributed pipeline without deduplication.
What should the final decision look like?
| Finding | Recommended decision |
|---|---|
| Comparable evidence suggests useful progress; implementation is sound; another defined question remains | Continue narrowly, with an updated hypothesis and capped scope |
| Work was completed, but the relevant visibility gap remains | Change the hypothesis or supplier capability, rather than automatically increasing output volume |
| Collection or scoring changed enough to undermine comparison | Repair measurement before judging performance |
| A genuine accuracy or access problem was fixed, but recommendation lift is unproven | Record the operational benefit honestly; do not relabel it proven GEO growth |
| No actionable gap, no implementation capacity or no decision value remains | Stop or reduce monitoring and document the re-entry trigger |
Sources cited on this page
- support.google.com support.google.com/analytics/answer/9164320