3. How many prompts and repeated runs are enough to distinguish visibility lift from sampling noise?

There should be no universal sample-size answer without specifying the quantity being estimated and the decision it must support.

Put to: Rand Fishkin and an independent statistician.

Why it matters to buyers: NIST’s statistical guidance relates sample requirements to assumptions about the proportion being estimated and the desired precision. It does not provide a universal minimum number of AI-search prompts. (itl.nist.gov)

Ask: “Are you estimating behaviour within our fixed test set, across a category’s buying intents, or among actual US buyers? What sampling design supports that claim?”

A good answer should cover:

• Separate uncertainty from repeated answers, prompt selection, timing and scoring.
• Explain the assumptions behind any interval, including whether observations are independent enough for the method used.
• Specify the smallest change worth acting on before increasing collection volume.
• Examine stability across dates and intent groups—not only a pooled total.
• Explain what additional repetitions cannot resolve about an unrepresentative prompt library.

Illustrative triage design: six prompts, two surfaces and three repetitions produce 36 observations, not 36 customers. We would treat this as an initial recurrence check, not a precise estimate of market-wide exposure.

Press for: a written rule for “collect more”, “investigate” and “act”. Reject confidence language that is unsupported by a disclosed method.

Expert quote — pending: ____________________