Direct questions · No. 07 / 31
7. How many prompts and repeated runs do we need?
Last updated
There is no universally justified prompt count or repetition count for every category, engine and decision. Start with a manageable diagnostic, estimate variability, then expand the parts of the sample that could change the buying decision.
A March 2026 statistical preprint studies repeated citation sampling and argues that visibility should be treated as an estimate of a response distribution rather than a fixed number. Its experiments cover particular platforms and consumer topics, so they do not establish a universal B2B SaaS sampling requirement. (arxiv.org)
Our proposed progression is:
| Decision | Starting design | What it can support |
|---|---|---|
| Is the screenshot worth investigating? | Six questions, two surfaces, three observations per question–surface pair | Initial triage |
| Which buying themes have recurring gaps? | Expand to a validated prompt library and collect across multiple dates | Prioritisation |
| Did a supplier’s work improve visibility? | Fixed baseline, repeated follow-up, comparison groups and uncertainty analysis | A stronger intervention assessment |
| Should we scale expenditure? | Visibility evidence plus implementation costs and commercial observations | A budget decision |
These are design suggestions, not statistical guarantees. Add repetitions where answers vary materially; add new prompts where an important buying need is missing. Those solve different problems.
Repeated runs are not independent buyers. Report the number of prompt families, surfaces, dates, completed observations and failures—not just a large aggregate answer count.
Sources cited on this page
- arxiv.org arxiv.org/abs/2603.08924