A trustworthy baseline fixes the prompt set and records test conditions. Repeated observations help distinguish a trend from answer variability.
Build a prompt set from real work
Use sales questions, support conversations and category research as inputs. Select prompts across discovery, comparison and evaluation. Keep the first set manageable enough that someone can review every answer, rather than generating hundreds of synthetic questions with unclear value.
Write down the audience, market and buying situation represented by each prompt. A generic question about software and a specific question about a regulated enterprise workflow should not receive the same interpretation.
Record the conditions
Capture the product, model when disclosed, date, region, account state and whether browsing or search was enabled. Keep the exact prompt text and conversation history. Test a clean conversation when evaluating first-contact discovery.
Retain answer text and the URLs actually shown. A mention inferred from a screenshot or a truncated citation list is harder to audit later. Avoid collecting personal information from live customer conversations in your benchmark.
Define consistent labels
Mark whether your brand is present, whether it is recommended, whether a first-party URL is cited and whether a material description is incorrect. Add an optional context note for nuanced cases. Use the same rules for competitors.
For citation rate, divide runs citing a qualifying first-party URL by eligible completed runs. Keep failed or unavailable runs in a separate count. The denominator belongs in the report, next to the percentage.
Repeat before interpreting
Rerun the set under comparable conditions. If answers are volatile, report the range rather than selecting the most favorable run. Small samples are useful for diagnosis but weak evidence for broad market conclusions.
A baseline is an instrument for prioritization. It should reveal which topics and sources deserve investigation, not produce a certainty that the measurement process cannot support.
Your next steps
- Keep exact prompts and raw answers.
- Record test date, region and browsing state.
- Use explicit labels and denominators.
- Separate unavailable runs from negative results.
Source notes & further reading
These primary references support the technical and editorial context. The practical recommendations above are Arrow Signal’s editorial interpretation, not platform guarantees.