Concept testing checklist: before, during, and after the test
This checklist covers the full concept testing process from preparation through decision-making. Use it to make sure you define success criteria before testing, not after — the most common source of bias in concept testing is deciding what “success” means after seeing the data.
Before the test
- Define the research question: comprehension, desirability, preference, or all three
- Set success criteria: what scores or patterns constitute a “pass” for this concept
- Create concept stimuli at appropriate fidelity (description, mockup, prototype, or video)
- If testing multiple concepts, prepare all stimuli to equivalent fidelity to avoid bias
- Write a discussion guide (moderated) or survey (unmoderated) with neutral, non-leading questions
- Include comprehension questions before desirability questions
- For moderated tests: plan the concept exposure moment (how you will present the stimulus)
- Recruit 8-15 participants per concept (qualitative) or 30-100+ (quantitative) from the target audience
During the test
- Present the concept without explaining it — let participants react first
- Ask comprehension questions before desirability: “In your own words, what does this do?”
- Watch for non-verbal reactions during initial exposure (moderated tests)
- Probe every reaction: “Why?” “What specifically?” “Tell me more.”
- For multiple concepts: rotate presentation order to control for order effects
- Record sessions (moderated) or ensure survey captures open-ended responses
After the test
- Calculate comprehension rate: what percentage correctly described the concept
- Calculate desirability scores and analyze distribution (not just averages)
- Code qualitative feedback into themes: appeal, concerns, confusion, comparisons
- Identify patterns across participants — focus on recurring themes, not outliers
- Apply the success criteria from Step 1: does the concept pass or fail?
- If testing multiple concepts: produce a ranked comparison with rationale
- Write a recommendation: proceed, iterate (on what), or kill (why)
- Document the decision and data so the team does not revisit without new evidence