Nielsen Norman Group: One AI output is an example, not an evaluation
What the article is about
Raluca Budiu, a principal researcher at Nielsen Norman Group, published this article in August 2026 to address a common mistake in how product and design teams evaluate AI features: testing a feature once, seeing it work, and concluding it works reliably.
The core argument is precise. A single successful output demonstrates that a system can perform a task. It does not show how often or how consistently the system will do so. This distinction matters because language models are nondeterministic — they sample from probability distributions rather than follow fixed rules. The same input can produce outputs of different quality on different runs.
The problem with single-run testing
Product teams trained on deterministic software expect consistent behavior across runs. A feature either works or it does not. AI systems do not work this way, and evaluating them as if they do produces misleading conclusions.
Budiu describes two types of variability that evaluations need to measure separately. Test-input variability captures performance across different inputs — whether the system performs well on easy cases but fails on edge cases, or handles variation well. Run-to-run variability captures inconsistency when the same input produces outputs of different quality at different times.
The article demonstrates that two AI systems with identical success rates can have fundamentally different failure patterns. One might be erratic — performing well on some runs and poorly on others with no discernible pattern. The other might be predictable — performing well on most input types but failing consistently on a specific category. A single overall success rate obscures this distinction, which matters for deciding whether a feature is production-ready.
What rigorous evaluation looks like
The framework the article proposes applies quantitative research principles to AI evaluation:
- Multiple representative inputs, not just examples the team already knows work
- Repeated runs per input to measure consistency across the probability distribution
- Averages with confidence intervals rather than single-point success rates
Consequential decisions — launching a feature, selecting an AI vendor, claiming quality improvements — warrant this level of rigor. Early exploration may reasonably rely on single runs, but production decisions should not.
Who this is useful for
Useful for designers and product managers responsible for AI feature quality. Particularly relevant for teams preparing to ship AI features and for those conducting vendor evaluations. The framework also applies to user research: testing AI interfaces requires the same care about input variation and run consistency that the article recommends for technical evaluation.