NN/G: The core skill of design in the AI era: critique

Most writing about AI in design focuses on which tools to use. This June 2026 article from Nielsen Norman Group researcher Adam Elman focuses instead on a process problem that few design teams have solved: how to evaluate AI outputs when those outputs vary with every run.

The core argument is straightforward. Traditional design specifications work because they describe exact behaviors in deterministic systems — the button does this, the modal appears there. Generative AI breaks that model. The output changes each time, which means a specification cannot tell you whether any given result is good or bad. Something else must.

Elman’s answer is critique — not informal peer feedback, but structured evaluation using objective criteria defined before looking at any model output. Those criteria should come from user research and design intent, not aesthetic preference, and they should be specific enough to apply consistently across different evaluators.

The judge-evaluate-iterate loop

The article describes a three-phase process. In the judge phase, the team writes criteria for what makes a response acceptable, grounded in the actual user needs the feature is meant to address. In the evaluate phase, those criteria are applied to real model outputs, first by humans and then, where scale demands it, by an LLM trained to judge outputs against the established criteria. In the iterate phase, the evaluation results feed back into prompt refinement, model fine-tuning, or updates to the criteria when edge cases reveal gaps.

Elman includes specific technical guidance: target an F1 score of 0.8 when calibrating an LLM judge against human-annotated examples, break complex criteria into separate component judges for scalability, and monitor continuously for regressions — because a change in one part of the system can produce unexpected failures elsewhere.

Who this is useful for

The article is aimed at UX and product designers already working with generative AI features, not at teams in early exploration. It assumes familiarity with how AI-powered features are built. The framework is narrow and methodological — it answers one specific question precisely, leaving broader strategic questions to other resources.

Design teams that have shipped AI features and find informal review loops insufficient will get the most from this. Teams that are still deciding whether to use AI at all will likely find it premature.