NN/g publishes AI interviewer study results

Nielsen Norman Group published the results of their hands-on evaluation of AI-moderated interview tools, testing Marvin and UserFlix with 10 research professionals across 8 countries. The study produced the first evidence-based framework for deciding when AI interviewers are appropriate.

The key finding is that AI interviewers work for four specific scenarios: product feedback collection, recruitment screening, multilingual interviews, and structured studies requiring consistency across participants. They do not work for exploratory research, high-stakes decisions, or studies requiring domain expertise and real-time judgment.

For the research community, this study provides the kind of empirical evidence that has been missing from the AI interview debate. Most previous opinions were based on vendor demos or theoretical arguments. NN/g’s comparative evaluation, using experienced research professionals as participants, provides a credible reference point for teams deciding whether to invest in AI moderation tools. The study also establishes that the experience can feel “almost conversational” but remains unnatural, which has implications for participant data quality.