NNGroup: AI assistant design — why context access matters more than raw capability
Caleb Sponheim at Nielsen Norman Group published this article on October 9, 2026, drawing on a structured test of three AI assistants — the original Siri, the updated Siri AI, and ChatGPT — on real post-travel tasks such as finding a recent receipt, recalling a conversation from last week, or checking a flight itinerary. The central finding is counterintuitive: a less capable model with access to a user’s actual context can outperform a more capable model that requires users to supply information manually.
The test makes the mechanism visible. ChatGPT, despite its generally stronger language capability, required users to describe the situation, paste in relevant text, and reconstruct context that was already present on the device. The updated Siri AI, which has access to device data, messages, and calendar, could address many of the same tasks without that overhead. The original Siri, with access but limited capability, failed on interpretation. The useful combination is both: context access and the capability to use it well.
Sponheim identifies six design principles that follow from the findings. First, AI assistants should reduce user effort by connecting to relevant data sources automatically, not waiting to be given context. Second, when a query falls outside the assistant’s available data, it should explain what it searched and where the boundary is — users cannot troubleshoot absence they cannot see. Third, how the assistant interprets a request should be visible, so users can correct misreads before acting on a wrong output. Fourth, high-stakes actions should require confirmation before execution. Fifth, when a request is genuinely ambiguous, the assistant should ask before acting rather than guessing. Sixth, context should persist through a complete task, including into destination apps — an assistant that answers a question but drops the thread when the user opens another app has ended the task before it is finished.
The article also notes a testing caveat that most AI assistant evaluations skip: the workflow has to be measured end to end, including what happens in the destination application. An assistant that hands off to a broken or confusing endpoint has still failed the user, even if the assistant’s own output was correct.
Useful for designers working on AI assistant interfaces, on-device AI features, or any product where an AI component needs to decide how much context to request versus how much to infer from available data. The framework is concrete enough to apply directly to capability scoping and scope-communication decisions during design.