NN/g: Humanizing AI is a trap

Caleb Sponheim at Nielsen Norman Group examines a design pattern that has become common across AI products: giving language models personality, warmth, and emotional language to make them feel more approachable. The article argues this strategy produces the opposite of its intended effect.

Large language models already invite anthropomorphization through their fluency and context-appropriate responses. When product teams amplify this tendency further — adding personality modes, emotional acknowledgments, and conversational pleasantries — they are building user expectations that AI systems cannot meet.

The research findings are specific. Models tuned for warmth and empathy showed error rates 10–30% higher than baseline versions. When users perceive an AI as a person, they engage conversationally for its own sake rather than pursuing productive tasks. Studies found that attributing emotional traits to AI makes users less likely to accept its recommendations — the opposite of what humanization intends. Users also overshare sensitive information when they feel they are talking to something person-like, creating privacy risks even when systems are designed with protections in mind.

Documented real-world cases include users developing romantic attachments to chatbots and experiencing genuine emotional distress when services changed or shut down. These are not fringe outcomes — they are predictable consequences of design choices.

The alternative Sponheim proposes is simpler: optimize for practical utility. Write system prompts that prioritize accuracy over agreeableness. Design interfaces that emphasize the tool nature of the system rather than its personality. Make it clear what the AI is and is not. Users who understand they are interacting with a tool apply more appropriate expectations and use the system more effectively.

For designers building AI-powered features, the article is a useful corrective to the pressure — often from marketing and product strategy — to give AI interfaces more personality. The question worth asking is whether adding warmth to a system actually makes it more useful, or whether it only makes it feel more comfortable in a way that causes problems later.

The argument is not that AI should feel cold or hostile. It is that the design goal should be clarity about what the system does well, not the creation of an illusion of personhood.