Skip to content
Article Nielsen Norman Group Oct 2026

Nielsen Norman Group: why agentic AI systems turn fragile

Why agentic AI systems turn fragile

Tanner Kohler at Nielsen Norman Group published this article on October 2, 2026, examining a pattern among people who build agentic AI systems without engineering backgrounds — people the article calls “vibe architects.” Because AI makes building nearly free, there is little pressure to prioritize or plan. The result is what software engineers call a “big ball of mud”: a system that works until its complexity makes it hard to direct, diagnose, or rebuild.

The article draws on a study of these builders and maps their behaviors onto five established software fragility patterns.

Five patterns

Throwaway code. A quick proof of concept — a rules file, an automation — is never meant to be permanent but becomes something people depend on. It was never documented or tested systematically.

Piecemeal growth. New capabilities are bolted onto existing tools without any plan for how they fit together. The system reaches no finished state; it keeps growing until no one can hold it in their head.

Keep it working. When something breaks or drifts, users patch it with quick AI fixes rather than stepping back to understand why it failed. Over time, this erodes their model of where things live and what each part does.

Sweeping under the rug. Messy files and information are dumped into one unsorted location. The user relies on the AI to find things, which means structure exists only inside the model’s interpretation — not in the files themselves.

Reconstruction. Systems are rebuilt when they no longer fit, when no one understands them anymore, or when a better approach emerges. Reconstruction is sometimes the right choice, but systems built without architecture tend to force it prematurely.

What to do instead

The article recommends maintaining clear separation between three context types: global (applies across the whole system), local (task-specific), and ambient (information streams the agent searches rather than reads in full). When this boundary blurs — a common early sign — the system is already accumulating technical debt.

Other practical habits: name the context type explicitly when asking the agent to make changes; audit global context periodically and keep it short enough to review in one sitting; move local context that is no longer relevant rather than leaving it in place; and revisit the system periodically to check whether each part is still used and understood.

The article ends with a diagnostic question: can you articulate your system’s current limitations and what you would change about it? Being able to answer positions you to direct reconstruction before it becomes a forced reset.

Who it is useful for

Anyone who has built an AI workflow by adding pieces over time — designers using agents in Figma, researchers maintaining interview analysis pipelines, or writers with accumulated instruction files. The article is grounded in observed behavior rather than theoretical models, which makes it useful for recognizing patterns that are already present in your own system.