Mani Grewal: Building an AI support copilot — an end-to-end product lifecycle

Mani Grewal, who writes the Product Management with Mani newsletter, published this piece to demonstrate what end-to-end product work looks like when building an AI product—starting not from a technology decision but from a real user problem with measurable outcomes.

The use case is a customer support copilot for support agents. The article takes the reader through each phase of the product lifecycle: problem framing, system architecture, evaluation design, and a phased rollout plan.

Grewal’s starting point is a reframing of the opportunity: the problem is not “add AI to support” but “reduce resolution time while improving quality for agents handling refund requests.” This narrowing of scope is deliberate—the first version targets only one ticket category, which limits the risk of early errors while generating clean evaluation data. The article follows the product through four development stages: response drafting, ticket summarization, recommended actions, and limited autonomous execution. Each stage adds capability only after the previous one proves reliable.

The technical framing is structured around three layers: experience (what agents see and do), intelligence (what the model does), and data (what feeds the model and gets logged). Grewal’s argument is that PMs who focus only on the model miss how much product impact comes from getting the experience and data layers right.

The evaluation framework combines business metrics (handle time, resolution rate), user metrics (agent acceptance rate, CSAT), and model metrics (hallucination rate, latency). The article is direct about a common failure pattern: teams optimize for model metrics during development and then discover that actual handle time went up, not down, because agents spend time correcting AI output rather than using it.

The article returns throughout to a single framing: shipping an AI product means designing a decision-support system under uncertainty, not shipping a model. That means defining what confidence threshold triggers which action, what the fallback is when the model is wrong, and how the system improves as agents interact with it over time.

The piece is most useful for PMs building their first AI-assisted workflow in a domain where errors have real consequences—customer support, operations, or any context where false positives carry more cost than a poor user experience.