GrowthBook 5.0: Warehouse-native experimentation with AI evaluations

GrowthBook released version 5.0 on August 4, 2026, appearing on the same day’s Product Hunt leaderboard. The platform provides feature flags, A/B experimentation, and product analytics, and version 5.0 extends it specifically to support teams building and testing AI features alongside traditional product work.

The platform is open source and warehouse-native, meaning experiment data stays in the organization’s own data infrastructure rather than a vendor’s storage. GrowthBook claims over 3,000 organizations use it, and the 5.0 update centers on three additions that are relevant for teams working with LLMs.

What’s new in 5.0

The AI Data Analyst feature lets teams query experiment results in natural language. Instead of writing SQL or waiting for a BI request, a PM can ask “which variant performed better for mobile users in the EU last week” and get an answer without leaving the experimentation platform. The claim is a 5x increase in experiment velocity through faster iteration cycles enabled by removing the analysis bottleneck.

AI Evaluations add a native evaluation layer for LLM-powered features. Teams can run structured tests on model outputs alongside traditional metric-based experiments, which addresses a specific gap: most A/B testing infrastructure was built for deterministic features, and testing whether one prompt produces better outputs than another requires a different kind of measurement. GrowthBook 5.0 treats AI evals as a first-class feature type within the same experimentation workflow.

An MCP server integration allows AI agents to read experiment data, query results, and trigger flag changes as part of automated workflows. This enables scenarios like an agent that monitors an experiment, detects a significant result, and takes a configured action — pausing a test or alerting the team — without manual intervention.

GrowthBook also reports a 50% cost reduction relative to commercial alternatives in its own benchmarks. Organizations self-hosting on their warehouse pay for their own infrastructure rather than per-seat or per-event pricing.

Why it matters for product managers

The pairing of traditional A/B testing with native LLM evaluation matters practically for teams that have begun shipping AI features but still rely on separate tools for each type of measurement. A unified platform reduces the coordination overhead between the experimentation team and whoever owns AI quality, and it keeps all experiment data in one place for reporting and iteration.

The natural language query feature is most relevant for product managers who currently depend on data teams for experiment analysis and want to reduce that dependency during early rounds of iteration, before a result is significant enough to warrant a formal analysis request. Whether the AI Data Analyst feature performs accurately on complex experiment queries is worth validating for a given organization’s data schema before treating it as a replacement for SQL-based analysis.