ZML: free inference server runs LLMs across NVIDIA, AMD, and Apple chips

ZML, a Paris-based AI infrastructure startup backed by Yann LeCun, released ZML/LLMD — a free, open-source LLM inference server that runs open-source language models across a wide range of hardware. Supported chips include NVIDIA, AMD, Google TPU, Apple Metal, and Intel Arc. The project launched without a monetization plan; the team intends to measure adoption before deciding where to introduce paid tiers.

The core claim is performance portability: ZML/LLMD optimizes inference speed and throughput on each chip type rather than defaulting to NVIDIA with suboptimal results elsewhere. For organizations that already run a mix of hardware — cloud providers, on-premise data centers, or edge deployments — the server could reduce the cost and engineering work of running inference at scale.

For product managers building AI features, the practical implication is in cost modeling. Current AI product economics are dominated by inference costs, and those costs are largely determined by chip availability and pricing. A tool that makes switching between chip vendors feasible without re-engineering the inference stack gives teams more leverage when negotiating infrastructure contracts or absorbing supply fluctuations.

The launch also adds pressure on proprietary inference services. The more capable the open-source tooling becomes, the harder it is to justify vendor lock-in at the infrastructure layer. Teams building on hosted model APIs may continue to do so for convenience, but the price ceiling drops as credible alternatives emerge.