French startup ZML has released a free inference server capable of running open-source large language models at high speeds across multiple hardware platforms. This release comes amid skyrocketing AI inference costs and a growing demand from developers to break free of Nvidia’s hardware monopoly.

Key Takeaway: ZML Releases LLMD for Free

On July 8, 2026, Paris-based AI startup ZML released LLMD, a free inference server designed to run large language models (LLMs) across heterogeneous chips. As AI transitions from research to real-world applications, optimizing inference—the process of running a trained model to generate responses—has become far more critical than optimizing model training. Amidst a massive boom in the AI inference market, ZML’s entry has garnered significant attention. The startup has raised \$20 million in early-stage funding from prominent venture capital firms like 20VC and an elite group of angel investors, including Turing Award laureate Yann LeCun, Hugging Face co-founders Clément Delangue and Julien Chaumond, and Docker founder Solomon Hykes. This funding round ranks in the top 1% of comparable early-stage enterprise software deals in France. LLMD allows developers to run open-source models seamlessly and at high speeds on diverse chip architectures, including Nvidia, AMD, Google TPU, Apple Metal, and Intel Arc.

Technical Highlights: A Lightweight, Python-Free Architecture Built on Zig

The most defining feature of ZML’s LLMD is its complete elimination of Python, the traditional language of machine learning development. Built on Zig, a systems programming language, the compiler-driven server compiles models into standalone native binaries. In typical AI inference setups, a patchwork of PyTorch and various Python libraries introduces significant runtime overhead and bloat. In contrast, LLMD adopts a lightweight, Python-free architecture that strips away unnecessary intermediate layers, pushing hardware performance to its theoretical limit. Steeve Morin, co-founder and former VP of Engineering at Zenly (acquired by Snap), stated that the goal of the project is to break existing software silos and squeeze maximum performance out of whatever hardware users already own.
Please refer to the system architecture diagram below.

Challenging Nvidia’s CUDA Lock-In

Industry Hurdles: Nvidia Dominance and Developer Lock-In

Nvidia currently commands a dominant share of the global AI hardware market, largely due to its software ecosystem, CUDA, which has deeply locked in developers. De facto standard frameworks like PyTorch are natively optimized for CUDA. Consequently, attempting to run models on rival hardware—such as AMD GPUs, Google TPUs, or Intel chips—often leads to steep performance degradation or requires massive code refactoring. This high switching cost has historically left enterprises with little choice but to stick with Nvidia. However, as the demand for AI inference skyrockets, severe Nvidia GPU shortages and allocation limits have become a major bottleneck for companies deploying AI models. The industry is actively seeking a portable software stack that can run across mixed hardware setups to break this self-perpetuating monopoly.

The Value of Multi-Chip Operations: Slashing Costs and Enhancing Flexibility

The multi-chip portability offered by ZML’s LLMD directly addresses the urgent challenges of soaring AI operational costs and hardware shortages. By utilizing LLMD, enterprises are no longer bound to a single hardware vendor. Instead, they can dynamically mix and match accelerators based on market availability, power efficiency, and cost. This mirrors the adoption of multi-cloud strategies in enterprise IT, bringing much-needed supply-chain flexibility to the hardware layer. For organizations already owning non-Nvidia hardware assets like AMD, Apple Metal, or Google TPUs—which were previously underutilized due to CUDA barriers—ZML unlocks immediate value. Furthermore, LLMD’s custom compiler technology aims to deliver faster speeds than the hardware manufacturers’ own software stacks, significantly lowering the total cost of ownership (TCO) for customers.

Europe’s Emerging AI Ecosystem and Future Outlook

European Technological Sovereignty: Collaborating with Emerging Chipmakers

ZML’s roots in Paris—the heart of European AI research—are deeply tied to the continent’s push for technological sovereignty. Since launching its National Strategy on AI in 2018, the French government has committed nearly €1 billion to its second phase (2022–2025) to foster innovations in embedded AI, trustworthy AI, and edge computing. ZML stands as a crucial pillar in both the French and broader European AI ecosystems, with Morin noting that the project “couldn’t have been built anywhere but in Paris.” ZML has strategically positioned itself to support emerging European AI chip startups—such as Axelera, Fractile, Kalray, SiPearl, and VSORA—by treating their silicon as first-class hardware on the software level. This collaborative approach aims to pave the way for sovereign European hardware to become highly viable alternatives in the global AI infrastructure market.

Future Outlook and Challenges: Adoption Barriers and Monetization

For now, ZML is distributing LLMD for free to maximize adoption, gather telemetry, and collect developer feedback. However, the startup faces fierce competition in what has been dubbed the “inference gold rush.” Heavyweights like Baseten, recently valued at \$13 billion, and commercial entities built around open-source runtimes like vLLM (such as Inferact) and SGLang (such as RadixArk) are all competing for the same market. For LLMD to truly disrupt the status quo, its real-world performance on mixed-chip setups must prove competitive against Nvidia’s highly optimized, proprietary environments. Since comprehensive, head-to-head public benchmarks are not yet available, many enterprises will likely monitor community adoption and performance stability before committing to production deployments. ZML plans to build a broad user base first before monetizing high-value enterprise features, a trajectory that industry observers will be watching closely.

References

• 【ZML Official Website】https://zml.ai/
• 【ZML/LLMD Product Page】https://zml.ai/llmd/
• 【National Strategy on AI – OECD.AI】https://www.economie.gouv.fr/actualites/strategie-nationale-intelligence-artificielle

【#ScienceAndTechnology #DeepTech #ArtificialIntelligence #Semiconductors #InferenceServer #Startup #OpenSource #France】