On July 10, 2026, the inference engine “Colibrì,” which runs the large language model “GLM-5.2” on a typical home PC with 25GB of memory, was released. It is a groundbreaking attempt to release top-tier intelligence, which originally requires hundreds of gigabytes of memory, into individual control through streaming using SSDs.

The “magic” that runs 744 billion parameters on a PC with 25GB of memory

On July 10, 2026, developer vforno (JustVugg) announced the inference engine “Colibrì,” which runs the state-of-the-art AI model “GLM-5.2” with an astonishing scale of 744 billion parameters (744B) on a consumer PC with only 25GB of memory. GLM-5.2 is a flagship model developed by China’s Zhipu AI (Z.ai), and has been reported to outperform Anthropic’s Claude Opus 4.7 and Fable 5, supporting contexts up to 1 million tokens. However, the enormous number of parameters meant that, even with memory-saving methods like quantization, operation would normally require hundreds of gigabytes of memory. For example, even an FP8 model requires eight NVIDIA H200 GPUs to operate. Colibrì broke this conventional wisdom by building a system that sequentially reads only the data needed from storage instead of loading all model data into VRAM or RAM, paving the way for feasibility in personal desktop environments. For an image of its operation, please refer to the diagram below.

The developer’s determination to keep moving no matter how slow it is

At the core of this project is the “democratization of operation,” enabling anyone to operate top-tier AI right at their fingertips. From the beginning, Vforno’s top priority was not improving processing speed, but reliably running large models in even low-spec environments. He reported that he successfully ran GLM-5.2 on a standard laptop equipped with a 12-core CPU and 25GB of memory. Development took weeks of enormous nights and the support of coding agents. Built purely in C, this engine marked a historic step in bringing frontier-level intelligence—previously only handled by large corporations with expensive GPU clusters that exhausted video memory—under the control of general users. Developers share the joy of actually chatting in Italian and receiving practical-level responses on communities like Reddit.

The MoE structure supporting giant models and how streaming works

Mixture-of-Experts and 40 billion active parameters

The main reason Colibrì can operate the massive GLM-5.2 in a lightweight environment is its focus on the “Mixture-of-Experts (MoE)” structure adopted by the model. Although GLM-5.2 has a massive total of 744 billion parameters, it does not use all of them for computations for input questions. In the MoE structure, it is divided into 21,504 small networks called “experts” specialized in specific knowledge, and the actual activation of parameters per token is limited to about 40 billion, roughly 5.4 percent of the total. Colibrì skillfully leveraged this characteristic to build a mechanism that calls only a portion of the activated data at the moment of computation. This made it possible to execute models that greatly exceeded the capacity of physical memory by expanding only the parts needed at any given time into memory. Below is a diagram of the engine’s structure.

On-demand streaming from RAM resident layers and SSDs

As a specific data management method, Colibrì separates model data into two layers: the ‘resident layer’ and the ‘streaming layer.’ First, about 17 billion parameters that are always needed throughout the computational process—such as attention mechanisms for determining important parts of text and embedded layers that convert words into numerical data—are compressed to 4-bit and resided in RAM as about 9.9GB of fixed data. On the other hand, the expert group, which can be selected depending on the input, totals about 370GB and is placed in high-speed external storage such as NVMe SSDs. Only the experts needed during response generation are loaded on-demand from storage into RAM and then computed. To reduce SSD access, it includes an LRU cache that holds recently used data, and a “learning cache” that prioritizes frequently used experts in memory based on usage history. This dynamic data swapping is the foundation of memory-saving operation.

Democratizing Inference and the Future Pioneered by Local AI

Local delegation overcoming barriers of cloud regulation and inference costs

The technology Colibrì demonstrates holds social significance beyond mere technical interest. In June 2026, due to the U.S. government’s export control directive, Anthropic’s highest-performance model, “Claude Fable 5,” was shut down for global use just three days after its release, highlighting vulnerabilities in cloud-based AI. In contrast, running open-weight models like GLM-5.2, distributed under the MIT license on personal PCs, This means reclaiming users’ “AI sovereignty,” which is not subject to regulations or vendor preferences. Furthermore, based on the “local delegation theory of inference costs,” even if reasoning takes time, it becomes possible to encourage infinite deep thinking without additional costs beyond electricity costs. For long-term agent operations that require tens of thousands of tokens overnight, this method will be an extremely powerful weapon. The division of roles between cloud and local is expected to be as follows.

Next-generation thinking engines accelerating alongside hardware evolution

The future focus will be on how far the evolution of storage technology will unlock Colibrì’s potential. Currently, the generation speed is extremely slow, about 0.05 to 0.1 tokens per second when the cache is empty, and even short answers can take several minutes. This is because about 11GB of data must be read from the SSD for each token generated, but leveraging next-generation memory technologies like Optane drives that are low-latency and robust for random access could dramatically improve this bottleneck. Currently, CPU-centric processing is also being considered, using a GPU with about 16GB of VRAM to accelerate computation. Developer vforno is calling for sharing benchmark results on PCs equipped with faster SSDs and larger RAM, with further community-driven optimizations expected. A future where massive intelligence quietly thinks inside personal PCs is steadily beginning from this small engine.

[#GLM52 #ローカルAI #Colibrì #推論エンジン #ZhipuAI #科学技術 #大規模言語モデル #MoE]