PrismML has released “Ternary Bonsai 2 27B,” a ternary model that compresses a 27B-class large language model down to approximately 6.7GB. This development is driven by background issues such as increasing costs due to traditional cloud dependency, a sharp surge in AI energy consumption, and challenges in protecting confidential data.
- Key takeaway of the event: Ultra-compression of a 27B-class model down to 6.7GB
- Mechanism of Ternary Quantization and Computational Efficiency Improvement
- Evolution from the First Generation and 98.2% Accuracy Retention Rate Based on Qwen 3.8
- System Requirements and Specs for Smooth Operation on 12GB VRAM Class PCs
- Practicality as an AI Agent and Complex Task Execution Capabilities
- Confidential Data Protection and Solutions to the AI Energy Crisis
- Future Outlook and the Future Opened Up by On-Device AI
Key takeaway of the event: Ultra-compression of a 27B-class model down to 6.7GB
On September 17, 2026, PrismML, an AI startup originating from the California Institute of Technology (Caltech), released its latest ternary quantization model, “Ternary Bonsai 2 27B.” Based on “Qwen3.8-27B,” a high-performance foundation model with 27 billion (27B) parameters developed by Alibaba, this open-source language model minimizes weight representation to the extreme. While conventional standard 16-bit floating-point (FP16) formats require a vast VRAM capacity of approximately 54GB to run a 27B-class model—making expensive datacenter GPU server environments indispensable—Ternary Bonsai 2 27B successfully reduces the model size to a mere 6.7GB (equivalent to a theoretical 1.72 bits). This achieves an overwhelming capacity reduction of over 85% compared to the traditional FP16 format. Furthermore, the model is licensed under Apache 2.0, allowing anyone to obtain and integrate it for free, including for commercial use. The following figure illustrates the compression rate of the model size.

This announcement is drawing significant attention from developers and companies around the world as a historic turning point for edge AI, unleashing massive language models from data centers to local devices.
Mechanism of Ternary Quantization and Computational Efficiency Improvement
Ternary quantization is an extreme compression technique that restricts and represents the numerical values of parameters inside LLMs using only three states: “-1”, “0”, and “+1”. From an information-theoretic standpoint, the amount of information required to represent three states is approximately 1.58 bits (log2(3) ≒ 1.585), and as an effective representation, it is encoded to approximately 1.71 to 1.72 bits by combining 16-bit scale values shared across every 128 weights. While this is an approach that drastically drops the amount of information from conventional 16-bit precision to nearly one-tenth in a single stroke, the greatest advantage of this technology extends far beyond just memory reduction.
In computational processing, the multiplication operations that were previously essential for accumulating weights and input values become almost unnecessary, replaced by simple operations centered around sign inversion and addition/subtraction. Since matrix multiplications—which consume the most power and carry the highest computational load in deep learning inference processing—are reduced, it becomes possible to run inferences at high speed and with low power consumption even on standard CPUs, NPUs, and smartphone processors not equipped with high-performance GPUs. To prevent the rapid degradation in accuracy that tends to occur with extreme low-bit quantization, PrismML performs optimization by combining an advanced quantization structure and post-hoc alignment techniques.
Evolution from the First Generation and 98.2% Accuracy Retention Rate Based on Qwen 3.8
The first-generation Bonsai series (with the 8B version released in April 2026 and the 27B version in July), based on models like Qwen3.6-27B, generated a major buzz for making it possible to run 27B-class models on smartphones and laptops. However, the initial models still had practical challenges, such as occasional thinking loops and tool-calling failures during code generation and complex multi-step instructions.
The newly introduced “Ternary Bonsai 2 27B” has achieved a dramatic evolution by updating its base model to the latest “Qwen3.8-27B” and through repeated algorithmic adjustments. According to published data from PrismML, it recorded an astonishing accuracy retention rate of 98.2% compared to the pre-quantization original model (Qwen3.8-27B) across major knowledge and reasoning benchmarks. Because the base model, Qwen3.8-27B, itself possesses frontier-class language and logical reasoning capabilities, it fits into a size of just 6.7GB while retaining the vast majority of that intelligence. In addition, it inherits its vast context window of 262k (262,144 tokens) intact, giving it the capability to handle the analysis of lengthy documents and the reading of entire large-scale repository codebases in a local environment. The graph below shows a comparison of accuracy retention rates.

Improved execution performance and agent capabilities in local environments
System Requirements and Specs for Smooth Operation on 12GB VRAM Class PCs
The greatest appeal of Ternary Bonsai 2 27B lies in its ability to run comfortably on realistic, consumer-grade hardware environments. Even with standard 4-bit quantization (Q4), running a 27B model requires roughly 15GB or more of VRAM, demanding high-end GPUs with 24GB or more (such as the RTX 4090 or 5090). However, with Ternary Bonsai 2 27B kept to around 6.7GB, it can be run locally with plenty of headroom on mid-range gaming PCs equipped with 12GB of VRAM (such as the RTX 4070 or RTX 5070) or on a MacBook Pro (Apple Silicon environment).
For execution software, it supports standard dedicated forks of llama.cpp and Apple’s MLX framework, and is distributed in the GGUF format. Below is a summary of estimated actual throughput in major operating environments.
– NVIDIA RTX 5090 (24GB VRAM): Up to 143 tokens per second
– Apple M5 Max-powered Mac: approx. 58 tokens per second
– NVIDIA RTX 5070 Ti (16GB VRAM): approx. 30–45 tokens per second
– iPhone 17 Pro Max (using 1-bit quantized version): approx. 11 tokens per second
Because you can achieve response speeds on your local PC that far exceed the speed at which humans read text (around 15 to 20 tokens per second), you will rarely feel any response lag in actual use.
Practicality as an AI Agent and Complex Task Execution Capabilities
Until now, ultra-low-bit local models, while capable of handling simple text summarization or one-off Q&A, have been considered unsuitable for AI agent tasks that involve sequentially calling multiple tools. However, Ternary Bonsai 2 27B demonstrates high adaptability in autonomous development and operation tasks when combined with agent harnesses (such as OpenHands and ZooCode).
As a practical verification example, results have been reported in which, for a task to build a password management server including a Python backend, Vue.js frontend, and database integration, simply giving instructions automatically completes the entire process from overall planning and environment setup to code generation and browser-based operational testing. It possesses the tenacity to see complex tasks through to the end—such as reading error logs on its own and making corrections without falling into incorrect tool calls midway. More than the numerical accuracy retention rate, the true value of this model lies in the fact that it has reached a practically usable level in the field as an autonomous development assistant or pair programming partner. The autonomous processing workflow configuration is as follows.

The Societal Impact and Future Outlook of Edge AI
Confidential Data Protection and Solutions to the AI Energy Crisis
The widespread adoption of Ternary Bonsai 2 27B holds profound significance for corporate IT infrastructure strategies and the resolution of ESG (Environmental, Social, and Governance) challenges. Currently, many companies face a security barrier in leveraging generative AI, as they are unable to send confidential data to external clouds. In fields requiring advanced security—such as electronic medical record analysis in healthcare institutions, internal document review in financial institutions, and mold and blueprint data processing in manufacturing—on-device AI, which can complete all data processing entirely within the device, serves as an extremely effective solution.
Furthermore, it will have a powerful impact on the globally worsening AI energy crisis. As of 2026, the electricity consumed by data centers worldwide is approaching 1,050 TWh annually, representing a massive power load equivalent to the national consumption of the world’s 5th-largest country. Slashing model weights to one-ninth and shifting to lightweight inference that eliminates multiplication will dramatically reduce data center power consumption, directly linking to green AI strategies that enable companies to promote AI adoption while curbing greenhouse gas emissions.
Future Outlook and the Future Opened Up by On-Device AI
Going forward, highly efficient quantization technologies such as Ternary Bonsai 2 are expected to further evolve, accelerating their implementation in mobile devices and various industrial edge devices. It has already been reported that Apple is evaluating PrismML’s 1-bit/ternary quantization technology for potential integration into the iPhone, bringing us closer to a realistic future where advanced intelligence comes built into next-generation smartphone and PC operating systems as a standard feature.
Furthermore, as the development of dedicated NPUs (Neural Processing Units) and semiconductor chips specialized for 1-bit and ternary operations progresses, processing speed and energy efficiency are expected to jump several times higher. We are currently transitioning to an era where advanced AI processing, which has traditionally relied on large-scale models in the cloud, operates autonomously 24/7 on everyday pocket-sized devices and factory control terminals (edge devices). Ternary Bonsai 2 27B will likely serve as a symbolic model that rewrites the conventional wisdom that AI usage requires the cloud, paving the way for a new future where on-device AI becomes established as social infrastructure.
Reference Page
- [PrismML Official Model Repository] https://huggingface.co/prism-ml
- 【PrismML Bonsai-demo Repository】https://github.com/PrismML-Eng/Bonsai-demo
【#TernaryLLM #TernaryBonsai2 #LocalLLM #EdgeAI #PrismML #Qwen38 #GenerativeAI #EnergyEfficientAI】


コメント