“Hayamimi,” a real-time multilingual speech recognition system that runs at high speeds using only a CPU, was released as open-source software in October 2026. It achieves low-latency transcription with less than 2GB of memory without relying on GPUs or cloud APIs, and is drawing attention as a technology that opens up new possibilities for AI utilization in local environments.

Key Points of the Discovery and Announcement: Achieving Astounding Ultra-High-Speed and High-Precision Processing with a CPU Alone

In October 2026, developer oboroge0 released “Hayamimi,” a lightweight, real-time multilingual speech recognition system that operates solely on a CPU without using any GPUs or external cloud APIs, on GitHub. Conventional CPU-based speech processing systems have many limitations in terms of processing speed and accuracy, and typically presuppose the use of expensive graphics cards or cloud services. However, “Hayamimi” achieves astonishing response performance, outputting finalized text with an extremely short latency of only about 100ms (0.1 seconds) from the end of speech in standard desktop or laptop CPU environments. Furthermore, it is equipped with a function to update and display provisional subtitles (draft text) in real time approximately every 0.5 seconds even while speech is ongoing, providing a natural conversational experience that does not cause user frustration. In recognition tests using Japanese audio from actual television broadcasts, it recorded a remarkably high recognition accuracy with a Character Error Rate (CER) of 3.8%. From the perspective of processing efficiency, it is capable of high-speed processing at 10 to 50 times real-time speed (RTF 0.07–0.10) in a 6-core desktop CPU environment, and the memory required for operation is kept under 2GB (a minimum of about 1.35GB). The following diagram illustrates the basic configuration and processing flow of “Hayamimi.”

Operating Mechanism: INT8 Quantized Models and Optimal Routing

The reason “Hayamimi” is able to achieve such fast and high-precision processing with just a CPU lies in its sophisticated model quantization technology and proprietary multilingual routing method. While conventional large-scale speech recognition models tend to process all languages at once using a single huge general-purpose model, “Hayamimi” adopts a design that automatically determines the language for each utterance and instantly routes the processing to a dedicated, specialized model optimized for each language. Specifically, it features a hierarchical structure where dedicated, advanced models are deployed for five major languages (Japanese, Chinese, Korean, Cantonese, and English) as well as 24 European languages, while falling back to the Omnilingual ASR developed by Meta for approximately 1,600 other languages. The models used are in ONNX format with INT8 quantization, which significantly compresses data capacity while maintaining accuracy, and they run on sherpa-onnx, a lightweight inference execution engine. This achieves a structure that operates lightly and smoothly with just a standard Python environment and ffmpeg, without the need to install heavy deep learning environments like PyTorch or CUDA. Furthermore, a memory upper-limit management function (LRU eviction) constantly controls the memory usage of resident models, enabling stable operation even on older-generation PC hardware.

Overcoming Conventional Challenges and a Multifunctional Practical Ecosystem

Comparison with Existing Technology and Cloud Dependency: The Driving Force for Privacy Protection and Zero Cost

The biggest obstacles in implementing conventional voice AI have been the privacy risks associated with sending confidential data externally and the high operating costs due to cloud API usage-based pricing. In applications such as electronic medical record input in medical institutions, customer service in financial institutions, and internal corporate meetings containing insider information, security concerns regarding sending voice data to external servers were extremely strong, serving as a major barrier to AI adoption. Furthermore, when continuously using cloud speech recognition APIs for large-scale call centers or daily live streams, it was not uncommon for API usage costs to range from several hundred thousand to several million yen per seat annually. Because “Hayamimi” achieves fully local operation without requiring an internet connection, it fundamentally eliminates the risk of confidential information leaking outside the device and guarantees complete security. Moreover, once the environment is set up, zero-cost operation is possible with no third-party API usage fees incurred whatsoever. Since its CPU resource consumption is also extremely low, it can add high-quality real-time subtitles even during live streaming such as game commentaries (OBS) without adversely affecting GPU rendering performance.

Multi-faceted Extensibility Supporting Practical Features: From Two-Pass Correction to OBS Integration

“Hayamimi” is not merely a transcription engine, but comprehensively equips a variety of practical functions so that it can be immediately utilized in actual business and streaming environments. As an approach to maximize recognition accuracy to the utmost limit, it is equipped with a “two-pass correction feature.” After outputting real-time breaking captions, when it detects about 2 seconds of silence, it automatically re-decodes the immediately preceding utterance to generate a polished transcript. Through this correction processing, an improvement in Japanese accuracy from an initial CER of 15.5% to 12.0% has been confirmed. In addition, it supports real-time speaker identification using the CAM++ model (--speakers) and precise speaker diarization during polishing processing using pyannote/segmentation-3.0. Furthermore, it provides a real-time translation feature (--translate) that instantly converts Japanese utterances into English, Chinese, Korean, and Spanish, a hotword feature to increase the recognition rate of proper nouns, and a post-replacement dictionary registration feature. In terms of system integration, by launching a local HTTP server (--serve), users can display a real-time dashboard (port 8833) or a browser source overlay for OBS from a browser, and it also supports a speaker loopback feature (--input speaker) that captures not only microphone input but also internal PC speaker audio. The following figure shows an image of subtitle display on the dashboard and streaming screen.

Current Limitations, Future Prospects, and Impact on Society

Current Technical Challenges: Limitations in Mixed-Language Environments and Specific Settings

Although “Hayamimi” boasts exceptionally high performance, evaluations by the development team and users have also highlighted several technical limitations and challenges. The most notable limitation is that language identification fails to function correctly during “code-switching” (mixed-language speech) where Japanese and English are intricately mixed within a single sentence, making misrecognitions prone to occur. Additionally, erroneous language identification may occur in extremely short utterances immediately following sound effects or background music (BGM), and room for improvement remains in the separation accuracy when two speakers talk completely simultaneously with overlapping voices. Regarding the real-time translation feature, because the built-in translation model is an ultra-lightweight compact model, it has been pointed out that there is a risk of errors in converting numbers and monetary amounts depending on the context, and caution is advised when using it alone in strict situations such as financial or contract negotiations. Furthermore, the opt-in 4-class punctuation model (--punct-model 4class) is designed not to output exclamation marks, requiring settings to be optimized according to the specific use case.

Future Outlook and Key Points: The Dawn of the Edge AI Era and the Wave of Practical Application

The emergence of “Hayamimi” can be said to symbolize the current situation in 2026 where the trend of AI technology is shifting significantly from “cloud-centric” to “device-local processing (edge AI).” Up until now, AI voice input has achieved high accuracy through large-scale servers on the cloud, but its adoption in the field has faced constraints due to issues such as network latency, security, and running costs. The empirical results demonstrated by Hayamimi—namely, “CPU-only, less than 2GB of memory, 100ms latency, and a CER of 3.8%”—have proven that practical voice interaction systems can be built using only existing PC assets without the need for special hardware investments. Moving forward, it is expected to be applied in a wide range of industrial fields, such as automated corporate minutes, local input assistance in medical and legal settings, real-time subtitle communication in multinational teams, and deployment in embedded devices and smart devices. With ongoing model feature expansion and optimization by the open-source community, it is seen as certain that the adoption of next-generation computing using the human “voice” as an interface will accelerate.

Reference page

• 【oboroge0/hayamimi: Hayamimi – Real-time multilingual speech-to-text on CPU only】[suspicious link removed]

[#SpeechRecognition #EdgeAI #LocalAI #EarlyBird #OpenSource #Transcription]