[Explanation] Local LLM Runtime Environment “Strata” Now Supports Mac

IT

The latest version (version 0.1.44) of the open-source local LLM execution environment “Strata” was released on October 13, 2026, adding experimental support for Apple Silicon-powered Macs. This release comes amid a rapidly growing demand for local AI operations aimed at protecting internal data and personal information, and reducing cloud communication latency.

Overview of the Strata 0.1.44 Release and Mac Support

The latest version “0.1.44” of Strata, an open-source local LLM (Large Language Model) runtime environment, was officially released on October 13, 2026. The biggest highlight of this update is the addition of highly requested experimental support for Apple Silicon-equipped Mac models.

Strata is an open-source inference engine designed to efficiently run and execute large language models on your own local PC. It has previously supported graphics cards (GPUs) from major manufacturers like NVIDIA, AMD, and Intel, and has featured local server capabilities compatible with the OpenAI and Anthropic APIs. With the newly added Mac support, high-speed inference leveraging Apple’s proprietary graphics API, Metal, is now possible on Apple Silicon-equipped devices ranging from the M1 to the M5.

In recent years, the execution infrastructure for AI models has been rapidly shifting from cloud to local environments. Amidst this trend, Macs—featuring a unified memory architecture where the CPU and GPU share memory—have been drawing significant attention as hardware capable of holding large-scale models at a low cost. With Strata officially incorporating support for the Mac environment, an environment is beginning to take shape where individual developers and researchers can more comfortably build and run local LLMs on Mac devices. As shown in the figure below, the evolution of local execution environments has been remarkable.

Figure 1

Verification figures for M5 Max and comparison with llama.cpp

According to performance testing conducted by the Strata developers, extremely high inference efficiency was demonstrated in an environment featuring the top-tier M5 Max (equipped with 128GB of unified memory).

In the verification, performance was compared with llama.cpp, an open-source standard inference engine, using the exact same GGUF-format model file. As a result, when MTP (Multi-Token Prediction) technology was enabled in Strata, it achieved a high performance of approximately 1.6 to 1.9 times the token generation speed of llama.cpp. Furthermore, an acceleration of approximately 1.5 to 1.6 times was confirmed in prompt processing speed when feeding long input sentences to the model.

The major factor behind this drastic speed improvement lies in the thorough internal optimization for Apple Silicon’s Metal framework and the efficient parallel processing architecture. Conventionally, llama.cpp has been considered the definitive choice for running GGUF files on Macs, but the figures presented by Strata have a significant impact as a new option for inference engines in the Mac environment. By drastically reducing processing wait times, convenience in practical tasks such as long-text summarization and code generation is expected to be dramatically improved.

Optimization for Diverse GPU Architectures and Quantization Formats

Expansion of the NVFP4 Format and Support for the Latest Graphics Boards

In Strata 0.1.44, significant progress has been made not only in supporting Mac environments, but also in expanding quantization technologies for Windows and Linux environments equipped with PC graphics cards.

Of particular note is the enhanced support for the “NVFP4 format,” which leverages the FP4 Tensor cores equipped in NVIDIA’s latest architecture, the GeForce RTX 50 series. Furthermore, the scope of support has also expanded for 4-bit to 8-bit quantized models released by Unsloth, which enjoys widespread support in the open-source community. Quantization technology is an indispensable technique for significantly reducing VRAM (video memory) consumption by appropriately compressing the parameter precision of AI models, making it possible to run large models on limited hardware resources.

This update enables ultra-lightweight and high-speed inference by fully utilizing the latest hardware features, providing optimal inference performance for users who own the latest generation of GPUs. The following diagram illustrates the execution process of the multi-layered quantized model.

Figure 2

As both hardware and quantization formats evolve in this way, an environment is becoming a reality where models of a scale that previously could only be run on massive enterprise servers can operate at practical speeds even on desktop PCs and high-performance laptops.

Inference Performance Improvements on AMD and Intel GPUs and Support for New Models

The Strata development team continuously performs well-balanced optimizations for GPUs from a wide range of manufacturers without favoring any specific hardware. Version 0.1.44 also simultaneously announced enhancements to processing capabilities for AMD and Intel GPUs.

In terms of supported models, new open models such as “Qwen3.6-35B-A3B” are now supported. According to measurements by developers, when using AMD’s latest-generation GPU, the Radeon RX 9700, enabling the MTP function achieved an astonishing text generation speed of approximately 161 tokens per second. Parallel processing has also been optimized for multi-GPU configurations centering on AMD’s RDNA 4 generation GPUs, improving the efficiency of distributed inference across multiple graphics boards.

Additionally, optimization efforts for the Intel Arc A-series are also yielding results. For example, in testing with the mid-range Intel Arc A750, the optimization process improved token generation speed by approximately 34%, increasing it from the previous 11.8 tokens/second to 15.9 tokens/second. This significantly expands the possibilities for running local LLMs at practical response speeds, not only on high-end GPUs but also on mainstream hardware.

Implementation Challenges and Key Areas to Watch in the Local LLM Market

Experimental Implementations and Known Issues at Present

Although the Mac support introduced in Strata 0.1.44 is a remarkably groundbreaking advancement, the development team emphasizes that the current implementation is strictly positioned as “experimental” at this stage.

In actual operation, several functional constraints and known issues exist. First, limitations are imposed on supported model architectures and the number of concurrent parallel requests that can be processed, so careful consideration is required when using the system in production environments, such as for large-scale simultaneous access in server applications. Additionally, there have been confirmed reports of issues where responses from the model cannot be obtained properly on certain Mac models or under specific OS settings.

The development team is aware of these issues and plans to release “0.1.44.1,” a minor patch addressing the bugs, in the near future. On the user side as well, when running the software on Macs with limited memory capacity such as 16GB or 32GB, practicing resource management unique to a local environment—such as temporarily closing browsers and other heavy applications to reduce memory pressure—is key to stable operation.

Expansion of the Local LLM Market and Future Outlook

In recent years, the market for local LLMs, which run AI entirely within a company’s own environment or personal devices, has seen rapid global expansion. Behind this trend are the rising costs associated with cloud services, communication latency, and, above all, growing security awareness regarding the risk of leaking corporate secrets and personal information.

As of 2026, running AI locally has moved beyond a mere hobby or verification exercise, and is becoming a strategic choice for achieving business compliance and zero-cost operations. Amid these market trends, Strata’s official support for Mac will further accelerate the movement to leverage Apple Silicon’s massive unified memory bandwidth for local AI infrastructure.

Moving forward, in addition to bug fixes and stability improvements, expectations are high for expanded support for multimodal (image and audio recognition) models and further maturation of distributed inference technologies. As open-source development of inference engines becomes more active, users will be able to flexibly select the optimal execution platform to suit their own hardware environments and use cases, which is expected to further expand the potential of local AI.

Reference page

  • 【Release Strata 0.1.44 · Niko1221/Strata】https://github.com/Niko1221/Strata/releases/tag/v0.1.44

【#Strata #LocalLLM #AppleSilicon #Mac #InferenceEngine #GenerativeAI #OpenSource #Metal】

コメント

Copied title and URL