On June 24, 2026, Fujitsu Limited announced the development of a new architecture called “PHOTON,” which dramatically improves the inference efficiency of large language models (LLMs). This technology increases processing performance per GPU by up to 475 times compared to the traditional mainstream Transformers, marking a major turning point toward reducing operational costs and adopting generative AI.

Outstanding processing performance and oral adoption at international conferences

On June 24, 2026, a research team from Fujitsu, RIKEN, Tokyo University of Science (formerly Tokyo Institute of Technology), and Tokai University announced the development of a new architecture called “PHOTON (Parallel Hierarchical Operation for TOp-down Networks),” which significantly reduces the operational costs of large language models. The greatest feature of this technology is that, compared to the current standard LLM structure, Transformer, it increases the number of output tokens (processing throughput) per GPU by up to 475 times.

The results of this research have been selected for the oral session of the world’s leading international conference in natural language processing, “The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026),” and details are scheduled to be presented at the conference held in San Diego, USA, starting July 2, 2026. The significance of a Japanese research institution leading the proposal of an architecture that can compete globally through the practical and important perspective of inference efficiency is extremely significant. Please refer to the diagram below.

Why is “inference efficiency” the top business issue right now?

In recent years, as the practical application of generative AI has rapidly advanced, methods to improve performance by making inference occur longer and more ‘thinking’ have been gaining attention. However, current mainstream Transformer-based models face serious challenges such as an explosive increase in memory access—called the “KV cache”—which holds past interactions—when input documents become longer or when many users use them simultaneously, significantly slowing processing speed.

This phenomenon is due to the fact that memory bandwidth tends to become a bottleneck over computing power in modern GPUs, leading to wasted computing resources and increased power costs. Fujitsu aims to achieve economic and environmental sustainability for the massive AI demand by enabling advanced processes that require multiple inputs and outputs in parallel, such as multi-agents, to be executed at low cost and efficiently. For companies to fully utilize generative AI in their operations, it is essential not to rely on a “brute” strategy of stocking large quantities of high-performance GPUs, but to improve efficiency at the structural level.

Technical breakthrough: Hierarchical models for building an information pyramid

Transition from “horizontal” scanning to “vertical” hierarchical processing

Traditional Transformers performed “horizontal” scanning, breaking down text into fragments of just a few characters called “tokens” and sequentially calculating relationships with all past tokens one by one. In contrast, PHOTON adopts a completely new approach called “vertical and multi-resolution scanning,” which treats text as a “collection of meanings” and processes it hierarchically.

The structure of PHOTON consists of two layers: a “bottom-up encoder” that compresses information in multiple stages, and a “top-down decoder” that generates concrete words from concepts. First, the input token sequence is divided into certain chunks, then summarized and gradually compressed into abstract latent representations, constructing an “information pyramid.” During generation, it predicts abstract concepts like “what will come next” from the coarsest level of context, passes it to lower layers, and generates concrete tokens in parallel. This idea is very similar to how humans first consider the general structure when writing and then fill in the details.

Innovative memory management to stop the bloated KV cache

The biggest reason PHOTON can achieve such overwhelming speeds is the dramatic reduction in memory usage. In traditional models, the longer the context, the more KV cache grows linearly, but with PHOTON, each level of processing is limited to chunks, allowing cache size to remain constant regardless of context length.

Additionally, the computational overhead of “re-encoding generated tokens and returning them to higher layers,” which was a challenge in traditional hierarchical models, is addressed by PHOTON through a unique mechanism called “Recursive Generation.” This is a clever combination of three different loss functions during the learning phase (subtoken loss, recursive reconstruction loss, and next-context loss) to maintain contextual integrity even by completely omitting bottom-up redoing during inference. This design successfully eliminates memory bandwidth bottlenecks and maximizes the GPU’s computing power. Please refer to the mechanism diagram below.

Verifying Quality and Efficiency: Overwhelming Superiority Demonstrated by Experimental Data

Approximately 475 times the computing power achieved with the 1.2B model

In numerical experiments conducted by Fujitsu, performance was verified across three model sizes: 600 million (600M), 900 million (900M), and 1.2 billion (1.2B) parameters. As a result, it was confirmed that it can achieve higher generation throughput while reducing memory usage compared to conventional Transformers across all sizes. Especially for models with 1.2 billion parameters, it achieved an astonishing multi-query processing capability of about 475 times that of conventional Transformers in exchange for slight performance degradation.

Additionally, lead author Yuma Ichikawa has noted that there have been cases where throughput per memory (TPM: Throughput-per-Memory) reached up to 1000 times, demonstrating that this advantage is especially evident under harsh conditions close to actual LLM operations, such as long contexts and multiques. This is the result of PHOTON transforming the “structure itself,” whereas previous optimization technologies like FlashAttention maintained a “flat structure.”

High answer accuracy and stability through multi-query integration technology

Another key pillar of PHOTON is its “multi-query integration technology,” which efficiently bundles large volumes of output. Because PHOTON requires an extremely small KV cache per generation, it is possible to obtain multiple generation results in parallel within the same GPU memory budget. Leveraging this feature, it generates multiple slightly different candidate candidates for the same problem, enabling fast execution of majority voting or the process of selecting the best candidate (ensemble).

Verification data shows that simply integrating nine queries can achieve performance comparable to traditional Transformers. In other words, the concern that “speed comes at the expense of quality” is overcome through volume strategies that leverage overwhelming computational efficiency. In fact, integrating multiple answers offers the advantage of consistently achieving higher correct answer rates than a single reasoning. In the trade-off between speed and quality, PHOTON has successfully pushed the traditional Pareto frontier far forward.

Fujitsu’s Management and AI Strategy: Transformation from SIer to a High-Profit DX Company

Integration and ripple effects into the AI platform “Fujitsu Kozuchi”

Fujitsu positions PHOTON as the core technology of its AI service platform, “Fujitsu Kozuchi.” The company has continuously developed proprietary technologies related to AI lightweighting and power saving, such as the world-class “1-bit quantization technology” that maintains the highest accuracy rate, and “specialized AI distillation technology” that achieves accuracy surpassing the original model. When combined with these technologies, PHOTON will serve as a driving force to further enhance Fujitsu’s proprietary LLM, Takane.

Specifically, we aim to enable large models that previously required multiple high-end GPUs to be executed at high speed on edge devices such as a single low-end GPU, smartphones, and factory machinery. This makes the use of AI agents in real-time response environments more realistic. Fujitsu’s strategy is not just to provide a general-purpose model, but to provide a full-stack, lightweight, and highly efficient environment tailored to customers in the “sovereign AI” domain, which demands advanced security and expertise such as finance, manufacturing, healthcare, and defense.

Technical expertise and market recognition that break away from past ‘labels’

From an investor’s perspective, Fujitsu has long been labeled an “old SIer,” with its stock price undervalued compared to competing IT companies in the US and domestically. However, developing structurally innovative technologies like PHOTON in-house and being recognized at top conferences like ACL suggests that the company has fully shifted from pure coding subcontracting to high value-added consulting and DX firms responsible for overall optimization (such as Fujitsu Navance).

Additionally, Fujitsu has established comprehensive strategic partnerships with global big tech giants such as NVIDIA, Google, Microsoft, and Anthropic. Rather than directly confronting them, it serves as a “bridge of trust” for major Japanese corporations and government agencies when adopting AI, while incorporating their proprietary technologies (MONAKA processors, optical transmission systems, and PHOTON architecture) to solidify their position as the “provider” of the AI ecosystem. Such a business transformation backed by technical backing signals a dramatic improvement in profitability over the medium to long term.

Future Developments and Focus: Will Japanese-Originated Architecture Change the World?

Spread of Edge AI and Realization of Autonomous Multi-AI Agents

With the success of PHOTON’s development, the most anticipated future is the explosive spread of AI utilization in edge devices. According to Fujitsu’s roadmap, from fiscal 2025 to 2026, the sophistication of the “multi-AI agent” that evolves to adapt to changes in work and external environments is planned. PHOTON’s lightweight and high-speed reasoning capabilities will accelerate the evolution of personal AI assistants running on smartphones and “spatial robotics” that control complex movements in real time.

Furthermore, the next-generation AI-HPC platform technology developed this time is expected to be utilized in the supercomputer “Fugaku NEXT,” for which Fujitsu has been commissioned for basic design. This AI supercomputer platform, aiming to be operational around 2030, will serve as the infrastructure supporting Japan’s scientific and technological innovation. PHOTON is expected to work closely with hardware technologies such as the next-generation processor “FUJITSU-MONAKA,” leveraging the strengths of Made in Japan as a vertically integrated AI platform, not limited to a single software technology.

Validating scaling and challenging global standards

The future focus is on whether PHOTON’s design will perform similarly well in the currently mainstream ultra-large models with billions to hundreds of billions of parameters—a “scaling verification.” Current experiments cover up to 1.2B parameters, but the compression effect from layering can benefit even greater as the model size increases or the context handles longer.

Fujitsu openly presents these achievements, demonstrating its commitment to refining its technology in collaboration with the global research community. We will keep an eye on how Japanese-originated architectures will be standardized as strong candidates for the “post-Transformer” used by developers worldwide, or whether they will secure decisive market share in specific industrial applications such as financial and manufacturing edge applications. The emergence of PHOTON, which taps into the critical points of AI society such as operational costs and environmental impact, holds the potential to accelerate the realization of a world where generative AI benefits can be enjoyed everywhere, within just a few years.

[#富士通 #LLM #PHOTON #生成AI #DX戦略 #ACL2026 #半導体 #Kozuchi]