[Explanation] Japanese LLM “LLM-jp-4.1” Series Released

IT

The Large Language Model Research and Development Center at the National Institute of Informatics has released the “LLM-jp-4.1” series of open large language models with superior Japanese language performance. This series significantly improves upon the post-training of the previous generation model, achieving enhanced performance on STEM tasks and initial support for tool-calling functions.

An Overview of the New-Generation Homegrown LLM Released by the National Institute of Informatics

The Large Language Model Center (LLMC) at the National Institute of Informatics (NII) publicly released the open large language model “LLM-jp-4.1” series on September 28, 2026. This model is a revised version that fundamentally restructures the post-training (fine-tuning) process of the “LLM-jp-4” series released since April 2026, improving practicality and processing accuracy. The lineup consists of three models that can be chosen according to the use case: the dense-type “llm-jp-4.1-8b-thinking” with approximately 8.6 billion parameters, the MoE-type “llm-jp-4.1-32b-a3b-thinking” which maintains approximately 32 billion parameters while dynamically operating only about 3.8 billion parameters during inference, and the dense-type “llm-jp-4.1-33b-thinking” with approximately 33.2 billion parameters that leverages all parameters to deliver high expressiveness. All models support an expansive context length of 65,536 tokens and are released under the commercially viable Apache License 2.0. Not only the model weights, but also the instruction datasets and preference datasets used for training are made publicly available as much as possible, ensuring the transparency of open AI research. Please refer to the architecture diagram below.

Figure 1

3 major improvements: Enhanced STEM capabilities, tool calling, and more concise answers

The biggest evolution in LLM-jp-4.1 lies in the significant improvement in both the quality and quantity of post-training data. There are three main areas of specific improvement. The first is the enhancement of performance in the STEM (Science, Technology, Engineering, and Mathematics) fields. By expanding the STEM-related data used in post-training to approximately 6 times and the mathematics-related data to about 15 times compared to previous versions, the model’s ability to answer science and mathematics questions has been improved. The second is the support for the “Tool Calling” feature, which is implemented for the first time in the LLM-jp series. A structure that automatically calls external programs and APIs has been incorporated, enabling the construction of complex agent systems that go beyond mere conversational responses. The third is the adjustment of response detail and redundancy. To suppress the tendency observed in the previous generation model to “provide long answers including unrequested information,” instructions prompting concise answers were applied during SFT data creation. As a result, when the inference setting is low or medium, the number of tokens in the final response is significantly reduced compared to conventional models, achieving a level of usability where only the necessary key points can be efficiently obtained.

Technical Background and Detailed Performance Evaluation

Post-Training Pipeline and Large-Scale Data Training Process

To advance post-training, a two-stage pipeline consisting of SFT (Supervised Fine-Tuning) and DPO (Direct Preference Optimization) was introduced. In the SFT process, approximately 12 billion tokens of high-quality data were trained over 3 epochs, bringing the total training volume to about 35 billion tokens—roughly three times that of the conventional model. The training process was also optimized, achieving about a 2.7x speedup in training for the 8B model through optimized conversation-level attention calculations. As the computing infrastructure, up to 16 nodes equipped with eight NVIDIA H200 SXM5 (141GB HBM3e) units each were used, completing the training of the 8B model in approximately 17 hours and the 33B model in about 67 hours. In the DPO process, multiple response candidates generated by the SFT-tuned model were scored and filtered using an evaluation model (gpt-oss-120b), creating over 300,000 high-quality preference pairs for the 8B model to drive optimization. Please refer to the training flowchart below.

Figure 2

High Processing Capability and Challenges Proven in Benchmarks

On “swallow-evaluation-instruct,” a comprehensive evaluation tool comprising high-difficulty tasks, performance improvements were confirmed across all model sizes in the 29-metric average. The 8B model increased significantly from 0.53 to 0.64, the 32B-A3B model from 0.58 to 0.67, and the top-tier 33B model saw a large score jump from 0.67 to 0.75, achieving a level comparable to the overseas competing model gpt-oss-120b. Additionally, on “AnswerCarefully,” a metric evaluating Japanese safety, the 33B model improved from 3.79 to 4.08, demonstrating an appropriate ability to respond to harmful questions. In multi-turn evaluations on the Japanese version of the tool-calling benchmark “BFCL-nejumi,” the 33B model also achieved results surpassing Qwen3-32B and Qwen3.5-9B. On the other hand, phenomena involving confusion with general knowledge due to the side effects of filters that correct self-awareness have also been confirmed, and continuous improvements are underway for the next version.

Use Cases and Future Outlook

Model Selection Guidelines and Deployment Methods in Various Execution Environments

When implementing the model, it is recommended to select a model based on system requirements and the intended purpose. When performing preprocessing in resource-constrained environments such as laptops or single GPUs, the 8B model, which runs on approximately 5.5GB of memory using the 4-bit quantized version (Q4_K_M), is suitable. For service operations seeking high throughput and cost-effectiveness, the MoE-structured 32B-A3B model is ideal, whereas the 33B model is the best choice for agent use cases involving advanced reasoning and multi-step tool calls. Officially supported execution frameworks include Transformers 5.0 or higher, vLLM 0.30.0 or higher (used alongside the dedicated parser llm-jp-vllm), or llama.cpp v0.6.0 or higher for running GGUF quantized versions. Since it is governed by the Apache License 2.0, businesses are free to integrate it into their in-house systems and develop commercial services.

Future Developments, Key Points, and Expectations for the OpenAI Ecosystem

As an environment for researchers and developers, trial provision of APIs and chat services via the cloud infrastructure “mdx-MaaS,” operated by the University of Tokyo, is scheduled to begin in phases from mid-October 2026. In addition, the development team has already started working on the next-generation model, “LLM-jp-4.2.” With LLM-jp-4.2, the team aims to achieve a further leap forward in autonomous AI agent capabilities and advanced mathematical and logical reasoning skills by introducing full-scale reinforcement learning (RL) and further expanding the training corpus. Furthermore, within fiscal 2026, the release of “LLM-jp-4 332B-A31B,” an ultra-large-scale MoE model boasting a total parameter count of approximately 332 billion, is also planned. LLM-jp’s commitment to maintaining a true open stance—making all training data, source code, and model weights publicly available—will continue to attract significant attention for establishing Japan’s sovereign AI infrastructure and advancing generative AI technology.

Reference page

  • [National Institute of Informatics LLMC Official Announcement] https://llmc.nii.ac.jp/topics/llm-jp-4-1/
  • [LLM-jp Official Website Release Announcement] https://llm-jp.nii.ac.jp/news/llm-jp-4-1/
  • [Hugging Face llm-jp-4.1 Repository] https://huggingface.co/llm-jp/llm-jp-4.1-33b-thinking
  • 【GitHub llm-jp-4-cookbook】https://github.com/llm-jp/llm-jp-4-cookbook

【#LLMjp #DomesticLLM #NationalInstituteOfInformatics #GenerativeAI #OpenSource #ToolCalling #ArtificialIntelligence】

コメント

Copied title and URL