National research institutes and government agencies are advancing the development of evaluation systems and foundational tools to comprehensively verify and evaluate the safety and reliability of generative AI. Behind this is the emergence of various risks accompanying the rapid adoption of generative AI, such as the spread of misinformation, misuse in cyberattacks, and the limitations of conventional evaluation methods.

Development and Open-Sourcing of Evaluation Tools by AISI and NICT

Japanese government agencies and research institutions are rapidly developing evaluation infrastructure to address the risks of generative AI. With the cooperation of relevant ministries and agencies such as the Cabinet Office, the Ministry of Economy, Trade and Industry, and the Ministry of Internal Affairs and Communications, the AI Safety Institute, established within the Information-technology Promotion Agency, has released evaluation tools for conducting safety assessments of AI systems as open-source software. This tool features highly versatile evaluation items and is provided under the Apache 2.0 license, creating an environment where businesses can easily conduct safety evaluations by reducing the effort required to configure evaluation items and build environments. Additionally, businesses can customize and utilize this tool to suit their own organizational requirements. Furthermore, the Ministry of Internal Affairs and Communications and the National Institute of Information and Communications Technology have begun developing a reliability evaluation infrastructure system aimed at border measures upon the introduction of external AI and regular inspections of models distributed domestically, with the goal of trial use and the provision of prototype models during fiscal 2026. As shown in the figure below, the evaluation tool has a structure that consistently and automatically inspects everything from input to output.

.

How the Evaluation System Works and Automated Red Teaming Features

The safety evaluation system developed by the government adopts a mechanism that automatically inputs a variety of evaluation prompts into the generative AI model being evaluated, and quantitatively and qualitatively verifies the safety of the output results. For example, in response to requests for harmful information such as “how to manufacture hazardous materials,” it statistically scores whether the model can appropriately output a refusal response or if it leaks harmful information. This evaluation tool is equipped with an automated red teaming function that exploits vulnerabilities in the AI system from an attacker’s perspective. Unlike conventional simple feeding of test data, its major technical feature lies in automatically extracting business-specific requirements from operational documents and the like, generating advanced attack scenarios, and conducting evaluations. A wide range of evaluation perspectives are set, including discriminatory expressions, privacy violations, the presence of content leading to crime, verification of misinformation and bias, and compatibility with Japan’s unique culture and values. By combining automated inspection by AI with periodic human audits, a dual-check structure is established to ensure the fairness and objectivity of the evaluation process.

Background and Technical Challenges Behind the Need for Safety Evaluation Infrastructure

The Expanding Risks of Generative AI and the Asymmetry of Evaluation Problem

With the dramatic evolution of generative AI technology, while convenience in society and industry is increasing, serious risks such as the spread of misinformation, leakage of personal information, malicious use in cyberattacks, and deviation from autonomous control are coming to light. Particularly in advanced AI systems built with techniques like reinforcement learning from human feedback, a structural challenge known as “evaluation asymmetry” has been pointed out, in which it becomes difficult for humans to correctly verify and evaluate complex responses generated by the model. There are concerns that significantly enhanced AI systems may cause “evaluation deception,” where they detect that humans are in the evaluation phase and feign safe behavior only during evaluation, or “reward hacking,” where they exploit humans’ imperfect evaluation criteria to optimize in ways contrary to the original intent. Against this backdrop, conventional methods relying solely on static manual human evaluation are reaching their limits, making it a critically important challenge to sophisticate and automate the evaluation methods themselves and to provide objective evaluation metrics as social infrastructure.

Standardization of Quality Management by NEDO and AIST

Toward the social implementation of safety evaluations, the New Energy and Industrial Technology Development Organization (NEDO) launched a project in fiscal 2025 to promote research, development, and verification concerning the assurance of AI safety. In this project, centering on the National Institute of Advanced Industrial Science and Technology (AIST), a public-private partnership of three entities—including Citadel AI and Coby—is collaborating to establish foundational technologies for the quality evaluation and management of generative AI. As a core achievement, the Multimodal AI Quality Management Guidelines have been published, catering to advanced generative AI that simultaneously handles multiple modalities such as text and images. The guidelines define the ability to correctly recognize and generate semantic correspondences between text and images as a new quality metric, and classify and organize its levels into four tiers. Furthermore, by illustrating utilization levels in specific industrial application scenarios—such as the automated generation of product descriptions in e-commerce and support for infrastructure inspection—the guidelines achieve practical standardization that on-site practitioners can immediately reference.

Applications for Social Implementation and Future Prospects

The Significance of Establishing AI Governance and Practical Application in Enterprises

Evaluation tools and guidelines developed by the government hold significant value not merely as administrative regulations, but as a foundation for businesses to provide safe and reliable AI services. Businesses can introduce the evaluation tools released under the Apache 2.0 license into their internal environments, enabling them to build custom evaluation environments tailored to their company-specific operational requirements and compliance standards. This makes it possible to quantitatively grasp fluctuations in safety standards before and after AI model updates, as well as the improvement effects after implementing safety measures. In line with the direction of the AI Guidelines for Business Providers by the Ministry of Internal Affairs and Communications and the Ministry of Economy, Trade and Industry, as well as the government’s Basic Plan for Artificial Intelligence, companies are required to establish governance frameworks at each stage of development, provision, and utilization. As shown in the figure below, the introduction of appropriate evaluation processes is directly linked to enhancing a company’s social credibility.

.

International AI Regulation Trends and Japan’s Standardization Strategy

Movements surrounding AI safety evaluation are closely linked not only within Japan but also to global regulatory and standardization trends. The creation of AI governance frameworks is accelerating worldwide, such as the EU AI Act—a risk-based comprehensive legal regulation by the European Union—and the formulation of international guidelines through the G7 Hiroshima AI Process. In Japan, while centering on a promotional, soft-law approach that ensures reliability while avoiding the stifling of innovation, international cooperation is being deepened with AI safety institutions in other countries, including the UK and the United States, through the AI Safety Institute. Moving forward, it is expected to play a role as a hub for international information sharing by standardizing developed evaluation tools, evaluation datasets, and domain-specific evaluation protocols. Establishing evaluation infrastructure that balances the speed of technological development with the assurance of safety will be key to enhancing our nation’s industrial competitiveness.

Reference page

• [AI Safety Evaluation Tool OSS Repository (GitHub)] https://github.com/Japan-AISI/aisev
• [AI Safety Institute (AISI) Official Website] https://aisi.go.jp/
• [Regarding the Establishment of the Cabinet Office AI Safety Institute] https://www8.cao.go.jp/cstp/stmain/20240214.html
• 【NEDO Project for Promoting R&D and Verification on Ensuring AI Safety】https://www.nedo.go.jp/activities/ZZJP_100327.html

[#GenerativeAI #AISafety #AISI #AIEvaluationSystem #AIGovernance #NEDO]