Indian SaaS company Zoho has announced a unique AI strategy that avoids blind reliance on large language models, placing the utmost importance on cost efficiency and data protection. This move comes amid a backdrop of soaring AI inference costs for enterprises and rising concerns over the risk of confidential data leakage.

1. Zoho’s Strategy Announcement at Zoholics Japan 2026

At the press conference for “Zoholics Japan 2026” held on September 25, 2026, major SaaS vendor Zoho announced a new promotion strategy for enterprise AI. Ramprakash Ramamoorthy, Head of AI Research at Zoho India, and Hideki Hattori, Executive Officer of Zoho Japan, took the stage to detail how the company balances cost optimization and data protection centered on its proprietary models. In recent years, general-purpose AI models such as ChatGPT and Claude have seen rapid global adoption; however, it was pointed out that their benefits tend to remain limited to individual-level efficiency, leaving challenges in improving organization-wide productivity and expanding returns relative to invested costs. When general-purpose models are adopted wholesale in corporate activities, operational costs—including token usage fees—tend to balloon, easily creating a gap with business value. Given this situation, rather than directly competing with ChatGPT and others, Zoho presented a unique approach of combining small-scale AI models specialized for business use with a multi-agent infrastructure. As shown in the image below, the company aims to build an AI environment optimized for enterprise use, based on a full-stack philosophy that handles everything from applications to infrastructure entirely in-house.

2. Expansion of Inference Costs and Challenges in Token Budget Management

As corporate adoption of AI gains full momentum, the soaring cost of AI inference is emerging as a critical management bottleneck. According to recent research and operational data, inference costs account for approximately 85% of corporate generative AI budgets, with companies running models from multiple providers in production incurring token expenses totaling hundreds of thousands of dollars annually. In particular, factors such as over-prompting—adding complex instructions to every API call—and model over-specification—assigning expensive top-tier models to simple data extraction tasks—are amplifying wasteful costs. While large general-purpose models can handle a broad range of knowledge from work-related consultations to daily conversation, in business settings, their excessive knowledge scope and high unit prices contribute to driving up costs. As a result, companies are being forced to establish strict token budget management and model restriction policies to evaluate the returns against the costs invested. As shown in the conceptual diagram below, right-sizing—optimizing model sizes according to the specific use case—is required.

Solutions Using Small LLMs and Domain-Specific Agents

3. Architecture of Zia LLM and the Multi-Agent Platform Zia Agents

The solution advocated by Zoho is “right-sizing” tailored to business contexts. Its proprietary language model, “Zia LLM,” consists of three scales with 1.3 billion, 2.6 billion, and 7 billion parameters. While lightweight compared to general-purpose models, it delivers high response accuracy for specific tasks. Individual tasks such as document verification in financial workflows, automatic receipt reading, and escalation handling do not require large models; combining smaller-scale models helps keep processing costs down. Furthermore, the multi-agent infrastructure “Zia Agents” and the development environment “Agent Studio,” which complement this, are already used by over 9,000 companies globally, with more than 13,000 agents in operation. In addition to Zoho’s proprietary secure models, user companies can also connect third-party LLMs as needed, making it possible to build automation processes optimized for their own operations while maintaining robust access control and privacy protection.

4. Proprietary Server Nathu La and the Strengths of Full-Stack In-House Development

What supports Zoho’s cost optimization strategy from the physical layer is its philosophy of complete in-house development, which has been upheld since its founding. Rather than relying entirely on third-party public clouds, the company owns and operates its own data centers in more than 20 locations worldwide. In June 2026, the company announced “Nathu La,” a server platform custom-designed from the motherboard to the GPU chassis. Nathu La maintains processing performance equivalent to industry-standard servers while successfully reducing power consumption by 12% to 18% and total cost of ownership (TCO) by 20% to 30%. The power and infrastructure efficiency of its in-house hardware serves as a crucial foundation that is directly linked to lowering AI inference costs. By maintaining a full stack in-house—from data centers and hardware to middleware, applications, and AI models—the company ensures stable service delivery, shielded from the risks of price increases and specification changes by external vendors.

Data Sovereignty, Privacy Protection, and Future Developments

5. Frontier AI Risks and the Importance of In-House Managed Models

In addition to cost, ensuring security and data sovereignty is a critical theme in the adoption of enterprise AI. Sridhar Vembu, Chief Scientist at Zoho, has expressed strong apprehension about readily handing over corporate proprietary know-how, intellectual property, and trade secrets to frontier AI labs. Sending sensitive data to frontier AI models carries the risk that a company’s technical information and processes might be unexpectedly incorporated into training data or leaked to others. In fact, leading companies—particularly in the pharmaceutical industry—are increasingly establishing operations where they build custom AIs based on open-source models in on-premises or on-device environments, keeping confidential information off external servers. Zoho strictly enforces a policy of not using customer data to train its own AI, meeting corporate confidentiality and data governance demands by incorporating access control management and Explainable AI (XAI) mechanisms.

6. Reinvestment in the Japanese Market and Future Outlook

Armed with this robust AI strategy and cost performance, Zoho is accelerating its investment and growth in the Japanese market. Hideki Hattori, Executive Officer of Zoho Japan, emphasized that the Japanese subsidiary, which is celebrating its 25th anniversary, has now entered a “reinvestment phase.” The company plans to double its current workforce of approximately 160 employees over the next one to two years, and is significantly expanding its sales offices in Shinagawa, Tokyo, and Osaka. In the Japanese market, Zoho’s immensely affordable licensing structure compared to competitors, intuitive user interface, and suite of over 60 integrated applications are gaining tremendous support. For Japanese companies struggling with the soaring costs of cloud and AI, Zoho’s pragmatic approach—which is not tied to a single large-scale LLM—serves as a compelling option that balances cost containment with data security. Moving forward, along with the expansion of the application scope for task-specific agents, there are high expectations for the deepening of implementation case studies in the Japanese market.

Reference page

• [Zoho’s Privacy Commitment] https://www.zoho.com/privacy-commitment.html
• 【Enterprise LLM Inference Governance: Token-Level Cost Attribution】https://arxiv.org/
• 【How on-device infrastructure is emerging as a platform for enterprise AI development and deployment – Apple】https://www.apple.com/

【#Zoho #EnterpriseAI #AICostOptimization #PrivacyProtection #ZiaLLM #GenerativeAI #ITStrategy】