Alibaba Group has implemented an “AI Eligibility Classification” system in its e-commerce customer service operations to bypass LLM (Large Language Model) processing for routine inquiries, achieving over one year of stable operation. This strategic shift was driven by the need to manage system loads during extreme traffic spikes and to optimize the massive computational costs associated with continuous LLM inference.

Optimizing Inquiries via AI Eligibility Classification

Alibaba has established a system on its world-class e-commerce platform, Taobao, that automatically screens customer service inquiries to bypass LLM processing when appropriate. This operational model was refined following a large-scale randomized field experiment conducted between August 15 and August 31, 2024, and has now been fully deployed for over a year. The core of this mechanism is the classification of chats as either AI-eligible or AI-ineligible. Taobao generates up to one million chats per day, supported by approximately 38,000 operators. Every inquiry is initially triaged by a rule-based chatbot; if the issue remains unresolved, it undergoes classification. Only standardized, “AI-eligible” inquiries are handled by LLM-powered agentic AI, while inquiries involving complex emotions or non-standard requests are immediately routed to human operators to prevent the waste of expensive LLM computational resources. Data from the implementation phase indicates that AI-eligible sessions account for less than 10 percent of the total volume, demonstrating a design that strategically limits high-load processing.

Computational Resource Limits and the Reliability Gap

The necessity of a mechanism to bypass LLM processing stems from the technical limitations generative AI faces within commercial infrastructures. In February 2026, Alibaba utilized its conversational AI, Qwen, for a massive promotion during the Chinese Lunar New Year, but the system became overloaded after receiving over 10 million orders in just nine hours. While LLMs enable sophisticated dialogue, their inference requires significantly more time and power compared to traditional systems, leaving unresolved challenges in scalability during peak traffic. Alibaba manages a hyperscale production AI cluster known as ASI (Alibaba Serverless Infrastructure), which oversees 155,410 GPUs. However, physical constraints such as resource fragmentation and CPU shortages are constant issues. Furthermore, users often over-reserve resources to prepare for sudden traffic surges, which historically lowered overall cluster efficiency. To mitigate these infrastructure loads and ensure system reliability, a hybrid operational design that decouples routine tasks from LLMs became indispensable. Please refer to the figure below.

Human-AI Role Division and Its Impact

Managing Emotional Escalation and the Redefined Human Role

One year of operation has revealed that AI failures typically fall into two categories: technical limitations and emotional deterioration. Alibaba’s system includes an “escalation” mechanism where the algorithm detects customer dissatisfaction or frustration and promptly transfers the session to a human. Statistics show that technical escalations (where the issue exceeds AI capabilities) account for 44.1 percent of cases, and service quality is generally maintained by human takeover. However, it was also discovered that if a customer has already reached a state of strong frustration—termed emotional escalation—it is difficult for a human to restore satisfaction even after intervening. Consequently, it is critical for humans to monitor AI movements and intervene proactively before negative emotions accumulate. Over the past year, the human role has been redefined from performing simple answering tasks to acting as a supervisor who oversees difficult AI scenarios and manages exceptional events.

Ripple Effects on Service Quality and Statistical Evidence

Limiting LLM processing to a subset of inquiries has produced significant statistical results. Field experimental data shows that delegating eligible inquiries to AI reduced average chat duration by 16.8 percent, contributing greatly to efficiency. Conversely, customer ratings for sessions completed solely by AI were 0.412 points lower than those handled by humans. This decline is thought to reflect customer reactions to AI-specific communication styles rather than a lack of problem-solving ability. A noteworthy finding is the positive spillover effect: by offloading routine tasks to AI, the quality of human responses to complex, “AI-ineligible” inquiries improved. Customer ratings for human-led chats rose by 0.091 points, as operators were relieved from multitasking burdens and could concentrate on individual difficult cases. Narrowing the scope of LLM application has thus become an optimal solution for highlighting the respective strengths of both humans and AI.

Scientific Infrastructure Management and Future Outlook

Behind the Efficiency of the ASI GPU Cluster

Alibaba’s AI strategy is supported by advanced infrastructure management known as ASI. In this massive cluster housing over 150,000 GPUs, mechanisms to utilize idle resources are constantly active. For example, the SpotGPU framework detects “standby” GPUs not currently used by high-priority jobs and assigns them to low-priority spot jobs, successfully raising GPU allocation rates from 68 percent to 93 percent. To address “fragmentation,” where GPU memory is split in a way that prevents large jobs from running, Alibaba introduced the IPC (Iterative Partitioned Consolidation) algorithm. This algorithm transparently redistributes active jobs, increasing available empty nodes by 20.2 percent. These scientific management practices enable the stable provision of high-load models like LLMs to a vast user base.

Evolution of AI Agent Strategies Toward 2027

Looking ahead, Alibaba is pushing plans to evolve AI from a simple dialogue tool into Agentic AI that can autonomously judge and act. As outlined in the “AI 2027” predictive roadmap, once a self-reinforcing loop begins where AI accelerates its own R&D, the pace of research progress is expected to jump several-fold. This roadmap is being closely analyzed by Japanese consulting firms like Unkaisekkei Inc. to help businesses prepare for the birth of Superhuman AI (ASI). Alibaba is also strengthening external partnerships, such as integrating its AI model “Qwen” into Apple’s “Apple Intelligence” for the Chinese market, aiming to lead the global ecosystem. However, as AI approaches superhuman levels, system alignment and governance are becoming recognized as major management risks. Building on the current success of bypassing routine processing, the focus will shift to implementing highly autonomous agents while balancing infrastructure loads and societal requirements. Please refer to the figure below.

References

• 【Agentic AI and Human-in-the-Loop Interventions: Field Experimental Evidence from Alibaba’s Customer Service Operations】https://arxiv.org/abs/2603.29888

• 【Heterogeneity at Hyperscale: Characterization and Scheduling of Large Production AI Clusters at Alibaba (Operational Systems)】https://www.usenix.org/conference/osdi26/presentation/li-suyi

• 【AI 2027 Summary | Explaining Management Decision Timing – Unkaisekkei Inc.】https://www.unkaisekkei.co.jp/ja/blog/ai-2027-%E8%A6%81%E7%B4%84%E7%B5%8C%E5%96%B6%E8%A6%96%E7%82%B9%E3%81%A7%E8%AA%AD%E3%82%80agi%E3%83%AD%E3%83%BC%E3%83%89%E3%83%9E%E3%83%83%E3%83%97%E3%81%A8%E6%84%8F%E6%80%9D%E6%B1%BA%E5%AE%9A%E3%82%BF%E3%82%A4%E3%83%9F%E3%83%B3%E3%82%B0-om8q

• 【Alibaba Cloud Strengthens AI Capabilities with Innovations for International Customers】https://www.alibabacloud.com/press-room/alibaba-cloud-strengthens-ai-capabilities-with-innovations-for-international-customers

【#Alibaba #GenerativeAI #LLM #CustomerService #GPU #TechnologicalInnovation #ScienceAndTech #AIAgent】