[Analysis] Cloudflare Unveils “Clef-omni,” an AI That Also Processes Audio and Video

IT

US-based Cloudflare has released “Clef-omni,” a decision-making AI model that can directly accept and evaluate audio and video inputs in addition to text and images. Amid the growing demand for fast and low-cost decision-making models, the company aims to accelerate market competition for agentic AI infrastructure through fully multimodal capabilities that require no preprocessing and significant price revisions.

The Emergence of Clef-omni and Key Points of the Announcement

On October 9, 2026, U.S.-based Cloudflare announced “Clef-omni,” an open-weight decision-making AI model capable of evaluating and processing audio and video in addition to text and images. This model is built with a 30-billion-parameter Mixture of Experts (MoE) architecture (featuring 3 billion active parameters) based on Qwen3-Omni-30B-A3B-Instruct, and its weights have been released on Hugging Face. Its pricing is set at $0.15 per million input tokens. At the same time, the company reduced the price of the previously announced compact model “Clef-flash” by approximately 58%, from $0.09 to $0.038 per million tokens. Furthermore, by migrating the serving infrastructure for the standard model “Clef” to SGLang, Cloudflare roughly doubled its speed, reducing the median response time for a 3,400-token input from 616 milliseconds to 305 milliseconds. The development team’s incredible speed—completing the model’s training and release in just one week from conceiving the idea—has also been drawing significant attention.

Figure 1

The Difference Between Traditional Generative AI and Decision-Making Models

Unlike general-purpose language models (LLMs) like ChatGPT or Claude that generate free-form long-form text, a decision model is an AI model specialized in outputting confidence scores or probabilities for each predefined choice based on input situational data and structured questions. Since it does not incur the overhead associated with text generation and token decoding processes, it enables automated judgment with ultra-low latency and a consistent data format. Because judgment results are returned as JSON or probability values, developers can integrate them directly into workflows such as automated support ticket routing and threat detection without inserting complex regular expression parsing. The Clef series is fully API compatible with TypeSafe’s preceding decision model “Jev”, allowing it to be replaced in existing agent systems without any code changes.

Figure 2

Technical Features and Performance-Cost Optimization

How Architectures That Directly Process Audio and Video Work

In traditional AI workflows, it was necessary to individually build and operate speech recognition models to transcribe audio files into text, as well as preprocessing pipelines to extract frame images from videos. In contrast, Clef-omni directly receives audio data such as wav or mp3 and video data such as mp4 or webm in a single API call, evaluating them within the same context. The model adopts a mechanism known as “two-stage attention routing,” which extracts relevant features from input elements (text, images, audio, and video frames) in a single prefill process and simultaneously calculates probability scores for all options of typed questions. Its processing speed is also exceptionally fast, achieving a median of approximately 130 milliseconds for text-only decisions, around 150 milliseconds for image inputs, and ranging from a few to several hundred milliseconds for audio. Even with a 21-second video clip including audio, it boasts high real-time performance, completing the judgment in about 1.5 seconds.

Price reduction for Clef-flash and infrastructure-layer acceleration

To achieve a significant price reduction for “Clef-flash,” Cloudflare has changed the context window in its hosted service from the conventional 64k tokens to 24k tokens. Analyzing production telemetry data revealed that 99.76% of all requests (with a mere 0.24% exceeding 24k tokens) fell within the 24k token limit, prompting the company to make the wise decision to prioritize cost efficiency without sacrificing usability. Note that the open weights distributed on Hugging Face natively support up to a 256k context, and longer inputs can still be used in self-hosted environments. Meanwhile, the speedup of the standard Clef model was not achieved by modifying model weights, but rather by migrating the inference serving stack to SGLang (PR #42721). It has achieved significant latency reductions across a wide range of input sizes, dropping from 262 milliseconds to 152 milliseconds (1.7x) for an input of approximately 800 tokens, and from 2,721 milliseconds to 1,635 milliseconds (1.7x) for about 16,000 tokens.

Practical Application and Future Outlook

Practical Use Cases in Security and Business Automation

Within Cloudflare, the Clef series is already in production use as an automation infrastructure for a diverse range of products. The company’s threat intelligence team is utilizing it in a system that combines website domain names with rendered screens captured via Browser Rendering to classify the risk levels and categories (such as e-commerce sites and phishing) of suspicious domains. While the acquisition and classification process previously took 4.7 seconds with a conventional general-purpose large model (gpt-oss-120b), using Clef completes the task in 2.2 seconds—less than half the time—while providing more precise category judgments. In addition, multifaceted automation has been achieved, including the automatic detection and closing of spam issues in public GitHub documentation, phishing reviews of plugins in the proprietary CMS “EmDash”, and scans for personally identifiable information (PII) such as government-issued IDs by the Data Loss Prevention (DLP) team. Looking ahead, the company also plans to provide a reinforcement learning fine-tuning infrastructure leveraging AI Gateway and Containers.

The Agent Development Cycle and Future Focus Points

The introduction of Clef-omni further accelerates Cloudflare’s proposed “Agent Development Lifecycle (ADLC)” vision. In realizing a software factory where AI agents autonomously handle everything from code generation to CI/CD pipeline operations, test automation, and incident self-healing, fast, affordable, and robust decision-making models serve as an essential foundational component. In the decision-making model market—with TypeSafe’s Jev acting as a trump card—Stanford and NVIDIA’s CLM-8B, OpenAI’s Decisions API, and Microsoft’s Decision-1 have emerged one after another, evolving the space into an intensely competitive field in a matter of just a few weeks. With its weights fully open-sourced, Clef-omni is expected to become a powerful option for enterprises and developers seeking to avoid vendor lock-in, bringing the practical deployment of multimodal AI agents closer to reality.

Reference Page

[#Cloudflare #ClefOmni #MultimodalAI #DecisionMakingModel #OpenSource #AIAgent]

コメント

Copied title and URL