Cloudflare AI Gateway is a hosted proxy that observes and controls AI application traffic on Cloudflare's network. Smartflow is an AI governance control plane that runs inside the customer's own environment. Both sit in the request path and both enforce policy inline. The difference is whose infrastructure the request passes through, and what the resulting record is designed to prove.
The short version
Cloudflare AI Gateway is free for its core features, takes minutes to adopt, and does more than it is usually given credit for — guardrails and data loss prevention run inline with a choice of flag or block, and identity-aware access arrived in August 2026. Where it does not go is on-premises. Cloudflare documents no self-hosted deployment of AI Gateway, and for organizations whose AI traffic cannot transit a third-party network, that is the end of the evaluation rather than a trade to negotiate.
Feature Comparison
| Capability | Smartflow | Cloudflare AI Gateway |
|---|---|---|
| Deployment | On-premises, Kubernetes-native, air-gapped capable. | Hosted proxy on Cloudflare's network. Requests route through a Cloudflare-operated endpoint, a custom domain fronting it, or a Workers binding. |
| Self-hosted option | Yes. The primary deployment model. | No self-hosted, on-premises or air-gapped deployment is documented. |
| Data path | Traffic stays inside the customer perimeter. | Traffic transits Cloudflare's network. |
| Caching | MetaCache resolves a lookup in four phases, first hit wins: intent signature, intent near-miss, exact key, then a VectorLite BERT semantic match. Hit rates are workload-dependent. | Exact-match only. The cache key is a SHA-256 hash of provider, endpoint, model, provider auth header and the full request body, so any difference in messages, tools or parameters creates a separate entry. Cloudflare documents semantic caching as planned, not shipped. |
| Guardrails | Visual policy editor. PII, topic restriction, jailbreak detection, output moderation. | Inline on prompts and responses, per category, with a choice of flag or block. Billed as Workers AI token-based inference. |
| Data loss prevention | Inline content inspection with policy-versioned decisions. | Inline PII, financial and sensitive-data scanning with flag or block. Free on all plans. |
| Identity | Entra ID, LDAP, SAML, OIDC. Per-user audit trails. AIDA cryptographic agent credentials. | Account-scoped API tokens, which cannot be restricted to an individual gateway. Cloudflare Access can front a custom domain and place the verified user ID in metadata. Service-token requests carry no user identity. |
| MCP and agent governance | MCP JSON-RPC gateway with tool caching. A2A orchestration under agent identity. | Not documented as an AI Gateway feature. MCP capability exists elsewhere in Cloudflare's portfolio as separate products. |
| Regulatory evidence | Examination packages and model inventory mapped to FINRA 3110, HIPAA, SOX and the EU AI Act. | Operational telemetry: prompts, responses, tokens, cost, duration, DLP action and matched policy IDs. Regulatory framework mapping is not a documented feature. |
| Log retention | Customer-controlled. No vendor-imposed ceiling. | Per-gateway and plan-capped. When the cap is reached new logs stop saving unless auto-delete-oldest is enabled. |
The caching row on this page previously read "basic caching with TTL controls," which was vague rather than wrong. Cloudflare documents the matching strategy precisely, so the row now states it.
The edge is the architecture, and that is the trade
Cloudflare's design is coherent. Putting the AI gateway on the same network that already carries the organization's web traffic means no new infrastructure, no capacity planning, and latency characteristics that are difficult to match from a single region. For a team shipping an AI product quickly, that is a genuine advantage and the free tier removes the last objection.
It is also inseparable from the constraint. The AI gateway is the point at which prompts, responses and the governance record all exist in one place. Locating it on a third-party network means the prompt content, the enforcement decision and the evidence of that decision all transit infrastructure the enterprise does not operate. For most organizations that is fine and the same is true of their CDN. For a bank under examination, a hospital under HIPAA, or a defense program under ITAR, it is not a risk appetite question but a control requirement, and AI sovereignty is the term procurement will use for it.
Exact-match caching, and what it misses
Cloudflare states the cache key explicitly: a SHA-256 hash of provider, endpoint, model, the provider authentication header and the entire request body. Any variation in the body produces a separate cache entry. Cloudflare's documentation also says semantic search for caching is planned.
That is the right thing to know before modelling cost savings. Enterprise AI traffic is repetitive in meaning and rarely repetitive in bytes — the same policy question asked a hundred ways by a hundred people. An exact-match cache catches almost none of it. Where the workload is machine-generated and genuinely identical, exact matching is efficient and free, and there is no reason to pay for more.
Identity, and the agent problem
Cloudflare shipped identity-aware controls in August 2026: Cloudflare Access can front a custom gateway domain, and the verified user identifier lands in request metadata. That is a real improvement and closes the "who made this call" gap for human users.
Two constraints remain in Cloudflare's own documentation. API tokens are account-scoped and cannot be restricted to an individual gateway, so any token with run permissions reaches every gateway in the account. And service-token requests carry no user identity, because a service token does not represent an Access user — which is precisely the path an autonomous agent takes. AIDA exists to give that principal a name of its own.
Where Cloudflare AI Gateway excels
Core features are free, adoption friction is close to zero, and the 2026 release record is fast: spend limits, automatic retries, user insights with anomalous-session detection, unified billing across providers, and a single Workers AI binding. DLP scanning is free on every plan, which is an unusually generous line for a control of that kind. For an organization already on Cloudflare that needs AI traffic management without a regulatory constraint on the data path, it is the fastest path to value in the category.
When to choose Smartflow
- On-premises is a requirement: Cloudflare documents no self-hosted deployment of AI Gateway
- Regulated data paths: financial services, healthcare and defense, where AI traffic cannot transit a third-party network
- Semantic caching: where the repetition in the workload is in meaning rather than in bytes
- Agent identity and MCP governance: neither is a documented AI Gateway feature
- Retention on a regulatory timescale: customer-controlled rather than plan-capped
Common questions
Can Cloudflare AI Gateway run on-premises? No self-hosted or on-premises deployment is documented. Cloudflare does not publish a statement ruling it out, but nothing in the AI Gateway documentation describes one.
Does Cloudflare AI Gateway do semantic caching? Not today. Caching is exact-match on a hash of the full request. Cloudflare's documentation describes semantic caching as a future addition.
Does it enforce guardrails inline? Yes, on both prompts and responses, per category, with flag or block. Data loss prevention runs inline as well and is free on all plans.
Does it support MCP? Not as an AI Gateway feature. Cloudflare ships MCP capability in other products.
Related reading
Enterprise AI gateway guide · AI sovereignty · Semantic caching · AI governance for financial services · All comparisons · Model allowlist and denylist · AI guardrails · Agentic AI · Air-gapped AI deployment
Verified against each vendor’s published documentation in September 2026. Vendor capabilities change. If something here is out of date, tell us and we will correct it.
Cloudflare sources: developers.cloudflare.com/ai-gateway and its get-started, features, features/caching, features/guardrails, configuration/authentication, configuration/cloudflare-access, observability/logging, observability/user-insights and reference/pricing pages, plus the AI Gateway changelog.
Put this in the path of your own agents.
Policy enforced inline between your agents and every model and tool they reach, with a record bound to the human who owns it.
Request a Demo Read the docs