Model Coverage, Routing, and Failover in Smartflow: What Enterprise AI Gateway Support Actually Means

The short answer

Smartflow supports any model endpoint that speaks a standard inference interface. There is no fixed integration list, because coverage is handled by an endpoint registry rather than by per-provider code.

That distinction matters when comparing gateways. Most AI gateways publish a model count, because each provider requires a maintained integration in the gateway codebase. A count is a real number for those products. Applied to a registry-based architecture, the same question produces a misleading answer: a low published count reads as narrow coverage when the correct reading is that no count exists to publish.

What the endpoint registry covers

Smartflow registers endpoints in five categories.

Hyperscaler and frontier APIs. OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Vertex AI, called directly or through a regional deployment of the same service.

Self-hosted open-weight models. Llama, Mistral, Mixtral, Falcon, and any OpenAI-compatible inference server. This includes regional and sovereign models where local regulation or procurement requires them.

Internal fine-tunes. Any custom model served over a standard REST or gRPC inference endpoint, including models the enterprise distilled from its own governed traffic.

On-premises and air-gapped deployments. Private cloud, customer datacenter, or fully disconnected environments where no external provider call is permitted.

New endpoint types. The registry accepts new endpoint definitions without gateway code changes, so a provider that launches after deployment does not require a Smartflow release to be usable.

Bidirectional API translation sits underneath this. Applications continue calling one interface. Smartflow resolves the actual provider format at runtime, which is why swapping a provider does not become an application change.

Coverage is not the same as fidelity

A gateway can list a provider and still degrade what passes through it. The questions worth asking any vendor, including this one:

• Does policy enforcement apply on every registered endpoint, or only on the providers with a first-class integration?
• Does the audit record stay identical in structure whether the request was served by a frontier API, a self-hosted open-weight model, or an internal fine-tune?
• Does the audit record survive a failover, so the evidence trail does not break at the moment the incident happens?
• Are tool calls governed on every endpoint, or only where the provider exposes a native tool interface?

In Smartflow the answers are designed to be uniform across the registry. Policy evaluation runs before prompt transmission and after response receipt, on every endpoint. The audit log captures the prompt and response pair, the routing decision, the policy evaluations, and the enforcement events in one structure regardless of which endpoint served the request. Governance follows the traffic across a failover path rather than resetting with it.

Routing

All routing is client-defined. Smartflow does not select models on its own logic, and every routing decision is logged and versioned.

Task-optimized routing classifies incoming requests by type, for example extraction, reasoning, summarization, or compliance review, and then routes according to the matrix the client defines. The effect is a higher-capability model on the work that needs it and a cheaper model on the work that does not, without application changes and without an unlogged decision anywhere in the path.

Failover

Health probes run continuously against every registered endpoint, testing liveness and latency against client-configured thresholds. When an endpoint crosses a failure threshold, traffic reroutes to the designated backup immediately and transparently to the calling application.

Two properties matter more than the mechanism in a regulated environment. The failover event is itself an auditable record with a timestamp and a cause. And the governance trail continues without interruption on the new path, so a provider outage does not produce a window of ungoverned, unlogged traffic.

No-code model switching

Applications call one endpoint. Model identity is resolved at runtime against client policy. Provider changes, model version upgrades, and migration from a hosted API to an on-premises deployment execute as policy changes rather than engineering projects. Every change is logged.

The practical value is not convenience. It is that model deprecation notices, provider pricing changes, and regional availability restrictions stop being application release cycles.

Where cost control sits in the same path

Because every request already passes through one governed path, the cost controls run on the same wire as the policy controls. Smartflow uses a three-layer cache: an in-memory exact-match layer, a semantic layer matched on cosine similarity, and a persistent layer for long-lived structured content. Prompt compression runs against a pattern library, and provider-native caching is injected transparently where the provider supports it.

Observed figures across design-partner deployments over 90-day windows: 20 to 30 percent net reduction in AI spend on mixed workloads, up to 80 percent provider-side cost reduction on cache-eligible traffic, 34 percent of the average payload removed through compression, and a 62 percent semantic cache hit rate by day 90. Results are workload-dependent and should be measured on the customer's own traffic during a scoped evaluation.

Evaluation checklist

Use this against any gateway, not only this one.

1. Ask how a new provider gets added. If the answer is a vendor release, the model list is a dependency.
2. Ask whether policy enforcement is uniform across all registered endpoints.
3. Ask what the audit record looks like during a failover, not during normal operation.
4. Ask who owns the routing decision, the vendor's logic or the client's policy.
5. Ask whether routing policy is versioned and whether prior versions can be reproduced for an examiner.
6. Ask where prompts, responses, and logs physically reside, and whether any of it transits vendor infrastructure.
7. Ask for the cost measurement to be run on your own traffic rather than a reference workload.

Frequently asked questions

How many LLMs does Smartflow support? There is no fixed number. Smartflow registers endpoints rather than maintaining per-provider integrations, so support extends to hyperscaler APIs, self-hosted open-weight models, internal fine-tunes, and on-premises deployments, including providers that launch after the deployment date.

Can Smartflow route between a self-hosted model and a frontier API in the same policy? Yes. Mixed topologies are the common case. Routing, failover, and task selection treat a self-hosted endpoint and a provider API the same way.

Does Smartflow decide which model to use? No. Model selection executes against client-defined policy. The policy is versioned and every decision is logged.

What happens to governance during a provider outage? Traffic reroutes to the designated backup endpoint on a health-probe trigger. Policy enforcement and audit logging continue on the new path, and the failover itself is recorded.

Is Smartflow a routing gateway or a governance layer? Both, in one path. It performs the routing, failover, and caching functions of a gateway, and adds identity-bound policy enforcement on every prompt, response, and tool call, plus regulator-ready evidence, deployed on-premises.

APERION | Smartflow is the on-premises runtime governance layer of the APERION Enterprise AI Trust Fabric. Comparison framework based on publicly documented product capabilities as of July 2026.