On-prem Smartflow · one cloud key

OpenRouter is the hosted bill.
It isn't the control plane.

One key to a hundred models is a good idea. Leaving that key in the app is not. You still need something on your network that caps spend, tokenizes PII, keeps the audit, and keeps embeddings and repeats off that invoice. That's this box: Smartflow in front of OpenRouter, plus a GPU for the cheap work.

One OPENROUTER_API_KEY Local GPU intercept Same proxy compliance Not Halo · not air-gap
1
Cloud key
$0
Local GPU on the bill
Hit
Cache never leaves
Full
PII · kill · audit
Why this exists
01
Teams are consolidating hosted models on one invoice
why
OpenRouter already sits in front of a lot of providers. That's the point of it. If that's your hosted lane, you don't need us to also speak Bedrock, Vertex, and Azure on day one. You need the thing OpenRouter does not do: sit on your network, know who called, apply policy, tokenize PII, keep a log you can export, and kill a runaway before it burns the month.

The other half of the why is the invoice itself. Embeddings, near-repeats, and a lot of code-shaped work do not need to leave the building. Every one of those that hits OpenRouter is money you didn't have to spend. This profile intercepts that lane onto a local GPU and lets cache hits die on the box.
What you actually get
02
The same Smartflow proxy
Identity, virtual keys, policy, PII tokenize, MAESTRO / compliance scan, VAS, Shield, spend caps, kill switch, audit export. None of that is stripped. This is a deploy profile, not a fork.
03
A thinner upstream
One cloud secret. OpenRouter does provider routing, BYOK, ZDR checkboxes, openrouter/auto. We pass their provider / plugins / transforms through. We do not pretend we are the catalogue.
04
What happens to a request
path
Point the app at http://127.0.0.1:7775/v1. Models can be OpenRouter ids (anthropic/claude-sonnet-4.5, openrouter/auto) or a local name (ollama/llama3).
app → Smartflow (identity · policy · PII · cache · kill) │ ├─ cache hit ────────────────────────── stays on the box ├─ ollama/… · vLLM · LM Studio ──────── your GPU ├─ embeddings (default intercept) ───── your GPU ├─ optional code lane ───────────────── your GPU └─ everything else ──────────────────── OpenRouter (one hop back if local 5xx / 429) Spend: OpenRouter usage.cost when they send it. Local GPU is $0 on the report.
Bring-up
One line
curl -fsSL https://docs.aperion.ai/openrouter/install.sh | sh
cd ~/smartflow-openrouter
# put OPENROUTER_API_KEY in .env
docker compose up -d
./smoke.sh
With a local GPU
docker compose --profile local up -d
# then pull an embed model:
# docker compose exec ollama ollama pull nomic-embed-text
Image
langsmartai/smartflow-enterprise:openrouter-v0.1.0
# air-gap, no Hub:
export SMARTFLOW_IMAGE_TAR=/path/to/image.tar.gz
The installer writes compose + env + smoke into ~/smartflow-openrouter. No Smartflow source is downloaded. The image is langsmartai/smartflow-enterprise:openrouter-v0.1.0 — same runtime as cluster1, edition tag. Air-gap gets a docker save tarball. Host Ollama at host.docker.internal:11434 if you don't want the sidecar. Health comes up without an OpenRouter key; cloud calls wait until it's in .env.
Knobs
EnvDefaultMeaning
SMARTFLOW_OPENROUTER_EDITIONoff unless allowlist is edition-shapedForce the profile on
OPENROUTER_API_KEYrequiredThe one cloud secret
OPENROUTER_INTERCEPT_EMBEDtrueEmbeddings stay local
OPENROUTER_INTERCEPT_CODEfalseCode-shaped models stay local
OPENROUTER_INTERCEPT_PROVIDERollamaWhere intercepts go
OPENROUTER_LOCAL_FALLBACKtrueLocal fail → OpenRouter once
How this is different

This profile

  • On-prem Smartflow in front of one hosted bill
  • Your GPU for embed / repeats / optional code
  • Full proxy compliance on every call
  • Upgrade later: turn the flag off, add Azure / Bedrock / Vertex keys, keep the same virtual keys and policies
Smartflow OpenRouter edition is a deploy profile of the on-prem proxy  ·  built by APERION  ·  Halo is the home-agent product  ·  Use the toolbar above to save this page as a PDF.