Local routing · cache · SmartflowLM · frontier

The cheap lane is yours.
Frontier stays the teacher.

Every call through the gateway is a labeled example. Cache takes repeats. A local model (SmartflowLM) takes the work it has proven it can do, on your GPU. Frontier handles novel and high-stakes turns — and those answers train the next local version. Same policy and same audit on every lane. No judge LLM in the middle.

Cache first SmartflowLM on your box Frontier teaches Policy does not change

What you get

Cache

Exact and near-repeat, $0, still logged

Local

Proven task-classes on your GPU

Frontier

Novel, hard, high-stakes — the teacher

0 judges

No extra LLM to pick the lane

The loop

Three lanes. Then the log trains the cheap one.

the loop

Policy and identity run first. Then:

request │ ▼ guardrails PII · allowlist · Shield on tools │ ▼ MetaCache exact, then near-repeat → hit: $0, still logged │ miss ▼ fork policy has this task-class been proven? ├─ yes → SmartflowLM (LoRA on Qwen / DeepSeek / Llama, your VPC) ├─ no → frontier teacher (Claude / GPT / …) └─ teacher down or slow → SmartflowLM as last hop, not a 502 │ ▼ VAS prompt, answer, quality, corrections, identity, policy │ ▼ scheduled LoRA → promotion gate per class → fork table

A class of work (FNOL classify, ACORD extract, coverage letter) only moves local after a held-out replay beats the teacher. Coverage letters can go local while bind questions stay on frontier. Adverse / producer-license paths can stay pinned to frontier and a human hold. The loop does not override policy.

auto is the scorer until a class has a measured win-rate: length, intent, stage — no judge model. Then the fork table wins. Header X-Smartflow-LCR still pins a single call to local or frontier. Streams skip scoring.

Three modes

auto — fork table if the class is proven; otherwise score the prompt
local — always SmartflowLM / the cheap lane
frontier — always the teacher

A request header beats a policy. A policy beats the gateway default. Set off if a key or a team should skip routing.

Quality retry

Cheap lane fails (5xx, empty, refusal) → one retry on the teacher. Teacher down or slow → SmartflowLM as last hop, not a 502. Halo Route already fails over on 502 / 503 / 504 / 529. You don't eat a failed cheap call as the final answer.

agent / app │ ▼ Smartflow cache → fork / auto → local or frontier │ ├─ cache hit → $0, still logged ├─ class proven → SmartflowLM on your GPU ├─ class not proven → frontier teacher └─ teacher down/slow → SmartflowLM last hop

Smartflow (gateway)

Hosted gateway

cluster1.aperion.ai

Same path on hosted as on-prem. Cache first. Proven work goes to the Efficient pool (SmartflowLM / Ollama / vLLM on the key, or a same-provider mini until that class is proven). Novel and high-stakes stay on the teacher. Streamed calls skip scoring so playground SSE still works. Pin a call to frontier if you don't want the cheap lane:

curl https://cluster1.aperion.ai/v1/chat/completions \
  -H "Authorization: Bearer $SMARTFLOW_KEY" \
  -H "X-Smartflow-LCR: frontier" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"hi"}]}'

Your own cluster or on-prem box

Same binary. Point the cheap lane at SmartflowLM on the box — Ollama, vLLM, a GPU in the VPC. Provider mini is a stand-in until that class is proven. Restart the proxy.

SMARTFLOW_LCR=auto
SMARTFLOW_LCR_THRESHOLD=0.5
SMARTFLOW_LCR_EFFICIENT_PROVIDER=ollama    # or vllm / openai / openrouter
SMARTFLOW_LCR_EFFICIENT_MODEL=smartflow-lm:vN
# leave CAPABLE unset to keep the caller's model as the teacher

Kubernetes:

kubectl -n smartflow set env deployment/smartflow-proxy -c proxy \
  SMARTFLOW_LCR=auto \
  SMARTFLOW_LCR_EFFICIENT_PROVIDER=ollama \
  SMARTFLOW_LCR_EFFICIENT_MODEL=smartflow-lm:vN

Docker / compose: add the same keys to the proxy service environment. OpenRouter-edition installs already have a local GPU — that box is the specialist lane, not only embed intercept.

Per team, per key, per request

A policy attachment can set the mode and the two models. Same scoping you already use for model rewrite (directory group, virtual key, team). The request header still wins, so a debugger can force frontier without changing policy. Pin a class to frontier (bind, adverse, producer license) the same way.

{
  "lcr_mode": "auto",
  "effort_threshold": 0.5,
  "efficient_model": "smartflow-lm:vN",
  "capable_model": "gpt-4o"
}

Headers: X-Smartflow-LCR: auto|local|frontier. Open the trace on that turn — Request → Routing shows lcr:efficient, lcr:capable, or lcr:skipped stream. FinOps has the Efficient share from the same stamp.

Halo (home / laptop agents)

Install

curl -fsSL https://halo-get.aperion.ai | sh

Route license

halo license apply
halo license show

Cheap agent

halo agent add glm --provider openai
halo serve

Route turns it on. Name the cheap agent glm.

$100 / mo

Free and Cut never hop. Route scores the prompt and, when the turn looks easy, uses the glm agent's provider and key. Hard turns stay on the inbound agent (the one whose virtual key the runtime holds). Spend still counts against that inbound cap.

If first-run config has no routing.effort block, the binary is older than this feature. Re-run curl -fsSL https://halo-get.aperion.ai | sh and restart halo serve.

First run already writes this. If you named the cheap agent something else, change efficient_agent. If glm isn't registered, nothing hops — the call stays on the original agent.

routing:
  effort:
    mode: auto
    threshold: 0.5
    efficient_agent: glm
    efficient_model: glm-4.7
    capable_agent: researcher   # optional; omit to keep the inbound agent

Force a lane from the runtime: X-Halo-LCR: auto|local|frontier. That header wins over YAML. Restart halo serve after you edit config or apply a license.

OpenClaw: halo openclaw apply. Hermes: halo hermes apply. Same as the rest of Halo — don't hand-edit three runtime files if you can avoid it.

Modes at a glance

Mode	Smartflow	Halo Route
`auto`	Proven class → SmartflowLM. Otherwise score the prompt. Easy → cheap lane, hard → teacher.	Score the prompt. Default. Easy → `glm` agent, hard → inbound agent.
`local`	Always SmartflowLM / the cheap provider you configured.	Always the `efficient_agent`.
`frontier`	Always the teacher (request model, or `capable_model` if set).	Always the inbound / `capable_agent`.
`off`	Leave routing alone. Use this to pin a key or a class.	Free and Cut. Route with `mode: off` if you want YAML but no hops.

What this is not

It does

Cache repeats. Train the cheap lane from governed exhaust
Promote a task-class to SmartflowLM only after it beats the teacher
Keep novel and high-stakes work on frontier
Fail over to local when the teacher is down
Let a header or a policy pin a team, a class, or a single call
Stamp the trace so you can see which lane ran

It does not

Call a judge LLM. There's no second bill for the decision
Skip policy or audit on local answers
Use OpenAI raw chain-of-thought as training fuel
Move Halo keys off the machine
Hop on Free or Cut Halo

Quick checks

You want…	Do this
See it on hosted Smartflow	Repeat a short ask — cache hit, still logged. A proven class lands on SmartflowLM. A novel or high-stakes turn stays on the teacher. Trace Request → Routing shows `lcr:efficient` or `lcr:capable`. Streamed playground chat stays on the model you asked for (`lcr:skipped stream`).
Never downgrade this call	`X-Smartflow-LCR: frontier` or `X-Halo-LCR: frontier`.
Always use the GPU box	Smartflow: `SMARTFLOW_LCR=local` and the Ollama/vLLM provider. Halo: `mode: local`.
Turn it off for one tenant	Policy attachment `lcr_mode: "off"` on that key or group.
Halo hops not firing	`halo license show` should read Route. `halo agent` list should include `glm` (or whatever you set). Restart `halo serve`.

The cheap lane is yours.Frontier stays the teacher.

It does

It does not

The cheap lane is yours.
Frontier stays the teacher.