Three different mechanisms, filmed the same day. Cache skips the model. On-the-wire compact folds repeated language into something the model can read. Aggressive crush is the opt-in path that collapses a fat tool result before it hits the provider. None of these is lowest-cost routing. That one picks a cheaper model, it doesn't shrink the prompt.
How the stack is supposed to work is in the cache ecosystem note. This page is the receipts. Every number below came from a run on 3 Sep 2026. Unique prompts without repeats barely compact. We didn't pretend they did.
We sent a unique prompt twice through POST /v1/chat/completions with cache on. First call missed. Second call came back with x-smartflow-cache-hit: 1. The model was not called again. Latency dropped from 1,834 ms to 101 ms on gpt-4o-mini.
The usage object still reports 425 tokens on the hit. That's the size of the stored completion, not a new provider bill. The skip is the header, not a zero in usage.
| Call | Header | Wall clock | Provider |
|---|---|---|---|
| 1 · miss | x-smartflow-cache-hit absent |
1,834 ms | Called. 424 prompt + 1 completion. |
| 2 · same bytes | x-smartflow-cache-hit: 1 |
101 ms | Not called. |
The shortened text is what the provider bills. Duplicate paragraphs become a readable [same as paragraph N] marker. The model reads that English. There is no {REF:…} dictionary and no expand step. Unique prompts that never repeat themselves barely move. The case that actually saves is the one claims teams already write: the same coverage paragraph pasted into the FNOL, the adjuster notes, and the supervisor checklist.
We posted that FNOL to POST /api/metacache/compression/compress. 1,427 characters in, 940 out. The API reported token_savings=122 and a 1.52× ratio, with on_the_wire: true. That's a 34% cut on characters, not a 65% average. Unique text would come in closer to zero.
[same as paragraph 3]. That is the text the model sees.role=tool only aggressive · opt-inDefault on the gateway is still standard. Crush of the newest tool result is SMARTFLOW_COMPRESSION_MODE=aggressive. It never rewrites the frozen prefix (system, user, older tools). FATAL lines stay. Identical consecutive log lines fold; unique timestamps do not.
Token estimates here are chars/4, matching the gateway's tokens_est. Synthetic rows used seed 20260903. The kubectl tables came from a live cluster pull, redacted (no managedFields, env, or secret-shaped keys), then crushed the same way.
| Scenario | Source | Before | After | Saved | Strategy |
|---|---|---|---|---|---|
| GitHub search · OIDC refresh | synthetic · seed 20260903 | 12,794 | 2,093 | 84% | json_sample |
| Nightly claims rollup | synthetic · seed 20260903 | 15,057 | 114 | 99% | log_collapse |
| Filesystem tree · /opt/smartflow | synthetic · seed 20260903 | 2,810 | 1,781 | 37% | toon |
| GitHub issues · northstar-claims | synthetic · seed 20260903 | 7,991 | 1,626 | 80% | json_sample |
Same crush, payloads that look like what an agent actually stuffs into role=tool. CI and CloudWatch keep the FATAL / CrashLoop line. The live pods JSON at 40k tokens was crushed offline and not sent through chat. Too fat for that path.
| Scenario | Source | Before | After | Saved | Strategy |
|---|---|---|---|---|---|
| GitHub Actions npm ci log | MCP-shaped | 21,922 | 175 | 99% | log_collapse |
| CloudWatch Logs Insights (250 events) | MCP-shaped | 12,648 | 846 | 93% | json_sample |
| GitHub code search (100 hits) | MCP-shaped | 13,024 | 2,128 | 84% | json_sample |
| kubectl get pods -A (table) | live cluster, redacted | 1,325 | 800 | 40% | toon |
| kubectl get events (table) | live cluster, redacted | 1,124 | 808 | 28% | toon |
| Analytics SQL result (200 rows) | MCP-shaped | 11,081 | 4,722 | 57% | toon |
| kubectl get pods -A -o json | offline crush only — not sent live | 40,683 | 26,926 | 34% | json_sample |
SMARTFLOW_COMPRESSION_MODE=aggressive on the proxy. Leave it on standard and the newest tool result goes through as-is. Cache and on-the-wire compact still run.
20260903 (nightly claims rollup). Cache and the realistic crush film are from earlier the same day./v1/chat/completions calls. Hit = x-smartflow-cache-hit: 1 plus wall-clock. Model gpt-4o-mini./api/metacache/compression/compress. Savings are the API's token_savings and char lengths, not a tokenizer dump. Response includes on_the_wire: true.role=tool only. Tokens = character length ÷ 4, same as tokens_est in the proxy. Python port of src/tool_payload.rs in scripts/tool_payload_proof/. Reproducible with python3 scripts/tool_payload_proof/proof_table.py --seed 20260903 --check.Support Home · Cache Ecosystem · API Reference · SDK Reference
© 2026 APERION, Inc. · aperion.ai