LiteLLM Lens reads agent traces, groups what went wrong, and points back at the step. That is useful. It is also late. The tool call, the answer, and anything the model sent out the door are already done by the time a finding shows up.
| Smartflow | LiteLLM Lens | |
|---|---|---|
| When it acts | Before the provider call, and again after | After the run |
| What you describe | A policy that can stop the call, plus a written "good run" the review pass checks later | A written "good run" checked against stored traces |
| Where the trace goes for review | The judge is a normal call through Smartflow. Point it at a model you host and the trace stays in your environment. The call is metered and logged. | Trace content goes to the analysis model through the LiteLLM proxy |
| Evidence | A hash of each finding is on the tamper-evident chain. High findings can open a ticket in ServiceNow, Archer, or OneTrust. | Findings stored with the proxy's database, linked back to spans |
| Trace format | OTLP JSON or protobuf, gzip in and out. Parent spans kept, so a run is a tree, not a flat log. | OTLP into ClickHouse, then a worker |
You can write the expected behavior in plain language, sample a slice of traffic, cap the monthly spend, and run it on a schedule. Findings come back grouped, with a quote and a link to the step. You can tell it "this is expected" and it carries that forward. If you already run LiteLLM and you only need a look-back, that is a reasonable add-on.
It is not a control. A bank that has to show a request was stopped, not merely noticed, still needs the decision in front of the model.
Retrospector is our look-back. Five passes hunt for things a quality review is not built to name: sensitive data in the model's reply, a jailbreak stretched across a conversation, someone rephrasing the same prompt until a guardrail slips, volume that doesn't match the person's history. A sixth pass is the expectation review, closer to Lens, with one difference that matters in a regulated shop: the judge call is itself governed.
Their docs describe Lens as something you turn on once agents are in production and you can no longer read every trace. We agree with that sentence. We just don't think it replaces the gate.
Compared against the Lens docs as published at docs.litellm.ai, October 2026. Retrospector · the gateway comparison