Third-Party Ingestion

Overview

Smartflow's core value comes from sitting inline — as a proxy or via Halo — in front of a call, so it can gate the call before it happens. That's enforcement, and it's the strongest guarantee Smartflow makes.

But almost every enterprise customer also has agents and workflows that aren't in front of Smartflow yet — a legacy pipeline, a framework already wired up with its own observability, a ticketing system that already owns an incident process. Asking a customer to re-platform all of that before they get any governance value is a real adoption tax. Third-party ingestion is how those systems join the same dataset without it:

All three land in the same place: the VAS log stream, sealed into the tamper-evident audit chain, scanned by the same compliance detector that runs on live-enforced traffic. One dataset, one audit chain, regardless of how the data got there.

OpenTelemetry Ingestion

If an agent, framework, or pipeline already speaks OTLP — which most modern agent frameworks and LLM SDKs do, natively or via an instrumentation library — point its OTel Collector's OTLP/HTTP exporter at Smartflow. No code changes on the agent side.

POST /api/otel/v1/traces

Smartflow parses the standard ExportTraceServiceRequest JSON body and maps the OpenTelemetry GenAI semantic conventions into structured fields — provider, model, token counts, prompt/completion text, and latency:

A span without any gen_ai.* attribute still gets stored — as name + timing only — rather than dropped, but there's nothing for the compliance scan to read on those.

Every span that does carry real prompt/completion text is run through Smartflow's existing compliance detector — the same PII and regulatory pattern matching that runs on live-enforced traffic — so ingestion isn't just a second copy of a trace store; it's a retroactive compliance pass over data Smartflow never had eyes on before.

Enforced vs. Ingested

Ingestion is deliberately retroactive, not preventive — by the time a span arrives at Smartflow, whatever it describes already happened. Nothing about that gets hidden. Every record carries a provenance field:

ServiceNow GRC

Smartflow's ServiceNow integration is a closed loop, not a one-way export. When the compliance detector, quality scoring, or cost-anomaly detection flags something that warrants a human process, Smartflow opens a ServiceNow incident through the Table API — carrying the VAS record, the matched pattern or anomaly, and enough context to triage without switching tools. ServiceNow stays the system of record for the incident lifecycle; Smartflow stays the system of record for what actually happened on the AI traffic itself.

This sits alongside Smartflow's broader GRC connector framework, which speaks the same closed-loop pattern to other GRC platforms (Archer, MetricStream, OneTrust) and generic webhooks — so "we already have a GRC tool" is a config entry, not a re-architecture.

SIEM & Splunk Export

Smartflow ships a pluggable observability exporter framework — every VAS record can be fanned out, in real time, to one or more downstream sinks alongside the primary Redis/Mongo storage. Point it at whatever your SOC already watches:

• Native Splunk HEC exporter NEW — posts straight to a Splunk HTTP Event Collector, no Collector or forwarder required. Promotes a practical set of CIM-aligned common fields (action, app, dest, dvc, object, severity, signature, user, vendor_product, risk_score) to the top level of every event, so a blocked call shows up correctly in Splunk's own dashboards and correlation searches instead of as an opaque JSON blob — the full original payload is still nested underneath.
• Companion Technology Add-on — TA-smartflow-collector ships field extractions, CIM-aligned eventtypes/tags, four starter saved searches, and an overview dashboard. Download the .spl →
• OTel logs & traces exporters — standard OTLP/HTTP, for Splunk Observability Cloud, Microsoft Sentinel, Sumo Logic, Grafana Tempo/Loki, or a self-hosted Collector that fans out further.
• Datadog exporter — direct to Datadog Logs intake.
• File / stdout JSONL exporters — the escape hatch for anything else: point a Universal Forwarder, Filebeat, or Fluent Bit at the output.

Getting Started

Ingesting an existing OTel-instrumented agent

Point the agent's Collector config at Smartflow as an additional OTLP/HTTP exporter:

exporters:
  otlphttp/smartflow:
    endpoint: "https://your-smartflow-host/api/otel"
    headers:
      X-Admin-Key: "${SMARTFLOW_ADMIN_KEY}"

service:
  pipelines:
    traces:
      exporters: [otlphttp/smartflow, # ...your existing exporters]

Or send a payload directly for testing:

curl -X POST https://your-smartflow-host/api/otel/v1/traces \
  -H "X-Admin-Key: $SMARTFLOW_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d @otlp-traces.json

Exporting Smartflow's own traffic to Splunk directly

OBSERVABILITY_EXPORTERS=splunk
SPLUNK_HEC_URL=https://your-splunk-host:8088
SPLUNK_HEC_TOKEN=<your HEC token>
SPLUNK_INDEX=smartflow            # optional
SPLUNK_HEC_INSECURE_SKIP_VERIFY=true # optional — self-signed/internal-CA HEC endpoints

Set on the smartflow (proxy) deployment — that's where the observability dispatcher lives. Install TA-smartflow-collector in Splunk first to get field extractions and the overview dashboard for free.

Exporting to Sentinel, Sumo, or another OTel-speaking SIEM

OBSERVABILITY_EXPORTERS=otel_traces,file
OTEL_TRACES_ENDPOINT=http://your-collector:4318/v1/traces
# your-collector fans out to Sentinel, Sumo, etc.

Endpoints Reference

Observability exporters (Splunk, OTel, Datadog, file, stdout, GCS, Azure Blob, Langfuse) are configured via environment variables on the Smartflow deployment rather than an HTTP API — see the deployment guide for the full OBSERVABILITY_EXPORTERS reference.

Scope & Roadmap

• GenAI semantic conventions only for OTLP ingestion. Framework-specific span shapes that don't follow that convention will map as "bare" spans (name + timing, no compliance signal) until a dedicated adapter is written.
• HTTP/JSON only for OTLP ingestion today — no gRPC receiver yet. Point your Collector's HTTP/JSON exporter at Smartflow; nearly every OTel exporter supports both.
• Shared admin key. Ingestion currently reuses the same admin key as every other Smartflow write route. A narrower, ingest-scoped key is a natural next step before handing a Collector config to a customer's infrastructure team.
• Splunk CIM mapping is practical, not certified. TA-smartflow-collector promotes a useful common-field subset but isn't a Splunk-certified CIM add-on or a drop-in Enterprise Security data-model mapping yet.
• No HEC batching yet. The Splunk exporter posts one event per call; a buffered-batch mode (matching the pattern already used by the GCS/Azure Blob exporters) is the natural next step for very high-throughput deployments.