Event Pipeline and Analytics

Components: Kafka (Confluent 7.5), ClickHouse (24.1), File Archiver (gzip, rotation, retention), Ring Buffer (overflow policy). Events: llm.request/response, dlp.scan/violation, auth.success/failure, budget.threshold/exceeded, provider.health_change, security.sensitive_data/rate_limit, agent.heartbeat, agent.web_traffic, agent.identity_policy_decision, mcp.tool_call, guardrail.violation, schema.violation, webhook.delivery. Topic separation in Kafka: each event class lives on its own topic (agentwatch.llm.request, agentwatch.dlp.violation, agentwatch.auth.failure, agentwatch.rate_limit, …) so a SIEM can subscribe selectively — security teams take auth/DLP/rate-limit streams; FinOps takes usage/budget; SRE takes provider health + agent heartbeat. Filter + aggregate: ClickHouse columnar storage means analytics queries can slice by any metadata dimension (user, team, org, model, provider, route, tenant, agent, custom tag) with sub-second response on tens-of-millions of rows. Materialized views power the dashboards. On-prem-only: Kafka and ClickHouse run inside the customer environment; no event ever leaves the boundary unless an outbound webhook or OTLP exporter is explicitly configured. Management APIs: status, start/stop, test Kafka/ClickHouse, flush, analytics.

---