AgentKeel — Embedded Optimization

What Is AgentKeel?

AgentKeel is AgentWatch’s embedded optimization layer. When enabled on a deployed fleet agent (§7.20), it transparently reduces LLM cost and latency at the edge — before traffic ever leaves for a provider — by combining three techniques:

Technique	Effect
Exact + semantic response cache	Repeated or semantically-equivalent prompts are served from cache
Prompt / token compression	Redundant context is compressed, shrinking tokens sent upstream
CCR offload	Compute/cost-reduction offload for eligible work

On a cache hit the agent serves the response from cache and skips the provider entirely — 100% of the input and output tokens for that request are saved, and the caller sees near-zero latency.

AgentKeel is fail-open (any optimizer error falls back to a normal upstream call) and opt-in:

AGENTWATCH_OPTIMIZE=1

Telemetry: Direct vs. AgentKeel

Each agent reports optimization telemetry to the gateway via its 30-second heartbeat:

Metric	Meaning
Cache hits	Requests served from cache without touching a provider
Tokens saved	Tokens avoided via cache hits + compression
Tokens sent — direct	Tokens that would have been sent with no optimization
Tokens sent — optimized	Tokens actually sent upstream after optimization
Est. cost saved	Modeled dollar savings from the above

The Observe Platform → Optimize surface renders this as a direct-vs-AgentKeel comparison with a per-agent breakdown, so an operator can see exactly which agents benefit most and quantify the savings across the fleet.

API Reference

Method	Endpoint	Description
GET	/api/admin/agent-fleet/optimizer	Fleet-wide AgentKeel telemetry and per-agent optimization breakdown

Step-by-Step: Enable AgentKeel on a Fleet Agent

1. Deploy the fleet agent (§7.20).
2. Set AGENTWATCH_OPTIMIZE=1 in the agent’s environment (Helm value, DaemonSet env, or VM env file).
3. Restart / roll the agent so the optimizer initializes.
4. Drive representative traffic through the agent’s OpenAI-compatible endpoint.
5. Open Observe Platform → Optimize and confirm cache hits, tokens saved, and estimated cost saved are accumulating; compare the direct-vs-optimized columns per agent.

Because AgentKeel is fail-open, enabling it can only reduce cost/latency — a cold cache or an optimizer error simply behaves like a normal upstream call.

---