Skip to content
Docs

What AgentWatch Does NOT Provide

Verified via exhaustive source code search. This section exists for truthful RFP responses.

Capability Notes
Agent Registry No agent-as-artifact concept (versioning, publishing, rollback). Agents are runtime constructs, not stored artifacts.
Multi-Agent Orchestration No agent coordination, scheduling, recursion-depth limits, or agent profile management
Agent Kill-Switch Automation Manual suspend/disable exists; no policy-driven automatic global termination on anomaly
Prompt Registry No centralized prompts with versioning, tagging, or Dev/Staging/Prod environments
Prompt Evaluation Framework No continuous eval against test sets / scoring rubrics
Prompt Drift Analysis No baseline tracking for system-prompt modifications
Human-in-the-Loop Workflows No pause-and-approve flow for Tier-3 tool calls
Prompt Playground No interactive prompt-testing UI with code-snippet generation
Prompt Compression No token-reduction step before forwarding
A/B Testing / Canary / Blue-Green No traffic-splitting framework for model/agent comparison
Model Drift Detection No quality-degradation monitoring (refusal-rate baseline tracking, etc.)
Hallucination / Faithfulness Monitoring No automated faithfulness scoring
RAG Quality Metrics No Context Precision / Recall / Faithfulness scoring per request
Context Window Visualization Token counts captured; no “lost in the middle” inefficiency detection
FOCUS-Compliant Cost Export Cost tracking exists but not FinOps FOCUS schema
4-Tier Consumption Reporting (Org / Team / User / Model) 4-tier budget enforcement ships; the matching report deck is partial
Unit Economics + Forecasting No predictive cost modelling
Cost Anomaly Alerting Threshold-based budget alerts exist; no statistical anomaly detection (e.g., “> 3× normal rate”)
User Feedback System No thumbs-up/down or star rating correlated with trace_id
Throttle-Event Visibility UI Events emit; no dedicated dashboard view of when/why an agent capability was throttled
Topology / Lineage Visualization Knowledge-graph viz exists; no Agent → MCP → Data-Source topology map
Config Drift Monitoring Versioned configs exist; no automated drift alerting against a desired-state baseline
Configuration Validation Tool (vendor-spec) Schema validation exists; no separate “validate config matches vendor spec” CLI
Declarative Policy Engine No OPA/Rego or Google-CEL policy language
Guardrail Versioning + Rollback Configurable + auditable; not formally versioned with rollback
Per-Agent Tool Whitelists Server-level ACLs only; cannot whitelist tools per individual agent identity
Virtual MCP Servers Cannot construct virtual MCP servers by curating tool subsets from base servers
HashiCorp Vault / AWS Secrets Manager / Azure Key Vault / CyberArk Secrets encrypted in DB; no external-vault native integration
SPIFFE/SPIRE Workload Identity No hardware-attestation-based workload identity
NHI Governance (RFC 8705 cert-bound tokens) No non-human-identity registration, permission-intersection model, or cert-bound token enforcement
Okta as NHI Anchor Okta SSO supported for human users; no NHI registration anchored to an Okta-owning human
Acryl DataHub Lineage No native data-lineage integration (Kafka feed enables external reconstruction)
A2A Protocol (advanced) Core A2A shipped in the 2.0.64–2.0.105 window — agent registry, agent-card discovery, JSON-RPC 2.0 dispatch, X-A2A-Key + Bearer JWT auth, tenant scoping (§42.11, §13). Deferred: mTLS workload identity, RFC 8693 token exchange / on-behalf-of, push-notification webhooks, external-agent egress proxy.
LDAP Group-Based Org Routing Schema reserves signalType='group'; only signalType='domain' shipped
Anthropic-SDK Cross-Protocol Translation /v1/messages is currently a native Anthropic-only passthrough endpoint — does not translate to OpenAI / Bedrock / Vertex upstream providers in v2.0.105. Workaround: use the OpenAI SDK with the same upstream (the Anthropic adapter handles OpenAI→Anthropic translation in the other direction). See §3.6.
Bedrock Converse API as Inbound Endpoint The raw AWS Bedrock Converse API wire format is not an inbound endpoint. Bedrock is fully supported as an upstream via OpenAI / Vertex / Azure / Cohere SDKs — switch the app to one of those and AgentWatch routes to Bedrock for you. See §3.6.
Adversarial / Red-Team CI Framework Guardrails catch prompt-injection at runtime; no built-in red-team test harness, OWASP LLM Top 10 / MITRE ATLAS automation
Copyright / Proprietary-Code Mirror Detection Webhook-DLP integration surface available; no first-party similarity-scoring engine
GPU Multitenancy + Quota No GPU memory limit per NHI/app (inference-runtime concern)
Hardware Telemetry (GPU temp / power / PCIe) Not in scope; inference runtime + node-exporter cover this
Confidential Computing (TEE / NVIDIA CC / Intel TDX) Deploy on TEE infrastructure; not a product feature
Model Weight Encryption at Rest (HSM/TPM) Inference-platform responsibility
Micro-segmentation / East-West SDN Kubernetes/SDN layer; AgentWatch consumes mTLS but doesn’t orchestrate the mesh
Multi-Region Failover with 99.99% Uptime SLA Architecturally supported via stateless app tier + HA Postgres + multi-region LB; the SLA is the customer’s deployment contract, not a vendor commitment from the product
Auto-Forward of Test Results to External QA Systems Webhooks can forward; no dedicated test-result reporter
OpenAPI 3.x Import + Developer Portal Try-It-Out OpenAPI generation ships (Swagger UI for gateway routes); OpenAPI 3.x import for cataloguing arbitrary 3rd-party API products is not native
Interactive Code-Snippet Generation (Python / Node / LangChain / LangGraph) OpenAPI spec lets SDKs auto-generate; no first-party multi-language snippet exporter
Concern Recommendation
GPU Management Inference runtimes (vLLM, Triton, TGI)
TEE/Confidential Computing Deploy on TEE infra (AWS Nitro, Azure CC, SGX)
mTLS Service Mesh Agent mTLS supported; mesh via Istio/Linkerd
Model Weight Encryption (HSM/TPM) Inference platform responsibility
Network Micro-segmentation Kubernetes/SDN layer