Management — AI Insights

Navigate to: Dashboard → AI Insights

What it shows: An AI-powered analytics assistant that answers natural language questions about your organization’s AI usage, costs, and patterns. It queries an indexed vector store built from your usage logs.

Prerequisites:

• A System LLM must be configured (Settings → AI Assistant) — AI Insights uses this model to answer questions.
• At least a few days of usage data should be present for meaningful answers.

Step-by-step: Initialize AI Insights

1. Click AI Insights in the sidebar.
2. If not yet initialized, click Initialize AI Insights.
3. AgentWatch indexes your usage history into a vector store (this runs in the background and may take a few minutes for large datasets).
4. Click Backfill Historical Data to include usage records from before initialization.
5. Monitor Status — it shows Indexing… until complete, then Ready.

Step-by-step: Ask a question

1. Once initialized, type a question in the chat box. Examples:
   • “Which employee used the most tokens last week?”
   • “What is our total spend on Claude models this month?”
   • “Are there any employees approaching their budget limit?”
   • “Which model has the highest error rate?”
2. Press Enter — the assistant responds with an answer and cites the data sources it used.
3. Click a Citation chip to see the underlying usage record.

Step-by-step: Review conversation history

1. Click the Conversations tab.
2. Select a previous conversation to re-read the exchange.
3. Click Delete to remove a conversation from history.

How to test:

1. Send 10 test requests with different employee keys.
2. Initialize or re-index AI Insights.
3. Ask: “How many requests did we make today?” — the answer should match your Observability dashboard.

API endpoints:

Method	Path	Purpose
POST	/api/admin/insights/chat	Send question
GET	/api/admin/insights/conversations	List history
DELETE	/api/admin/insights/conversations/:id	Delete conversation
GET	/api/admin/insights/status	Initialization status
POST	/api/admin/insights/initialize	Initialize
POST	/api/admin/insights/backfill	Backfill history

---