Multi-Provider Configuration

Multiple providers can be added for the same underlying LLM API (e.g. two OpenAI accounts with separate billing). When the same model is available under more than one provider, employees can target a specific one or let AgentWatch choose automatically.

Specifying a Provider in API Calls

Employees have three ways to specify which provider to use for a model:

Format	Example	When to use
Provider prefix (recommended)	"model": "openai-2/gpt-4o"	Explicit provider selection; industry-standard (OpenRouter / LiteLLM style)
Provider suffix	"model": "gpt-4o::openai-2"	Returned by /v1/models when duplicates exist; usable directly
Bare model ID	"model": "gpt-4o"	Let AgentWatch pick automatically by priority

Provider names with spaces are normalized to hyphens ("openai 2" → "openai-2").

The Copy Model ID button in the model catalog outputs provider/model format automatically when a model exists under more than one provider.

Provider Priority Configuration

Set the Priority field on each provider (lower number = preferred). This controls which provider handles requests when a bare model ID is used:

Priority	Provider	Effect
1	openai	Preferred for all shared models
2	openai-2	Used when openai is unavailable or team-pinned

Selection order for bare model IDs:

1. Team-pinned API key — if the tenant’s team has an API key tagged to a specific provider, that provider wins.
2. Provider priority — lowest priority number wins.
3. Insertion order — tie-break when priorities are equal.

Failover and Circuit Breaker

AgentWatch monitors every upstream provider and automatically removes unhealthy ones from the rotation.

Health monitoring states:

State	Meaning
healthy	Responding normally
degraded	Elevated error rate; still used
unhealthy	Auto-disabled after 6 consecutive failures

Circuit breaker states:

           failures ≥ threshold
Closed ──────────────────────► Open (fail-fast, returns 503)
  ▲                                │
  │ successes ≥ threshold          │ after timeout (30s)
  │                                ▼
  └──────────────────────── Half-Open (test recovery)

• Closed — Normal operation. All requests go through.
• Open — Provider circuit is tripped. Requests fail immediately with 503 Service Unavailable without hitting the upstream. Opens after 5 consecutive failures (default).
• Half-Open — After 30 seconds, one test request is allowed. Two successes close the circuit; any failure reopens it.

Failover sequence (bare model ID):

1. Select provider by team-pinned key → priority → insertion order.
2. Attempt request. If the circuit is open, skip immediately to step 3.
3. Try next healthy provider in priority order.
4. If all providers fail or have open circuits, return 503 to the client.

Load balancing methods:

Method	Description
round-robin	Distribute evenly across all healthy providers
least-loaded	Route to the provider with the fewest active requests
weighted	Route by configured weight percentages
random	Random selection among healthy providers

Adding a Second Provider with the Same Models

After adding a new provider and API key, click Refresh Models in the model catalog (or POST /api/models/refresh). The catalog stores separate rows for each provider so both appear with their own copy/enable controls.